Documentation of the library data model

[bowlofeggs]

I've been told that hawkey is being replaced with dnf and libdnf. The README links to hawkey documentation but does not link to any documentation to dnf or libdnf. Please document the replacement project so your API users know how to use the replacement.

[Conan-Kudo]

As far as I'm aware, the only API that the DNF team is actually supporting right now is the one exposed by the DNF Python module. All other "APIs" have been marked as unstable so far (to my chagrin, as this makes life very hard if I need non-Python bindings...).

[j-mracek]

@bowlofeggs What kind of API do you search for? Python, C, C++?

[bowlofeggs]

@j-mracek I personally use Python (Bodhi), but I think the project should document all APIs that are intended for third party use.

[j-mracek]

@bowlofeggs I believe that there is a small misunderstanding. python-hawkey is replaced by python-dnf. API that is well described (https://dnf.readthedocs.io/en/latest/api.html). API of libdnf is under development and will be also documented.

[QuLogic]

It would be helpful if that link were in the README, because the only link is in the wiki, but the README says that's internal.

[abitrolly]

Tutorials would be very welcome. I need to get package information from go and I can use dnf from command line, but it doesn't provide all the necessary info I need.

[j-mracek]

In our future project libdnf-5/dnf-5 we have a plan to also provide go bindings. The work can be found in dnf-5-devel branch. If you want to get specific information about packages I would recommend to use dnf repoquery.

dnf repoquery dnf --qf "%{version} %{summary}"

will provide version of dnf packages and their summary.

Additional information could be found here 'dnf repoquery dnf --querytags' or 'https://dnf.readthedocs.io/en/latest/api_package.html'.

[abitrolly]

@j-mracek would be nice to actually get all keys with values for one package for an overview. Like this.

$ sudo dnf repoquery --installed --qf "name: %{name}\narch: %{arch}\nversion: %{version}" dnf
name: dnf
arch: noarch
version: 4.2.21

Otherwise the info is very useful. Although my use case - checking for updates - needs multiple queries. First a list of updates sudo dnf repoquery --upgrades --qf "%{name} %{version}". Then fetch their previous version with --installed <long name list>.

I've listed my ideal API for getting update information would be dnf list updates --format csv:"package, oldver, newver" as reported in https://bugzilla.redhat.com/show_bug.cgi?id=1836520

Or maybe something like https://www.nushell.sh/ )

[j-mracek]

With present API it is possible to get everything what you need, but only in python, and C.

[abitrolly]

@j-mracek but it is still two queries to DB to get available package update info (name, oldver, newver).

[dmach]

We're currently working on a new documentation in dnf-5-devel branch that includes references to the previous locations of objects and methods. We'll rather invest in that than in the code that is going to be replaced.

[abitrolly]

@dmach is there a non-legacy overview what the new API is all about?

[dmach]

@abitrolly since all the dnf5 work is currently done by the DNF team members (all Red Hat employees), we're having internal discussions and plans that are quite difficult to share. To be honest, there's only a small portion of the plans that might be interesting for people outside the team, the rest is mainly technical details, finding recurring patters in the objects, and making the API saner and more consistent.

If you want to check how the documentation on the new API looks like, you can generate it using the build instructions from https://github.com/rpm-software-management/libdnf/tree/dnf-5-devel and then run make doc to generate HTML documentation.

We hope we'll have a relatively stable version of the API available in several months. We'll release a package in Fedora and gather feedback from the users. Unstable builds are available in copr already: https://copr.fedorainfracloud.org/coprs/rpmsoftwaremanagement/dnf5-unstable/

[abitrolly]

@dmach I am not anyone's employee, but if you can address this simple user story before going with public API - https://gitlab.com/abitrolly/dnf-go-gui - I would be grateful. Right now there is no way to inspect the impact on the system from specific upcoming dnf update.

[Conan-Kudo]

@abitrolly There was an attempt to have Go bindings for libdnf in the v5 stack, but they're kind of broken and we can't really test or develop them because we don't write stuff in Go. If your interest is in using libdnf from Go, you might want to take a look and see if you can help fix it up.

[lukash]

@abitrolly WRT to Go API in particular @Conan-Kudo said it well, we will need help to make the Go API work well. On API design in general, I know we've had some discussions about this in the past, so if it's been mentioned please link to it... but if you have requirements on the API, could you write it down in form of function definitions with documented arguments and return values, possibly a brief description? We would appreciate it and if it makes sense we will make the API to work for you. Thanks!

[abitrolly]

@Conan-Kudo I don't mind parsing dnf output as long as it gives consistent machine readable output. Go is way better than Python in handling external commands. My interest is only knowing about upcoming updates. I wrote the tools, because there was no such tool, and my tool also doesn't deliver the full info, because the data that dnf gives out is insufficient. If dnf database was in SQLite format, I would read all the data from there.

The information that I could not get, got incompletely, or just didn't have time to write the parser for dnf output strings. Note that this is only about packages that are going to be installed as a result of dnf update.

• name - names of each package that will be updated, installed or removed as a result of update
• old, new version - either normal version without epoch, or "none"
• one line description - what the package is about
• changes - between installed and new version
• if update is critical - what are CVEs, should I wait another few weeks until rebooting
• installed package status - what packages are system installed (blue), what are user installed (yellow) and
  what are their automatic dependencies (white)

I used to describe my journey like here https://gitlab.com/abitrolly/dnf-go-gui/-/blob/master/docs/DNF-API.md then I took an arrow in the knee.

[dmach]

@abitrolly WRT to Go API in particular @Conan-Kudo said it well, we will need help to make the Go API work well. On API design in general, I know we've had some discussions about this in the past, so if it's been mentioned please link to it... but if you have requirements on the API, could you write it down in form of function definitions with documented arguments and return values, possibly a brief description? We would appreciate it and if it makes sense we will make the API to work for you. Thanks!

Yeah, we really need someone familiar with Go to put together several basic unit tests (e.g. instantiate libdnf::Base and call several methods + several assertions) and show us how everything works.

@abitrolly we have planned two dnf5 features that may work for you if we don't manage to bring the Go bindings back to life:

• dnfdaemon-server - new dnf5's DBus interface; but that doesn't work everywhere (chroots and containers frequently do not have a dbus session)
• dnf/microdnf - we're planning json outputs; it's not on top of the priority list, because we need to refactor the existing features first, but we know that this is needed

[abitrolly]

@dmach I used Go just to play with it. Next time I may use Node, so I am not too enthusiastic to face yet another API wrapper (DBus over Go) and deal with its corner cases. I am not surprised that JSON for microdnf is not on top of the priority list, although I would urge you to take a data oriented approach for the 5th try and start with JSON. Why? Because while all these access methods are good, the underlying non-solved problem with dnf API is absence of consistent data model. You will see it if you follow my user story above. By putting JSON first, there are two great possibilities:

1. Data driven tests
2. Web frontend --> Autogenerated OpenAPI spec

[abitrolly]

Having a documented data model will help to speed up things like dnfdragora too.

https://ask.fedoraproject.org/t/dnfdragora-very-slow-fedora33-mate-spin/11847

[Conan-Kudo]

A documented data model would be very nice (especially to improve the quality of service for dnfdragora!), but I don't necessarily want that propagated to microdnf's UI.

[FGasper]

What’s the status of this? It would be awesome to be able to write Perl bindings rather than exec’ing dnf.

[jan-kolarik]

This should be resolved after completing the #213.

[Conan-Kudo]

What’s the status of this? It would be awesome to be able to write Perl bindings rather than exec’ing dnf.

@FGasper We have Perl bindings with libdnf5. Install the perl-libdnf5 package. 😉

[FGasper]

@Conan-Kudo Is that on CPAN? Searching libdnf turns up nothing.

[Conan-Kudo]

It cannot be on CPAN, it's built from the dnf5 source using SWIG.

[ppisar]

I recommend reading the C++ header files. All the other language bindings are generated by a machine (Swig) from them. There are no substantial differences.

[abitrolly]

@jan-kolarik while textual descriptions and tests are better than nothing, for the data model I prefer to have dnf5 ERD diagram at hand.