AppStream/DEP-11 fully supported in Debian now!

AppStream on DebianBack in 2011, when the AppStream meeting in Nürnberg had just happened, I published the DEP-11 (Debian Extension Project 11) draft together with Michael Vogt and Julian Andres Klode, as an approach to implement AppStream in Debian.

Back then, the FTPMasters team rejected the suggestion to use the official XML specification, and so the DEP-11 specification was adapted to be based on YAML instead of XML. This wasn’t much of a big deal, since the initial design of DEP-11 was to be a superset of the AppStream specification, so it wasn’t meant to be exactly like AppStream anyway. AppStream back then was only designed for applications (as in “stuff that provides a .desktop file”), but with DEP-11 we aimed for much more: DEP-11 should also describe fonts, drivers, pkg-config files and other metadata, so in the end one would be able to ask the package manager meaningful questions like “is the firmware of device X installed?” or request actions such as “please install me the GIMP”, making it unnecessary to know package names at all, and making packages a mere implementation detail.

Then, GNOME-Software happened and demanded all these features. Back then, I was the de-facto maintainer of the AppStream upstream project already, but didn’t feel like being the maintainer yet, so I only curated the existing specification, without extending it much. The big push forward GNOME-Software created changed that dramatically, and with me taking control of the specification and documenting it properly, the very essence of DEP-11 became AppStream (that was around the AppStream 0.6 release). So today, DEP-11 is mainly a YAML-based version of the AppStream XML specification.

AppStream XML and DEP-11 YAML are implemented by two projects, GLib and Qt libraries exist to access the metadata and AppStream is used by the software centers of GNOME, KDE and Elementary.

Today there are two things to celebrate for me: First of all, there is the release of AppStream 0.9 (that happened last Saturday already), which brings some nice improvements to the API for developers and some micro-optimizations to speed up Xapian database queries. Yay!

The second thing is full DEP-11 support in Debian! This means that you don’t need to copy metadata around manually, or install extra packages: All you need to do is to install the appstream package, everything else is done for you, and the data is kept up to date automatically.

This is made possible by APT 1.1 (thanks to the whole APT team!), some dedicated support for it in AppStream directly, the work of our Sysadmin team at Debian, which set up infrastructure to build the metadata automatically, as well as our FTPMasters team where Joerg helped with the final steps of getting the metadata into the archive.

That AppStream data is now in the archive doesn’t mean we live in a perfect utopia yet – there are still issues to be handled, but all the major work is done now and we can now gradually improve the data generator and tools and squash the remaining bugs.

And another item from the good news department: It’s highly likely that Ubuntu will follow Debian in AppStream/DEP-11 support with the upcoming Xenial release!

But how can I make use of the new metadata?

Just install the appstream package – everything is done for you! Another easy way is to install GNOME-Software, which makes use of the new metadata already. KDE Discover in Debian does not enable support for AppStream yet, this will likely come later.

If you prefer to use the command-line, you can now use commands like

This will simply install the Kate text editor.

Who wants some statistics?

At time the Debian Sid/Unstable suite contains 1714 valid software components. It could be even more if the errors generated during metadata extraction would be resolved. For that, the metadata generator has a nice statistics page, showing the amount of each hint type in the suite and the development of the available software components in Debian and the hint types count over time (this plot feature was just added recently, so we are still a bit low on data). For packagers and interested upstreams, the data extractor creates detailed reports for each package, explaining why data was not included and how to fix the issue (in case something is unclear, please file a bug report and/or get in contact with me).

In summary

Thanks to everyone who helped to make this happen! For me this project means a lot, when writing this blog post I realized that I am basically working on it for almost 5 years (!) now (and the idea is even older). Seeing it to grow to such a huge success in other distributions was a joy, but now Debian can join the game with first-class AppStream support as well, which makes me even happier. Afterall Debian is the distribution I feel most at home.

There is still lots of work to do (and already a few bugs known), but the hardest part of the journey is done – let’s walk into a bright future with AppStream!

9 Comments

  • Charles Plessy commented on 14. December 2015 Reply

    Hi Matthias,

    congratulation for this release !

    In the upstream-metadata project (https://wiki.debian.org/UpstreamMetadata) we document some information, some of which is redundant with what is now found in AppStream, and some of which is not.

    For example, we document bibliographic reference to the scientific litterature where a given sofwtware has been published. This particular data has been the driving need behind the creation of upstream-metadata project. But the moment, I do not have enough free time to take care of the upstream-metadata project properly.

    Given that the scope of AppStream has grown, do you think that it could be further extended to support the kind of information that we distribute in the debian/upstream/metadata files ?

    Have a nice day,


    Charles

  • j425476 commented on 14. December 2015 Reply

    How is AppStream data currently distributed? Do I get it via apt-get update? Or is it contained within a special package?

    How will additional external package repositories (like Ubuntu’s PPAs) work with AppStream?

  • Matthias commented on 14. December 2015 Reply

    On Debian (& Ubuntu) you will get the data via a normal apt update, no special package is involved.
    For PPAs, Canonical could implement support in Launchpad, or the PPAs simply ship their metadata in a .deb package (this is AFAIK what Elementary OS is planning to do).

    Completely external repositories can simply add the metadata to their repo – will probably be a manual task, since tools like reprepro don’t support it yet, but that will become easier over time.

  • j425476 commented on 14. December 2015 Reply

    Cool, thanks.

    So can the tools that process AppStream data on the clients (e.g. Gnome Software) already deal with sources from multiple packages/repositories?

    • Matthias commented on 14. December 2015 Reply

      Yes, that should work without any problems 🙂

  • Diane Trout commented on 24. December 2015 Reply

    Hi,

    Very nice work.

    I was wondering if the Issues page could have anchor paragraph tags so I could easily link to a team’s packages.

  • cinemark discount coupons commented on 17. May 2016 Reply

    If you might be to photograph a family that is not cooperating, add another in order to individual the appearance internet coupons 842/- only while Nokia C2-00 price in Mumbai is Urs waikiki discount coupons Magic Keys and Tumble Books you’ll deal
    with several picture books miami zoo discount
    coupons When I in college I took an Anthropology class dick s sporting goods discount coupons the accomplishments and failures are still part of our history fiction costume discounters Coupon

  • App Development - Pro Lead Soft commented on 25. November 2016 Reply

    This app stream tool is great.

Leave a Reply

Your email address will not be published. Required fields are marked *