I was asked by some people to write a status report about the whole PK/AS/LI stuff – sorry guys that it took so much time to write it ;-).
PackageKit
(PackageKit is an abstraction layer for package-management systems, allowing applications to access the package-manager using simple DBus calls)
PackageKit is an incredibly successful project. With the 0.8.x series, it received many performance improvements, and has now the same speed on my computer than the distribution’s native tools. PackageKit is used in almost all major Linux distributions, except for Ubuntu[1]. But even Ubuntu has written some compatibility layer, so most calls to PackageKit will work.
The only major distro where PackageKit is currently not available, seems to be Gentoo (and I am not sure about the shape of the Gentoo PackageKit backend too).
Debian Wheezy includes PackageKit by default, and in Jessy we are going to replace some distribution-specific tools with PackageKit frontends (mostly the old and unmaintained update-notifier and Software-Updater – no worries, we are not going for a Synaptic replacement 😉 (currently this won’t be possible with PK anyway))
Unfortunately, some PackageKit backends are still not adjusted for the 0.8.x API and are only running on 0.7.x. This is bad, since 0.8.x is a huge step forward for PackageKit. But the situation is slowly improving, with the latest OpenSUSE release, the Zypper backend is now available on 0.8.x too.
Being able to run a PackageKit from the 0.8.x series is a requirement for both AppStream and Listaller.
AppStream
(AppStream is a cross-distro effort for building Software-Center-like 
applications. It contains stuff like a screenshot-service, ratings&reviews etc. The most important component is a Xapian database, storing information about all available applications in the distribution’s repositories. The Xapian DB is distro-agnostik, but distributors need to provide data to fill it. AppStream offers an application-centric way to look at a package database)
The AppStream side doesn’t look incredibly great, but the situation is improving. As far as I know, OpenSUSE is shipping AppStream XML to generate the database. Ubuntu ships the desktop-files, and I am working on AppStream support in Debian’s Archive Kit. On the Fedora side, negotiations with the infrastructure-team are still going on. I haven’t heard anything from Mageia and other AppStream participants yet.
Unfortunately, at least for OpenSUSE, the AppStream efforts seem to be stalled, due to people having moved to different tasks. But efforts to add the remaining missing things exist.
On the software side, Apper (KDE PackageKit frontend) has full support for AppStream. Apper just needs to be compiled with some extra flags to make it use the AppStream database.
On the GNOME-side, GNOME-Software is being developed. The tool will make use of the AppStream database, on distributions where it is available.
Also, a Software-Center for Elementary and other GTK+-based desktops is being developed, which is based on AppStream (already quite usable!).
Using the Ubuntu Software Center on not-Ubuntu-based distributions ist still not much fun, but with the AppStream database available and a working PackageKit 0.8.x with a backend which supports parallel transactions, it is possible to use it.
On the infrastructure side: I recently landed some patches in AppStream-Core, which will improve the search function a lot. AppStream-Core contains all tools necessary to generate the AppStream database. It also contains libappstream, a GObject-based library which can be used to access the AppStream database.
Also, we discuss dropping PackageKit’s internal desktop-file-cache in favour of using the AppStream database. If we do that, we will also add software descriptions to the AppStream db, to improve search results and to speed up search for applications. Because we have to deprecate API for that, I expect this change to happen with PackageKit 0.9.x.
As soon as the Freedesktop-Wiki is alive again and my account is re-enabled, I will create compatibility-list, showing which distribution implements what of the PK/AS/LI stuff, especially focusing on components needed for AppStream.
Only a few distributions package AppStream-Core so far. Although it is beta-software, creating packages for it and shipping the required data to generate the AppStream database would be a very huge step forward.
Listaller
(Listaller is a cross-distro 3rd-party software installer, which integrates into 
PackageKit and AppStream. It allows installing 3rd-party applications, which are not part of the distributor’s repositories, using standard tools used also for native-package handling. Everything which uses PackageKit can make use of Listaller packages too. Listaller also allows sandboxing of new applications, and uses an AppDir-like approach for installing software.)
Listaller is currently undergoing it’s last transition before a release with stable API and specifications can be made. Dependency solving will be improved a lot during the current release-cycle, making it less powerful, but working on all distributions instead. (Fedora always had an advantage in dependency-solving, due to RPM providing more package metadata for Listaller to use) This change was delayed due to discussing a possible use of ZeroInstall-feeds to provide missing dependencies with the ZeroInstall team. We did not come to a conclusion about extending the XML, so Listaller will contain an own format to define a dependency, which can reference a ZeroInstall feed. That should be a good solution for everyone.
All these changes will result in IPK1.2, a new version of the IPK spec with small changes in the pkoptions file syntax and huge changes in dependency-handling. The new code is slowly stabilizing in a separate branch, and will soon be merged into master.
The next Listaller release will be the last one of the 0.5.x series, we will start 0.6.x then. KDE currently has support for Listaller through Apper, which is enabled on a few distributions. In GNOME, optional Listaller support is being developed and will be available in one of the upcoming releases.
Currently, to my knowledge, only a few distributions package Listaller. This should improve, so it is easier for application developers to deploy IPK packages.
The upcoming changes in KDE and GNOME to build stable developer platforms will help Listaller a lot in finding matching dependencies, and for stuff which only depends on one software frameworks, installations should be a matter of seconds.
As you can see, lots of things are happening, and there is improvement in all components related to installing and presenting software on Linux machines. However, all these projects have a severe lack of manpower, especially AppStream and Listaller have the lowest number of developers working on the tools (at time, only two active developers). This is the main reason for the slow development. But I am confident that we will have something shipped in the next distribution releases. At least AppStream should be ready then.
[1]: I don’t blame Ubuntu for that – during the time they wrote an own solution, PackageKit did not have all the required features. (This situation has changed now, fortunately.)
NOTE: I might extend this post with feedback from the different distributions, as soon as I get it.
FYI, PackageKit and Apper exist and work fine on Arch Linux. It even shows the important messages which pacman shows on the command line when upgrading.
FYI Canonical is working on a Packagekit backend for USC. Not sure if they’re gonna make it default but they’re at least developing it. So to say Ubuntu is the only one not supporting Packagekit is utter bs.
Oops, I need to correct myself: actually, the Packagekit backend is already in the main Software Center branch (master) for quite some now. So Ubuntu actually ships with USC with a Packagekit option. It’s just not enabled by default.
Well, it is there because Alex and I wrote it 😉 (with Michael Vogt helping a lot by abstracting interfaces)
And no, Canonical currently has no plans for making it default, and the version shipped with Ubuntu is outdated and broken.
I don’t really blame anyone for Ubuntu not using PackageKit – PK had a few issues back then, which made it unusable for something like the Software Center. Unfortunately, it took a some time to add missing features to PK, and Ubuntu created Aptd instead of using PK.
Packagekit/Apper also works perfectly on Parabola GNU/Linux-libre, which is a 100% free fork of Arch.
I am surprised that you mentioned Ubuntu, and not Gentoo, as a distribution that didn’t fully integrate PackageKit. In Gentoo, PackageKit is “masked”, i.e. not available for regular installs.
Added information about Gentoo. To be clear, the Situation on Ubuntu isn’t that bad, because the compat-layer covers most of the API. Although there are some issues related to weird packaging (no PK modules enabled in GSD).
Do you know if Gentoo has a PackageKit backend? If it hasn’t, PK will not be useful at all for it.
As another comment already says, there is no backend for Gentoo. As far as I know, there was a (failed) GSoC project about that, and most of the difficulties are related to the fact that Gentoo has USE flags (things that govern optional software features at build time) that need to be inspected and possibly edited before installing or upgrading any package.
OTOH, during the past GUADEC, when discussing GStreamer plugins, it was suggested in a private conversation that in such situation a dummy backend that only shows the error box would still be better than nothing. The text can be something like this: “The AAA application wants you to install BBB”.
Actually, sabayon and gentoo don’t have a PackageKit backand as I know.
Thanks for the brief descriptions of what your projects are, before your update on their progress. So many blog posts forget that step!
How come this project gets almost NO press? I have to google search appstream and open as many search results as I can and most of the information and updates are years old!
I have been excited about this project since hearing about it how-ever-many-years ago, but it seems like you guys are developing under a rock. No one knows where you are to even talk about you.
Do you have like a % done update status or a % left to do?
Well, everyone who wants to know who we are and easily find us and contact us 😉 That the projects don’t get much press is kinda weird, but Listaller got a lot of attention due to the stuff Ubuntu does, AppStream is improving too and got some press, and PackageKit is a mature project, which doesn’t need that much press anymore.
But indeed, I probably need to blog more about this…
In terms of % done: AppStream is probably 40% done, but that varies depending on the distribution. Listaller is, depending how you look at it, 70% or 50% done. Many important parts are still missing.
If you are interested in progress, I recommend reading this recent blogpost by Richard Hughes too: http://blogs.gnome.org/hughsie/2013/05/29/the-future-of-package-management-in-fedora
Only 40% of Appstream?!
I really don’t understand this!!! I mean, it’s not you, it’s everyone else!!! Why people that do have the technical skills don’t see how important this piece of software is??? This is the kind of software that everyone should get on board and help. And what happen to the OpenSuse Boosters team that was working on the appstream? Why don’t they continue?!
Same thing goes to Listaller… i truly believe this would bring an enormous force to Linux and unite the distros… Why is only one guy *really* working on this? Why don’t this get more attention??
This 2 pieces and an automatic way to migrate a distro to a newer version are the things *I* believe are essential… Basic things!!! And these are things that are a MUST HAVE if Linux really wants to be competitive on the desktop (Canonical seems to see that…).
But… apparently, it’s me that’s wrong. Maybe it’s because I’m not a developer that i see things this way…
I’m sorry for writing all this in a blog post Matthias. 🙁 Has I said, it’s not you and it’s not anything against you. And I do know most developers are working on their free time and for free.
The thing is that I look back and I see I’ve been waiting for this for a few years now!!! Years… can you believe it?! I’ve watched Listaller be announced, *merged* with other projects, get great updates and so on…
And same thing goes to Appstream… When Linspire announced their CRN technology I went “WOW!!! This is it!!!”. Then it went down, then came the promise of other appstores (such as appstream and, since then, I believe I’ve been waiting for this…
It’s very frustrating… If only I had the required technical skills………
Once again… sorry for my outflow. :(. All I’m really saying is: “Can someone stop everything and help Matthias?! His projects are of extreme importance!!!”. Sorry…
I agree. I am doing something about it. I quit my job and am now going into my second year taking Computer Science / Computer Engineering.
Give me another two years… I’ll see if I can help.
I tried PackageKit with Apper today. IMHO, not only it’s far behind Synaptic, it has a nasty bug, which causes a lot of CPU usage for a few minutes.
With Synaptic, I can download packages without installing them, lock packages, maneuver between repositories (without changing /etc/apt/sources.list and reloading) and force a version. As far as I know, none of these is possible with Apper.
Really cool submit, highly informative and professionally written..Good Job