Software Center progress report

It’s been a very long time since my last blogpost, but you can be sure I haven’t gone lost! 😉 I was very busy with writing exams (and preparing for them) the last weeks, and university was giving me a hard time. Now I still have to write two more exams, but one of them is in August and I won’t have lectures anymore with the end of next week, so I have much more time again to work on my GSoC project on the Software Center.

So, what has happened so far? (If you read the mailinglists, you might want to skip that part)

I looked into the code, updated some very odd parts the SC used to access PackageKit, but there are still many pieces of code left which don’t look good and need optimization. During my work on the SC, I found out that with the current way PackageKit works it would be impossible to implement the Software Center in an user-friendly way. For example not being able to see the details of an application while another one is installing sucks. Also, the round-trips to the daemon slowed the whole thing down.

First I did some optimizations on the code which loads the package cache, so the SC now starts very fast. (It had an incredible startup time of ~30 minutes on my machine before, now it’s down to 4sec and faster (to show the UI, to be ready it takes ~20 more seconds)) I also did a few modifcations on PackageKit, which saved us some msecs to secs (depending on the requested action). To solve the general issues with PackageKit, I wrote a module for PK to create a cache of all packages. This solution resulted in a massive speed gain for the Software Center, but slowed down PackageKit actions a little, for example Refresh(). Also, the cache suffered – naturally – from all problems of a cache, e.g. it was going out of sync extremely fast. (You just needed to use a native package management tool) Additionally it was duplicating or even triplicating package data, and generally disliked by many people, including me in the first place. I first chose to implement the cache, because it was the easiest way to get the needed functionality (parallel access to package-data and fast cache-loading) in-time for my SoC project. But a workaround solution serves nobody in the long run. So I sat down with Richard Hughes, thinking about how we could change PackageKit to serve the needs of a Software Center. He rejected some of my proposals and we had a very extensive discussion about a suggestion by Daniel Nicoletti, but in the end we ended up with a solution which will allow PackageKit to execute software management tasks in parallel, if the backend supports it. The chosen solution is one which requires massive changes on the backend API and the most invasive changes in general, but it’s also a solution which does things right(tm), without any workarounds or more layers to access package information.

Implementing this functionality is not exactly trivial, we broke the PackageKit backend API completely, so all backends will now need massive changes to support the new functions and to even compile again. So, here’s a call for backend authors: Please fix your backends! You can see how the Yum, Zif and Aptcc backends were changed, there’s also very simple documentation of required changes in backends/PORTING.txt (file will be updated soon). The implementation is not complete, I’m working on the missing pieces now, so some changes might still happen.

All these changes mean PackageKit will soon be able to execute several actions in parallel if the backend supports it, for example running GetDetails() on a package while InstallPackages() is running too. We count on backend authors to implement this functionality, otherwise distributions running PackageKit without a backend which supports parallelization will not deliver a good “Software-Center experience”.

At the Software Center side, I’m currently implementing the PackageKit history feature, after that I will have to do some polishing and many ichanges on the code which talks to PackageKit, I guess – there’s lots of room for improvements! Also, there are some Ubuntu-specifics which need to be solved, as well as it is needed to split out the code which generates the AppStream Xapian database, so other alternative Software Center implementations are possible without the need to install the “original” Ubuntu Software Center. (I will work with Vincent and Michael on this)

If you want to try the current SC, just grab the code from Gitorious and try it – but be careful! You’ll need the latest development version of PackageKit (master branch) to make it work, but at the moment the only working backends are Yum, Zif, Aptcc and Dummy. So, Fedora, Debian and Ubuntu users are happy at time, all other distributions will need to update their backends as soon as the dust settled down. (There are also some more new and cool features in the unstable series at time, for example the systemd-support Richard has implemented, and the ofline-upgrade feature.) If your PackageKit is not the latest version, the SC will just crash on startup – I haven’t found out how to check version numbers of imported GObject Introspection data in Python to throw a proper error message. (in general it seems like version-checking is not used very much in Python-world) At the moment, using the SC is still not trivial, but in the end of this project everyone should be able to make use of it. (which is not the case at time, to be honest ;-))

During the next weeks you can expect more code changes again, as the discussion part seems to be over, and I have more time again. (Doing cross-distro projects is 60% talking and 40% coding) I’ll also try to blog more, so everyone can stay informed without having to read mailinglist threads.

In the end, (and I think I should make that my signature) kudos to Richard Hughes for beeing an excellent maintainer and for helping me so much with the most difficult changes on PK. Also, once again thank you OpenSUSE for letting me work on this!