GSoC AppStream Final Report

This year I did a Google Summer of Code Project for OpenSUSE as part of their cross-distribution collaboration track. (Again, many thanks for letting me work on this and for doing a cross-distro track!)

So, what did I achieve this summer? (Leaving out all the problems and stuff which didn’t work as I expected :P)

I did work on three components: AppStream, the Software Center and PackageKit.

At the beginning of the SoC, I thought I would be spending most of my time at the Software Center and in seeing Python code. Instead, I put lots of effort into PackageKit and writing C code. The reason for that was that when I started, all operations I performed in the Software Center were very slow, because PackageKit was slow. Also, some things weren’t possible, e.g. I couldn’t fetch details about an application from PackageKit while an installation was going on, because the installation blocked all other requests on PackageKit. So, I first focused on PackageKit.

PackageKit

I implemented various small speed optimizations, which made PackageKit a little bit faster. (on Ubuntu, it’s faster than Aptd now, but that’s not a fair comparison) I also added a package sqlcache, which can be used to access package data very fast, if enabled.

The biggest thing I implemented was support for parallel transactions. Parallel transactions allow PackageKit backends to run some transactions in parallel, if they support this feature. This means that you can now query package details or search the package database using PackageKit while installing or removing packages at the same time.

It also enabled frontends to query more data faster, which will speed up every PackageKit client if the PackageKit backend supports parallelization. At time, the Zif backend fully supports this feature and the Aptcc backend supports it partially.

Together with Richard Hughes I refactored the backend API, so backends now have some more options to optimize they way they handle cache openings and jobs. (most of the credits for these API changes go to Richard)

Software Center

The Software Center is a fork of the original Ubuntu Software Center, because it was not possible to have Canonical drop the CLA they apply on Software Center’s code. (and which is a problem for new contributors, especially those employed by companies)

At the Software Center side I did lots of bugfixing and speed improvements, for example the SC now starts really fast (if the PackageKit backend is fast too) and data fetching is super-fast too. I also removed some problems preventing the SC to work properly on non-Ubuntu distributions and ported the code to our newly created APIs.

I am still not happy with the state of the Software Center, as there are many unsolved bugs and the tool is not yet user-friendly. But I also think these problems will vanish when distributions start to ship AppStream data to fill the Software Center database and when PackageKit backends are improved.

Here are some screenshots of the PackageKit-based Software Center running on Debian and Fedora:

 

You can already test the code if you have at least PackageKit 0.8.4 installed. Unfortunately distributions don’t provide all AppStream data (the information which matches package names with applications and contains icons & stuff) yet, so using it is quite difficult at time.

AppStream

For the AppStream project itself I created infrastructure to create & maintain the AppStream Xapian database. This database is used by application managers to query data about applications. It also makes searching applications really easy. At the beginning, all code which created this database was hard-wired in the original Ubuntu Software Center. So, if you wanted AppStream data, you had to install the USC. Now, I wrote a nice tool in C++ and Vala, which will build the database from various sources and which allows querying the database using a very simple interface.

These changes will allow alternative Software Center implementations to use AppStream and it will make some other interesting development possible (e.g. application support in Apper, which I implemented after my SoC project was completed, see this post for details.)

There are already some new software centers in progress, for example the Light Software Center, which was designed for use with PackageKit from the beginning.

At time I’m running a request at Freedesktop for space to present the new project & APIs, as well as to upload release tarballs, so this code can hit distribution repositories soon. 🙂

Project conclusion

It bothers me a little that I cannot present you a 100% bug-free end-user-usable Software Center with the end of my project. But instead, I killed all the big technical problems which made implementing a Software Center on other distributions impossible before.

I also did lots of changes which will make using PackageKit even more pleasant to use now and I created Software Center infrastructure, which will allow many projects to implement AppStream features. For example, GNOME developers can get their application-centric software managers, and KDE people will receive an AppStream-ready version of Apper soon.

So in the end, this project was very successful and I’m really happy with the results. Also, it feels like the AppStream project itself gained momentum now, and I will of course continue developing stuff around AppStream – stay tuned, there will be more news later 🙂

GSoC conclusion

I can only recommend every student out there to consider applying for the Summer of Code! It has been an absolutely great time, meeting new people and discussing stuff with them has always been fun for me, but also the feeling of being able to really move a project forward and taking the time to do that is awesome!

During my SoC project, I really had some problems to find a balance between university stuff and GSoC work, because university was giving me a hard time (lots of stuff to learn!). In the end, even a harddrive crash in the hottest development period couldn’t stop me 😉 (now I have a new drive and I was fortunately able to restore most of the old data)

Working with Vincent and of course Richard (who I’m working with for years now) has been a lot of fun, and also interacting with various people in the community was great, but also sometimes very exhausting – in cross-distro and cross-desktop world you really have to put lots of energy in convincing other people to make certain changes happen. But during my SoC I achieved more things than ever in the area of application-centric software management, so it was totally worth it.

When I look at the projects fellow SoC students completed, there are even more great projects to see! (Just take a look at it!) Some other students I talked to also agree that the experience alone was already worth doing their SoC project.

So, if you are a student and are able to write code or do some other cool stuff in FLOSS projects: Consider applying for GSoC 2013!

So, that’s it for now, I already planned some new blogposts later, so stay tuned!