PackageKit now parallelized!

With the most recent release of PackageKit, PackageKit 0.8.3 (published last Monday!), all my changes regarding parallelization have been merged into our master branch, which means parallelization features are now available for backends to use! Yay!

The best thing about the new parallelization is that it will have an incredibly high impact on PackageKit’s speed: Frontends are now able to process many resolve-requests at the same time. Also, you will no longer have to wait for PackageKit to finish installing packages before you can continue browsing the list of installed packages again. (This was the main reason to develop this feature in my SoC, as it is crucial for Software-Center-like applications)

Internally, many other things have changed also, which allow backends to do very clever cache-handling, so the cache can stay open for a longer time, so we can avoid the delay while a backend reopens the cache when running many transactions. We now have a “Backend” and multiple “BackendJobs”, which process specific requests, for example Resolve() or InstallPackages() which provides backend authors with a flexible way to handle transactions.

Parallel processing is implemented in a way where it is nearly impossible to get to a situation where everything is waiting and the daemon is dead-locked. (only a bug in PK or the backend could cause this) Avoiding this situation was – of course – very important, as otherwise the package database could be damaged or a database lock could just be never released.

The new changes also require some work of our backend authors. By default, parallel-transactions are disabled for backends which haven’t declared that they support it. Backend authors need to explicitly enable the feature as soon as their backend supports it. We highly recommend to enable parallelization, as PackageKit frontends might start to rely on it and will be much slower without a parallelized PackageKit backend. Also, backend authors will have to port to the new PkBackendJob infrastructure. Most necessary changes have already been made by automatic scripts, but of course it is better if people using the backend and knowing the package-manager would take a look at it.

So, backend authors: We want you! (to fix the backends ;-)) Richard and I have created a porting guide for backends which summarizes all changes required for backends to be PK 0.8.x compatible. (backends/PORTING.txt)

Parallelization is a very invasive change and we have only one release with it enabled, so please help testing to find possible remaining bugs! 🙂

And end-users can be happy about a much faster PackageKit soon, which will also be working as Software-Center engine 😉