Listaller Project – An introduction

This article is not directly KDE-related, so you might want to skip it. But if you’re interested in a cool project simplifying the way software is distributed on Linux, read on! 😀

At time, if you develop a new application “Foo”, you as an upstream developer let other experienced people distribute your application in Linux-Distribution X through a package. Packaging an application is not as easy as some people think, a package of high quality requires some work. And the package wants to be maintained. The upstream -> downstream way we have at time works very nice, although there are some problems:

  • It takes time for downstream people (packagers) to package a new bugfix release / a new major upstream release
  • If a distribution is frozen (=stable), new upstream major releases aren’t sent to the archives anymore, so users will use an outdated version of an application
  • Some stuff is just not packaged, think of proprietary software as an example. But also free software needs someone to step in an package the stuff.
  • There are thousands of Linux-distributions. An application needs to be packaged for every single distribution (in the worst case)

To make new software available for users of stable distributions and to offer a possibility for everyone to package stuff, Ubuntu invented [Update] The OpenSUSE Build Service offers a similar, but cross-distro alternative and has been published earlier than Ubuntu’s PPAs. [/Update] the so-called PPAs.

PPAs have great advantages:

  • You can install PPAs easily
  • You always get fresh updates using the generic Update-Manager
  • It is nicely integrated with other parts of the system

People who know me might now want to skip the next section – I am a very well-known critic of PPAs 😀 Despite all the great advantages, PPAs also have their issues:

  • Security: If a package contains malware, it will have root-access to your full system. You’re practically giving someone else root-access to your system by installing a PPA.
  • Quality: Packaging is not only taking the upstream tarball and putting it into a DEB package. There are a lot of quality standards to comply with, if you want a high-quality package. The quality guidelines are defined to keep the Distribution working & stable. Some people might not know how to create a sane package and there is a serious risk to damage the system sooner or later with a “bad” package.
  • Upgrades: If you install new versions of a package, old packages provided by your distributor get replaced. And even if you only add new packages, you might break your distribution’s upgrade-path, so an upgrade to a newer distribution-release might fail. (nobody tests with every PPA installed)
  • Ubuntu-Centric: PPAs are Ubuntu-only. There’s no Fedora support and no Debian support too. A cross-distro solution would be better.

Because of these issues, I started Listaller, a new approach of a cross-distro software installer. Listaller is based on two great projects: PackageKit and AppStream. The development started years ago, but all old code has been thrown away with the availability of a stable PackageKit and the AppStream project, which covers some Listaller’s former project goals and will reach them in a much better and more generic way than Listaller could’ve done. Listaller is now rewritten in the Vala programming language.

The project was started to offer upstreams the ability to provide cross-distro packages of their applications. This means Listaller was designed to install applications and not libraries or system-components. So, Listaller was not designed to replace package formats like RPM or DEB or to replace package managers like APT/Yum/Zypper/etc. It’s main goal is to extend these systems to offer cross-distro application installations.

The “package”-format of Listaller is a tarball with the extension “.ipk” (Installation-Package) containing a GPG-signature, a XZ-compressed archive with XML control data and a (also compressed) archive with the payload data. This is very similar to the structure of a Debian archive.

Because Listaller works as a PackageKit plugin, all tools using PackageKit will magically get support for Listaller too. So you can remove Listaller-installed software through Apper on KDE and with GNOME-PackageKit on GNOME. Users won’t notice the difference. Same also applies to updates: Users will only see one updater application. This will avoid having many different UIs which do all the same.

How are dependencies handled? By default, Listaller always tries to use native distribution packages as dependency providers. (It searches them through PackageKit) If this does not work, it can make use of ZeroInstall feeds too. (this feature is under construction at time) Listaller distinguishes between applications, dependencies or resources and frameworks. Applications are packaged in an IPK package, while dependencies are fetched from somewhere else and can be used by many applications. Frameworks are something like D-Bus, PolicyKit, PackageKit, Nepomuk or Akonadi, which relies on being integrated with the system. Listaller will never try to install a framework. Frameworks always have to be provided by the distributor. (This has security reasons too, also Listaller just can’t do this cause of it’s design.)

And what about relocatable applications? Well, that’s a big issue. Binaries on Linux usually have paths to their data hardcoded, so they will always search in e.g. “$prefix/share/foo”. But Listaller puts applications and their data into /opt by default, so binaries need to find data independent where they are. To achieve this, we have code called “BinReloc”. Listaller inherited this from the Autopackage project, which has been merged into Listaller some time ago. BinReloc allows software developers to create relocatable applications by integrating it into their applications. For some people this is not necessary anymore: Qt4 already provides some nice methods to create relocatable apps.

To make installed apps find their libraries, every Listaller application is executed using a tool called “runapp”. This tool will set the LD_LIBRARY_PATH variable and run the applications. So you can easily use Listaller-installed tools in scripts. This also avoids collisions with applications in /usr/bin.

Is it secure? Yes, Listaller is secure. Every package should be signed, if it is not, the installer will reject it by default. (Unless you change the settings :P) Updating software is easily possible, and Listaller applications will never conflict with native distribution applications, so upgrading a distro is safe. Also, the “runapp” tool is able to execute every newly installed application in a sandbox first, so you can test the application. IPK packages don’t have much possibilities to damage the system, because they don’t contain custom scripts. So, yes, Listaller is secure. Only the user could do the wrong thing 😛

Okay, this was a quick, but still far too long information about what keeps me busy. 😛 Listaller has very few developers and to be honest, there was a time where I thought this project would fail. But now I’m sure that working on it is a good activity 😉 At time, there are only GNOME-frontends available for Listaller. This is because the Qt bindings aren’t updated and because of me, who is eager to try Smoke-GObject on this project 🙂

So, thanks for reading! And I promise, the next post will be more KDE-centric again 😛

13 Comments

  • Pierre commented on 1. November 2011 Reply

    “Ubuntu invented the so-called PPAs.”

    Hum… no. Look at the OpenSUSE Build Service, invented more than one year before the PPA, and supporting more than just OpenSUSE…

  • damian commented on 2. November 2011 Reply

    Definetly needed for linux, I hate when I have firefox 2/3 on old installs of linux and updating it means breaking the whole perfectly working distribution.
    How would it manage differences in libs dependances?
    For example If I try to install firefox 7 on opensuse 11.2 but it needs libsomething bigger than the version in opensuse?
    The best would be if it would install the new libraries in /opt or something like that, so they don’t conflict with the system.

    • Ximion commented on 2. November 2011 Reply

      Libs are installed as native packages, if possible. If this does not work (e.g. lib is not in the archive), they’re downloaded from the web and stored in /opt, so we don’t conflict with the main system.

      • damian commented on 2. November 2011 Reply

        Thanks for answering, awesome project by the way,
        Where is help more needed? bugs/documentation or direct development?
        Also, what’s that part made in vala? the gtk front-end? isn’t it enough with package-kit front-end?

        • Ximion commented on 2. November 2011

          Help is required everywhere 😀 We’re only very few developers, where I am the only active dev at time (if you don’t count Richard) At time, improving the PK bridge is the most important task.
          Everything is written in Vala, the library, the plugin, the tools, the GTK+-frontends…
          There are no explicit frontends needed, you could use the PackageKit-tools – but the Listaller frontends expose some of LI’s special-features. Maybe the extra-GUIs get included into GPK and Apper later.
          You can always find me on #PackageKit (Freenode) or reach me via e-mail.

  • CTown commented on 2. November 2011 Reply

    So, Listaller is a wrapper around PackageKit to help PK install IPK packages? So, it’s like giving PK it’s own package format? And since PK is cross-distribution, IPKs are also cross-distribution? Am I close? Anyway, it sounds like a good idea!

  • Govi commented on 2. November 2011 Reply

    If i understood well Listaller is not “wrapper around PackageKit” but PackageKit back-end. You have one backend for rpm/deb/whatever as distro specific and Listaller for IPK packages that is cross distro. Am I right?

    I only don’t get how dependencies are mapped for each distro. E.g. foo could be in one called foo in other foo-lib and or be split into few smaller packages.

    • Ximion commented on 2. November 2011 Reply

      Listaller is an extension of PackageKit. There are native backends for PackageKit, like aptcc, yum, zypper, etc. Listaller is a layer on top of the native backend inside of PK.
      But to make it more complicated: It can also work alone, e.g. if you want to have an application installed into user’s $HOME.

      Listaller dependencies are dependencies on libraries. E.g. if you debend on libfoo with ABI level 5, you tell Listaller to install “libfoo.so.5” – it will then search the native appdb for this library. As an alternative, you can also define native package names, but nobody will do this, I guess. 😛 Every entry should also reference a ZeroInstall feed or library tarball, which is used if the distribution does not provide the library.

  • Bugsbane commented on 2. November 2011 Reply

    Sounds good, and definitely something needed, although there is a touch of http://www.xkcd.org/927/

    ;P

    • Ximion commented on 2. November 2011 Reply

      Aaah, I wanted to add this comic to the blogpost too 😀
      (I’ve got this one in Listaller’s directory :P)
      At time there are ZeroInstall and Glick as competitors for Listaller, where ZI has a slightly different scope than LI – and LI uses ZI-feeds as dependencies, so there can be some kind of synergy I hope. Glick uses a completely different technology, and I think my approach is better – I hope I’m right ^^

  • jospoortvliet commented on 2. November 2011 Reply

    Having a cross-distro package format which does not integrate with the ‘native’ format (eg it doesn’t tell dpkg or rpm databases that a certain thing is installed) seems a bad idea to me, sorry. And why would you do it? Why create a whole separate layer if you can easily create native packages with the OpenBuildService? If your distro is not supported, write a patch. It’s Free Software, and we’d gladly help you.

    It’s better to make OBS create native packages for all distro’s, which then work properly. You solve the problems you mention on top of this blog post without the downsides and without having to create so much new technology.

    Simply add the ability to KPackagekit to search one or more OBS repositories for the packages and you’re done. And use the one-click-install technology from openSUSE to allow developers to put a link on their website which, cross-distro, installs their applications!

    This is, btw, what the Bretzn project has been working on. And what Appstream should have been doing 😀

    • Ximion commented on 2. November 2011 Reply

      Well, native packages are dangerous, as you have to install them with maximum permissions. Also, they’re “packages”, but users want to have “applications” having to install a whole set of packages manually is a pain. This is solved by 1-Click-Install, but then you get the PPA-is-dangerous-and-untested issue. OpenBuildService also requires you to create RPM spec files and Debian control data, which is not really nice. (btw, I love to use the OBS, but it’s not that easy to use for newbies :P)
      And what about proprietary software? I don’t like it that much, but Listaller was designed with this use-case in mind too, so proprietary software developers can target the whole Linux-Desktop-User audience with just one package. Proprietary game developers for example might have problems in using the OBS.
      Having RPM and DPKG talk to Listaller would be great, and having this feature is possible – but I don’t think someone will accept a patch for this until Listaller is used widely. Classical Henn-Egg problem.
      The project has it’s use-cases, that’s why the GNOME-people seem to like Glick very much at time. (I should have made more noise about Listaller too!)
      AppStream is a very cool project, which will make PPAs and other repos more AppCenter-like – I’m currently pushing integration on Debian as DEP-11 suggestion. Listaller can plug into this system for seamless integration – works very well in a test-environment here.
      The ultimate goal is that users won’t notice that Listaller is there – they shouldn’t even know the name of this tool (because it only appears in error-messages sometimes :))
      Of course I’m following the Bretzn project too (I guess since it started…) – it is an interesting approach, but I’m not sure if it will solve all issues.
      Btw: Do you know the state of AppStream in other distributions? “Having something shipped by the end of 2011” might become a problem, if I look at what we have today 😛 (but maybe I don’t see everything ^^)

  • http://.com commented on 30. April 2013 Reply

    Two thumbs way up for wonderful information.
    I must say that I am astonished your page is so low ranked in
    the search engine results. I think it was on page 5..
    .. and pages 1 through 4 had virtually no related information that
    I could use.

Leave a Reply

Your email address will not be published. Required fields are marked *