If I look back at the last post I made on ths blog… let’s say quite a lot of time has passed. The reason? Well, first of all, one would call it a lack of motivation¹, and afterwards, the emergence of a small yet quite annoying pathogen which caused a bit of a stir worldwide. But today I’m not going to talk about viruses: perhaps some other time when you can avoid hear about it for breakfast, lunch, and dinner.

KDE software from git and the OBS

Yes, today I’m going to talk about the OBS, that is the Open Build Service, not to be confused with another highly successful open source project.

As you know, since ages, the openSUSE KDE team provides a series of repositories which track the latest state of the git repositories in KDE, be them either Frameworks, Plasma, or the applications part of the Release Service. This also allows to create Live CDs which can be useful for testing out the software.

But the question that I’ve seen every now and then is… how it is actually done? Is everything provided by the OBS, or does someone need to add some glue on top of that?

Using the OBS to provide updates to KDE packages from git

Source services

From the official documentation:

Source Services are tools to validate, generate or modify sources in a trustable way.

Ok, that doesn’t tell us much, does it? Luckily, the concept is simple. It is that, for packages built in the OBS, we can use tools (internal or external) to perform operations on the source(s) the packages are built from. These are done before any actual building starts.

The openSUSE wiki has some examples of what a source service can do, of which one immediately jumps to our eye:

Checkout service - checks out from SVC system (svn, git, …) and creates a tarball.

That means that we can, theoretically, ask the OBS to do a checkout from git, and automatically generate a tarball from there. In addition, a source service can add version information to the RPM spec file - the blueprint of an openSUSE package - including some data taken off git, for example the date and the revision hash of the checkout.

Source services are implemented as files named _service which live in the OBS for each package. Let’s take a look at the one for kcoreaddons in the KDE:Unstable::

<services>
  <service name="obs_scm">
    <param name="url">https://invent.kde.org/frameworks/kcoreaddons.git</param>
    <param name="scm">git</param>
    <param name="versionformat">VERSIONgit.%ci~%h</param>
  </service>
 <service name="set_version_kf5" mode="buildtime"/>
  <service mode="buildtime" name="tar" />
  <service mode="buildtime" name="recompress">
    <param name="file">*.tar</param>
    <param name="compression">xz</param>
  </service>
  <service mode="buildtime" name="set_version" />
</services>

obs_scm

The first line inside service tells us that we’re using the obs_scm service, which is a built-in service in openSUSE’s OBS instance to checkout the sources from the url, using git. The versionformat parameter sets the version to a literal VERSION (more on that later) and adds the git suffix, and then adds %ci, which is replaced by the checkout date (example: 20210102T122329) and %h, which puts the short git revision hash in the version (example: 5d069715).

The whole data is compressed in a cpio archive with the obscpio extension. At this point, we don’t have a tarball yet.

After the checkout

THe next services run when the package is actually built, as you can see by the mode="builtime" in their definitions. The first one (set_version_kf5) is actually a service made by Fabian Vogt (fellow member of the KDE team) which replaces VERSION by the current version present in the KDE git (done manually: it is read from the OBS project configuration, and bumped every time it happens upstream). The following lines set the version in the spec file, and compress the whole thing into a tarball.

At this point, the build system starts building the actual source and creating the package.

Manual labor

Just when are these kind of source services run? If left by themselves, never. You need to actually signal the OBS (trigger, in OBS-speak) to run the service. An example is with the osc command line utility:

osc service remoterun KDE:Unstable:Frameworks kcoreaddons

Or there’s a button in the OBS web interface which can allow you to do just that:

There’s just a little problem. When there are more than 200 packages to handle, updating in this way gets a complicated. Also, while the OBS is smart enough to avoid updating a package that has not changed, it has no way to tell you before the service is actually run.

Automation

Luckily, the OBS has features which, used with some external tools, can solve the problem in a hopefully effective way.

Since 2013 the OBS can use authentication tokens to run source services, and you can for example trigger updates with pushes to GitHub repositories. To perform this task for the KDE:Unstable repositories, I based upon these resources to build something to do mass updates in a reliable way.

First of all, I generated a token, and I wanted to make sure that it could only trigger updates. Nothing more, nothing less. This was fairly easy with osc:

osc token --create -o runservice

I did not specify a project, so the token works with all the repositories I have access to. This was a requirement, because the KDE Unstable repositories are actually three different OBS projects.

To trigger an update in the OBS, one needs to call the https://api.opensuse.org/trigger/runservice endpoint, doing a POST request and include the project name (project parameter) and the package name (package parameter). An example (I know, I didn’t encode the URL for simplicity, bear with me):

https://api.opensuse.org/trigger/runservice?project=KDE:Unstable:Frameworks&package=kcoreaddons

The token needs to be passed as an Authorization header, in the form Token <your token here>. You get a 200 response if the trigger is successful, and 404 in other cases (including an incorrect token).

Like this, I was able to find a way to reliably trigger the updates. In fact, it would be probably easy to conjure a script to do that. But I wanted to do something more.

A step further

I wanted to actually make sure to trigger the update only if there were any new commits. But at the same time I did not want to have a full checkout of the KDE git repositories: that would have been a massive waste of space.

Enter git ls-remote, which can give you the latest revision of a repository for all its branches (and tags), without having to clone it. For example:

git ls-remote --heads https://invent.kde.org/pim/kitinerary.git/
22175dc433dad1b1411224d80d77f0f655219122        refs/heads/Applications/18.08
5a0a80e42eee138bda5855606cbdd998fffce6c0        refs/heads/Applications/18.12
2ca039e6d4a35f3ab00af9518f489e00ffb09638        refs/heads/Applications/19.04
2f96d829f28e85f3abe486f601007faa2cb8cde5        refs/heads/Applications/19.08
f12f2cb73e3229a9a9dafb347a2f5fe9bd6c7975        refs/heads/master
18f675d888dd467ebaeaafc3f7d26b639a97d142        refs/heads/release/19.12
90ba79572e428dd150183ba1eea23d3f95040469        refs/heads/release/20.04
763832e4f1ae1a3162670a93973e58259362a7ba        refs/heads/release/20.08
c16930a0b70f5735899826a66ad6746ffb665bce        refs/heads/release/20.12

In this case I limited the list to branches to branches (--heads). As you can see, the latest revision in master for kitinerary at the time of writing is f12f2cb73e3229a9a9dafb347a2f5fe9bd6c7975. So, the idea I had in mind was:

1. Get the state of the master branch in all repositories part of the KDE Unstable hierarchy;
2. Save the latest revision on disk;
3. On the following check (24 hours later) compare the revisions between what’s stored on disk and the remote;
4. If the revisions differ, trigger an update;
5. Save the new revision to disk;

This was slightly complicated by the fact that package names on the OBS and source repositories in KDE can differ (example: plasma-workspace is plasma5-workspace or akonadi is akonadi-server). My over-engineered idea was to create a JSON mapping of the whole thing (excerpt):

{
    "KDE:Unstable:Frameworks": [
        {
            "kde": "frameworks/attica",
            "obs": "attica-qt5",
            "branch": "master"
        },
        {
            "kde": "frameworks/kdav",
            "obs": "kdav",
            "branch": "master"
        }
    ]
}

(Full details on the actual repository)

It was painful to set up the first time, but it paid off afterwards. I just needed a few more adjustments:

• Check whether the remote repository actually existed: I used GitLab’s project API to obtain a yes/no answer for each repository, and skip them if they do not exist.
• Commit the changed files to git and push them to the remote repository holding the data.

As I am mostly a Python person, I just used Python to write a yet another over-engineered script to do all the steps outlined above.

Icing on the cake

Mmm… cake. Wait. We’re not talking about food here, just about how the whole idea was put into “production” (include several hundred of quotes around that word). I created a separate user on my server (the same which runs dennogumi.org) with minimal privileges, put the token into a file with 0600 permissions, and set up a cron job which runs the script every day at 20 UTC+1.

There was just one extra step. Historically (but this is not the case anymore nowadays) the OBS would fail to perform a git checkout. This would leave a package in the broken state, and the only way to recover would be to force an update again (if that did not fix it, it would be time to poke the friendly people in #opensuse-buildservice).

Inspired by what a former KDE team member (Raymond “tittiatcoke” Wooninck) did, I wrote a stupid script which checks the packages in broken state (at the moment just for one repository and one architecture: a broken clone would affect all of them equally) and forces an update. It needs to use tokens rather than osc, but I’ll get to that soon, hopefully.

Conclusion

There, you have it. This is how (at the moment) I can handle updating all KDE packages from git in the OBS with (now) minimal effort. Probably it is an imperfect solution, but it does the job well. Shout out to Fabian who mentioned tokens and prompted the idea.

• opensuse
• kde
• obs