Python on dennogumi.org

Making KDE applications Python 3 friendly

Sun, 02 Dec 2012 18:00:57 +0000

When I’m not on forum duty or handling openSUSE-related contributions, I try to improve my code contributions to KDE, even though I’m by no means an experienced programmer (I program all day long in Python, but I’m still a biologist after all). For the upcoming 4.10 release I’ve been polishing up Python 3 support.

As you may know, Python 3 isn’t the standard in many distributions (Arch Linux excluded), but despite the slow start, it is slowly gaining steam. PyKDE4 was built keeping in mind Python 3 support, so theoretically we supported it right from the start.

Some more Nepomuk, please

Sun, 12 Feb 2012 19:57:06 +0000

Recently we’ve seen several blog posts on Planet KDE related to Nepomuk. Reading those I thought that I could add some (little) semantic features to Danbooru Client.

Danbooru Client already makes use of Nepomuk: if enabled, tags extracted from Danbooru items are added as Nepomuk tags. But since at least some Danbooru boards are specialized in certain types of images (e.g., wallpapers only, for examples) I found it would be nice to have Nepomuk show me only the images that come from a specific Danbooru board.

PyKDE4: Queries with Nepomuk

Wed, 29 Jun 2011 19:27:42 +0000

In one of my previous blog posts I dealt with tagging files and resources with Nepomuk. But Nepomuk is not only about storing metadata, it is also about retrieving and _interrogating _data. Normally, this would mean querying the metadata database directly, using queries written in SPARQL. But this is not intuitive, can be inefficient (if you do things the wrong way) and error prone (oops, I messed up a parameter!).

Multiscale bootstrap clustering with Python and R

Sun, 29 May 2011 12:11:40 +0000

While reading the statistics for my blog, I noticed that a number of searches looked for hierarchical clustering with Python, which I covered quite a while ago. Today I’d like to present an updated version which uses more robust techniques.

Taking video snapshots quickly: KDE VLC Snapper

Sun, 10 Apr 2011 12:40:26 +0000

Some of the oldest readers of this blog are well aware of a certain hobby of mine. Over the years I’ve always wanted to write more about that, including the stuff I’m viewing nowadays, but I found a hassle to collect snapshots from videos / DVDs, selecting them, and so on.

Recently I learnt that VLC has some rather complete Python bindings, and I thought, why not make the process automated? Yesterday I had some free time on my hands and a quick session of hacking brought some results already.

PyKDE4: Retrieve data using KIO

Sat, 01 Jan 2011 21:09:26 +0000

One of the greatest strengths of KDE is undoubtedly the asynchronous and network-transparent I/O access, employed by the so-called “I/O” slaves, part of the KIO class. If you are developing an application that requires file or network access, those classes make things incredibly simple to do, and they don’t freeze your GUI when you are in the middle of a process.

In this post I’ll show how to use KIO to retrieve files from network resources using PyKDE4. The whole example is also available in the kdeexamples module.

PyKDE4: Tag and annotate files using Nepomuk

Tue, 26 Oct 2010 19:37:16 +0000

Some time has passed since I last blogged… this was not only due to lack of time but also due to motivation (writing long texts can be discouraging at times). In any case, I’d like to rectify for that. In this post, I’ll talk about Nepomuk, and in particular how to use it to tag and annotate arbitrary files using its API in PyKDE4.

Before starting, let me say that creating this tutorial was only possible thanks to the help of Sebastian Trueg, who helped me by pointing out some mistakes I was doing.

What this might ever be?

Tue, 27 Jul 2010 22:41:14 +0000

The rest is up to you to figure out.

OCS and KDE Forums - work continues

Sun, 25 Jul 2010 18:49:22 +0000

With my last entry, I announced the start of the work for an OCS library for the KDE Community Forums. Today I’d like to blog again about the recent developments.

First of all, now there isn’t one, but two Python modules:

_ocslib, _ a pure Python module that can be used to interface with OCS-based forum systems;
ocslibkde, a PyKDE4 based module that can be used to interface with OCS-based forum system in KDE applications.

Currently ocslib supports reading and posting, while ocslibkde only reading (as of now). Both can be retrieved from the kde-forum-mods repository under the ocs-client subdirectory. The Python lib needs unit-testing, then I’ll be able to push a tarball soon for people to test (but you can always check out the Git repository). With regards to the PyKDE4 library, I plan on making a proof-of-concept plasmoid soon that shows how to use the API.

Open Collaboration Services and KDE Forums

Sun, 18 Jul 2010 10:00:39 +0000

For KDE developers, web-based forums are often uncommon workflows. Indeed, for communication among developers mailing lists are much better tools, especially since you can handle everything inside a client (most of the time), compared to forums where you have to use a web browser. The ways of reading, replying and interacting with posters are dramatically different. And that is why some developers find themselves uncomfortable with the KDE Community Forums.

A dedicated application would be usually much better than a browser, because you can work around the intrinsic limitations of the browser itself. The problem is that you can’t really access a forum with anything else than a browser. That is, it used to be like this, but now things are changing.

The world of KIO metadata - checking the HTTP response from a server

Thu, 18 Feb 2010 21:42:46 +0000

Recently, I investigated how to perform some checks on web addresses using KIO for Danbooru Client. My old code was synchronous, so it blocked the application while checking, thus causing all sort of troubles (UI freezing, etc.). Therefore, making the switch to KIO was the best solution. However, I had one problem: how could I check the HTTP response?

I knew already that the various ioslaves can store metadata, consisting of key-value pairs which are specific on the slave used. Normally you can get the whole map by accessing the metaData function of the job you have used, in the slot connected from the result signal. For some reason, however, in PyKDE4 calling metaData() triggers an assert in SIP, which ends in a crash (at least in my application; I stil need to debug further). KIO jobs have also the queryMetaData function, which returns the value of the key you have queried. Unfortunately, there was no way I could find the name.

Learning by example

Wed, 13 Jan 2010 21:37:38 +0000

With my brand-new SVN account, I just committed some code to kdeexamples, KDE’s example code module. In particular, I committed a simple example which shows how to use KConfigXT via PyKDE4, a simplified version of what I wrote about here.

As most of KDE is C++, and the Python API docs are translated directly from the C++ API docs, it is essential to have good examples to help newcomers learn faster. There are some PyKDE4 examples in the kdebindings module already, but I put mine in kdeexamples for a number of reasons:

Danbooru Client 0.5 is out

Sun, 27 Dec 2009 08:54:54 +0000

Sometimes answering apparently harmless questions on instant messaging can have unexpected results. In particular, I was telling about Danbooru Client to someone and a question popped up “Why don’t you support pages?”. It seemed a nice idea, so I branched off the code (yay for git!) and started working on it.

Well, it took me more than a month to get this thing done… I didn’t spend every day coding, but it was a challenge. Glad it’s over now, which means that Danbooru Client 0.5 is finally available. Grab it at the usual place on kde-apps.org.

Danbooru Client - a client for Danbooru based sites

Sun, 25 Oct 2009 20:16:50 +0000

A while ago I presented “danbooru2nepomuk”, a small program to tag images coming from Danbooru-based image boards. Today I want to present the evolution of that program, that is a PyKDE4 client for those boards.

HOWTO: KConfigXT with PyKDE4

Mon, 19 Oct 2009 21:43:10 +0000

If you read around the KDE Techbase, or if you develop KDE applications, you may have heard about KDE’s KConfigXT. This is an extension of KDE’s KConfig, and can be used to generate nice configure dialogs with multiple pages with minimal effort, also taking care of saving and applying settings. In short, something really neat! But there are problems when using it with interpreted language bindings (such as PyKDE, which is the one I use):

KConfigXT requires an XML file and an INI-like file to be compiled by kconfig_compiler in order to produce C++ files
There is no such a tool (at least to my knowledge) that does the same job for bindings

So what to do? Either give up on the niceness of KConfigXT, or work around the issue. I chose the latter.

DataMatrix 0.8 is finally out

Sat, 13 Jun 2009 13:29:40 +0000

At last, after months of inactivity, I pushed out a new release of DataMatrix. Although the version bump is small (0.8) there are a lot of changes since last releases. The most notable include:

Ability to apply functions to elements of the matrix
Ability to filter rows by column contents
Ability to transpose rows with columns
An option to load text files produced by R (which are, by design, broken)
Removed the getter for columns, using dictionary-like syntax directly
A lot of bug fixes

The download links on the project page have been updated, along with the documentation. Also, there is another change, because from now on the official Git repository is hosted on gitorious.org, and no longer on github, because gitorious (the software) is also free, while github.com’s is not. It’s mainly a philosophical issue (the same that prompted me to switch from twitter to identi.ca).

DataMatrix 0.7 has been released

Sat, 27 Dec 2008 15:33:07 +0000

Finally a new entry! I’ve been extremely busy with other things, that is why I did not have time to write more. One of the main reason is related to an important landmark in my professional career, but I’ll write more about it after January 1st (hint: those who follow my Twitter updates may have already understood).

As a nice way to break the hiatus, I’m releasing a new version of DataMatrix, my implementation of R’s data.frame in Python. Although the version bump is small, there are loads of improvements. First of all, there is proper support for file-like objects, as well as support for appending and inserting both rows and columns. writeMatrix has been substantially improved and now writes files correctly, and I have added (experimental) support for a DataMatrix object that does not require files - EmptyMatrix. Also, there is now proper documentation. Last but not least, unit tests have been added, a good way to watch out for regressions in the code.

DataMatrix page up

Sun, 12 Oct 2008 20:14:14 +0000

Ok, ok… my definition of “tomorrow” is not like what most people use, apparently. Although I took quite a while, now there is a static page on DataMatrix. There you will find a summary of wht I wrote in my other blog posts regarding this module. Of course, it will be kept up-to-date should I release a new version.

Aside that, I put a contact form on this blog. It may be useful for reports that aren’t directly related to the posts.

DataMatrix 0.5

Fri, 19 Sep 2008 19:54:58 +0000

At last, since it’s been like ages, I decided to put out a new version of DataMatrix. For those who haven’t seen my previous post, DataMatrix is a Pythonic implementation of R’s data.frame. It enables you to manipulate a text file by columns or rows, to your liking, using a dictionary-like syntax.

In this new version there have been a few improvements and correction to a couple bugs (for example saveMatrix did not really save) and the start (only a stub at the moment) of an append function to add more columns (I’ll also think about a function to add rows).

data.frames in Python - DataMatrix

Sun, 29 Jun 2008 08:13:55 +0000

For a long time I have tried to handle text files in Python in the same way that R’s data.frame does - that is, direct access to columns and rows of a loaded text file. As I don’t like R at all, I struggled to find a Pythonic equivalent, and since I found none, I decided to eat my own food and write an implementation, which is what you’ll find below.

QSql vs DB-API?

Fri, 28 Dec 2007 12:58:40 +0000

I’ve recently begun trying to create GUIs for my Python applications with PyQt, and I can say I’m absolutely loving the toolkit, relatively easy to use and featureful. As I’m trying to create a GUI for some module I wrote that deals with databases (using MySQLdb), I also learnt that Qt has a series of classes for dealing with databases, mainly QSql.

My question, directed to whoever has experience with QSql and the Python DB-API, is: what are the advantages of one approach to the other? I’m leaning towards DB-API because like that I can create modules which work also in command line applications.

Gene identifiers

Thu, 15 Nov 2007 19:57:16 +0000

While working today on an annotation class in Python I stumbled on a problem. Normally I work with lists of genes that are consistent, i.e. all Entrez Gene IDs (or RefSeq IDs, or Genome Browser IDs…), but today I had a list of mixed identifiers.

The subsequent idea was “let’s implement auto-detection of common identifiers in the class”. The problem is… is there any actual documentation on how identifiers are made? So far, using regular expressions, I’ve tracked down a few:

Data clustering with Python

Wed, 07 Nov 2007 18:15:29 +0000

**Notice:**Just now I realized this has been linked to to a Stack Overflow question. I recently wrote a new post that uses a different technique and a combination of R and Python. Check it out!

Following up my recent post, I’ve been looking for alternatives to TMeV. So far I’ve found the R package pvclust and the Pycluster library, part of BioPython. The first one also performs bootstrapping (I’m not sure if it’s similar to what support trees do, but it’s still better than no resampling at all). I’ve found another Python project but it is still too basic to perform what I need.

Script to make captures of movie files

Tue, 16 Oct 2007 20:44:46 +0000

The other day I was thinking about how to make screencaps for the anime I watch. Windows users often use Media Player Classic, which can create a video contact sheet (i.e., a series of captures) out of a movie file. I had two problems with this:

The biggest is that it runs on Windows, and I don’t use Windows;
The frames needed to be manually cropped every time, which was slow.

SOFT file woes

Tue, 09 Oct 2007 20:00:23 +0000

Today I started working on a data set published on GEO. As the sample data were somehow inconsistent (they mentioned 23 controls when I found 28), I decided to parse the SOFT file from GEO in order to get the exact sample information.

I did a grave mistake. First of all, Biopython’s SOFT parser is horribly broken (doesn’t work at all) and quite undocumented: I could work around the lack of documentation (API docs) but not with the fact that it wouldn’t work. So I turned to R, which offers a GEO query module through Bioconductor.