KDE PIM, Google Integration & more

I haven’t blogged about my involvement in KDE PIM in a while, so let’s see what’s new there, especially in the Google integration part….

Reborn Google Resources

Just before the KDE PIM sprint in Berlin this month, I’ve sat down and written completely new API for LibKGAPI (the library that implements Google API and is used by the Akonadi resources for Google services). The new API is job-based, and therefore much more awesome than the old one (which is known to suck). Anyway – what does this mean? It means that the new resources are awesome as well!

Google Contacts rGoogle Contacts resource & contacts groupsesource now has a full support for contacts groups. All contacts are stored in the top-level collection and are linked to the respective groups, so it does not matter where you edit the contact, you are still modifying the same instance. Like in the web interface.

Google Calendar now supports limited sync, so you can choose to only sync events from last year, or last two years (the default is last 3 years) instead of the full history.

Both resources have improved status reporting, error handling, are more stable (no more mystery crashes due to unhandled exceptions thrown from LibKGAPI) and subjectively synchronization is faster too.

Murdered Google Resources

As most of you probably noticed by now, Google is planning to shut down Google Reader by July 1. It’s pitty, because I already had a fully working Akonadi resource for Google Reader ready in the akregator_port branch. Cost me lot of time and nerves. Well, the resource is not there anymore and the only memory of it is greader branch with API implementation in LibKGAPI (which will die as well sooner or later). The good news however is that I can now help Alessandro and Frank with ownCloud News and the ownCloud Akonadi resource, so that we rock when Akregator2 is out Smilie: :-) I can’t wait to see what has changed in ownCloud since I installed 3.0.0 some time ago…

Upcoming Google Resources

I have two feature requests in bugzilla: one is to support Google Bookmarks, which is fairly complicated because of missing official API and absolutely no write API. So this is not going to happen soon. The second feature request is for Google Drive KIO slave. This is much more interesting task. I already tried writing Google Docs KIO slave about three years ago and I failed epically. Retribution! There’s almost complete API implementation by Andrius in LibKGAPI git, so I plan to port it to LibkGAPI2 and see whether we can together fight the Dark side and create a nice and shiny KIO slave.

Finally, deep in the dark corners of my mind, my so far the most evil plan is slowly shaping. The plan includes modifying the current IMAP resource, reusing most of it’s code and subclassing some specific parts to build a native GMail Akonadi resource that would support some GMail-specific IMAP extensions. The main idea is to support one-mail-in-multiple-folders-at-once case. Right now the IMAP resource handles that by creating a new instance of the same email in multiple folders. My bold plan is to store all emails in Inbox and link them to respective folders. This means that marking an email as read in one folder, will automatically mark it as read in all other folders (because it’s a single instance). The IMAP resource looks scary though, so I don’t know yet when I’ll get the courage (and time) to sit down and actually start coding…I guess probably after Akademy, after I talk to some people.

Batch Operations in Akonadi

I have talked to Volker Krause during the KDE PIM sprint about how to effectively handle “Mark feed as read” in the Google Reader resource. Currently, Akonadi creates a new notification for every change, therefore marking 300 items as read generates 300 notifications, which are delivered to the Akonadi resource, which should then create 300 HTTP request to store 300 changes. You probably agree that this slightly suboptimal. (I temporarily solved the problem by caching the notifications in the resource itself and then sending a big request to Google Reader at once). The solution that Volker suggested sounds fairly simple (it’s not) – batch notifications – i.e. a single notification about single change involving multiple items. The supported changes can be flags change, deleting or linking of items. By being able to deliver single notification about mass-change to Akonadi clients and to Akonadi resources represents new possibilities for optimizations. For instance the IMAP resource could simply send a single command to add a flag to multiple emails at once, instead of doing it one by one. The same goes for other operations and other resources that are dealing regularly with operations on larger sets of items. The obvious result: performance boost! After two weeks the work is in semi-working state – it works, but it goes nuts if more than 5 items are involved. The cause is known, but solution not (but I’ll get there eventually Smilie: :-) )

Akregator 2

I’m occasionally helping with Akregator2 (Akonadi port of Akregator). Recently (ok, it was two months ago… ) I’ve written Akonadi Nepomuk Feeder plugin that is feeding RSS Articles into Nepomuk and a Search window (slightly inspired by the one in KMail) in Akregator2 where you can do full-text search (+ search via other criteria, including author’s name and date of publishing) based on data indexed in Nepomuk. Obviously, when I wanted to demo that on the KDE PIM sprint I found out that it’s not working as good as I thought, so there’s still some work to be done. But in general I’m happy to say, that from time to time it finds something Smilie: :-).

Akregator 2 Search Window

Ok, so that’s about what I was, am and will be working on in KDE PIM. Here I’d like to say big thank you to all KDE PIM devs, because they are doing an incredible job. Thank you!

Category(s): Akonadi Google, KDE
Tags: , , , , ,

26 Responses to KDE PIM, Google Integration & more

  1. Don’t ditch that Reader API work – it seems fairly likely to me that the major RSS reader app makers are going to get together and agree on a single drop-in replacement system which will replicate the Google API, so all their apps keep working. See, for e.g., http://www.marco.org/2013/03/14/baby-steps-replacing-google-reader . So I suspect it’ll still turn out to be useful work.

    But hey, ownCloud News? ownCloud continues to be ridiculously awesome and apparently implementing everything I ever wanted. Awesome stuff. And their CalDAV implementation *actually works*! It’s like a little miracle!

    • Well, I actually hope it won’t happen. The API is quite pain to use. Stupid way of changing data (you need to obtain an “edit token” first, although you are already authenticated), weird IDs (“feed/url-of-the-feed”Smilie: ;), no support for actual folders (you have only “categories| and you can’t get list of the categories – you need to fetch all feeds first and then you can reconstruct the “folder” tree from it). Also there is no simple \SEEN tag to mark item as read, but instead an obscure string like “tag/username@gmail.com/read”.

      Please understand I’m not blaming Google for designing a bad API. The API was never meant to be used publicly, only via the web-interface and they surely had their reasons to do it the way they did it. But if we are standing at a point where we can actually decide how a standardized API for future web-RSS readers should like, we should think again.

    Josef Filzmaier says:

    yeah, love seeing this! I’m using KDE PIM since about a year now, it is the best solution i have ever tried IMO. Keep up the good work! Looking forward to the reborn and upcoming google resources Smilie: :)

  2. Oh yeah, the GMail resource we’ve all been waiting for years.That would be awesome. Well this + kscreen, I would be grateful for ever.

      Djuro Drljaca says:

      That “copies of the same email in multiple folders” is one of the biggest reasons why I stopped using kmail and switched to gmail web interface directly…

    Lindsay Mathieson says:

    That looks amazing Dan, a huge improvement for Calendar and Contacts, which were already pretty good.

    Any idea as to which KDE release they make? 4.10.2? Smilie: :)

    Also, which modules would I need to build to try these out?

    • Ooops, I forgot to mention that the new resources will be available in KDE 4.11 (new strings, new features, new dependencies). If you want to try it out, you need to build libkgapi and kdepim-runtime from master (and if you decide to try out, please bug me with any issues you find there).

        Lindsay Mathieson says:

        Already running kdepim-runtime master, is https://projects.kde.org/projects/extragear/libs/libkgapi/repository the correct repo for linkgapi?


        • Yup, that’s it Smilie: :-)

            Lindsay Mathieson says:

            Does it need kontact trunk as well? built/installed libkgapi and kdepim-runtime with no issues – libkgapi2 definitely built and installed.

            I removed and deauthorised the existing contacts account then added a new one. It appears in Akonadi Console but not in kontact. Kontact is 4.10.1

            Also not seeing any groups in AkonadiConsole, just the account.

            thanks – Lindsay

            Lindsay Mathieson says:

            Furthur to that – Calendar is working beautiful, already fixed several quirks and bugs I had noted with the old one.

            Contacts lost the authorised account, I re-added it, still not showing in Kontact.

            Maybe issues with authorising with Google? I have two factor auth.

            Thanks – Linz

          • I don’t use 2-factor auth, so can’t tell right now. I’ll try during the weekend. Could you please open a bug report against Akonadi/Google resource/Contacts/Git? In case it’s not 2-factor auth, I’ll ask you there for more info.


    Maarten De Meyer says:

    Yes please, google drive kio implementation!
    I have also tried this myself in the past and found it to be way over my head. Hopefully you succeed this time Smilie: :)

    Is there even an api for google drive?

  3. Rekonq can sync to Google Bookmarks. You might have a look at its implementation Smilie: :-)

  4. I am looking forward too and at the same time I am little hesitant about the release of Akregator 2. I am looking forward too, because Akregator definitely needs some improvements and fixes. At the same time the past releases of the Akonadi-versions of KDEPIM-apps where no pleasant experience. Although improving e.g. Kmail still causes problems.
    So I would like to know if there is any chances, that there will be test-packages of Akregator which can be installed independent of an existing Akregator installation? It would be nice if this time as many bugs and regressions could be found in advance. I would be willing to test, but I would like to have it running parallel.

    • You can use Akregator2 in parallel with Akregator1, you just need to compile kdepim, kdepimlibs and kdepim-runtime from akregator_port branch (which means that you probably need Akonadi from master as well).

      I don’t know about any plans to do a pre-release. When akregator_port branches are merged to master, Akregator1 will probably be removed from git and user will be able to test during beta/RC release-cycle of KDE.

  5. Sad to see work wasted on supporting proprietary services and Google not even paying for it…

    Alejandro Nova says:

    The guys @ Feedly seem to be working in a Google Reader clone called Normandy. They are switching to this clone on July 1st and they are opening their API upon request. I find this an amazing alternative to keep my Google Reader feeds.

    Their official note says this:

    “Note 2: if you are a third party developer using the Google Reader API and would like to integrate with Normandy, please send an email to remi@feedly.com. We would love to keep the Google Reader ecosystem alive.”

    Lindsay Mathieson says:

    Hmmm – apart from the initial sync, Calender is failing to sync at all for me. Events created in Kontact to not appear in Google Calendar and visa versa.

    Want a bug report for that?