Akonadi Google Resource: complete rewrite

Standard

Hi there,

today I finally pushed a completely new version of the Akonadi Google Resource. Since last update cca 2 months ago I’ve completely rewritten the resources, moving most of the functionality into libKGoogle . The library now provides access to Google services and can be easily used by any other project. So what’s new? A lot of things changed for programmers. I’ve begun to fulfill my threats about providing a C++/Qt library for accessing Google Services. It’s not pure Qt, I’m using some KDE stuff in there, but that will change :)

LibKGoogle now provides API similar to QNetworkAccessManager (KGoogleAccessManager) through which you can send KGoogleRequest and if it works, you’ll receive a KGoogleReply. Each Google Service is represented by a KGoogleService subclass, implementing XML and/or JSON parser and serializer and providing informations about fetch/create/update URLs. Each object (contact, event, calendar) is represented by KGoogleObject subclass. For now, Event class is implements KCalCore::Event as well,  the Calendar and Contact classes provide their own implementation of everything. Porting it to KABC/KCalCore is on TODO.

libKGoogle provides unified authentication  framework and stores the tokens in KWallet. For now, you can only use one account per service (means that you can’t have two calendars from two different Google accounts), but this is on my TODO as well. The framework is also able to automatically refresh the tokens when they expire, so that should fix the problem with “resources forgeting the authentication“.

The API needs some more polishing and I must somehow force myself to write documentation, but the I think the libKGoogle can provide a solid base for any Google-related services.

The library is still in the same git repository with the Resources. I will try to move it to it’s own repo when it gets support for more services and the API will really stabilize.

Contacts Resource new Settings DialogContacts Resource new Settings Dialog 2From users’ point of view, there is just a very slight change in Contact Resource settings dialog. There are no new features in any resource. Though there are probably many new bugs :F So please try it out and don’t be afraid to spam me with any issue, idea, problem, comment, invitations to a beer etc…:)

With libKGoogle I hope more developers will begin to create Google-related applications for Qt and KDE. This library could one day be a good start-point for them.

Plans for future versions:

  • support more contacts and event properties
  • support multiple Google accounts per service
  • changing contact photos
  • tie KGoogleObjects to KCalCore and KABC
  • API cleanup
  • DOCUMENTATION :/
  • make the libkgoogle optionally KDE-independent (long-term plan)

44 thoughts on “Akonadi Google Resource: complete rewrite

  1. Henning Rogge

    Hi,

    I tried your new code this morning, but I have still the crash problem.
    I first removed all old Akonadi extensions for GoogleCalendar and installed your new code. Then I opened KOrganizer, added a new Akonadi Calendar and used “Manage Calendar Sources” to add a new resource. It worked, but it crashed (again) during synchronization.

    Is there anything I could to to help to debug the problem?

    This is the output of the kde crash manager:

    Application: Akonadi Resource (akonadi_googlecalendar_resource), signal: Aborted
    [Current thread is 1 (Thread 0x7f228b411780 (LWP 2678))]

    Thread 2 (Thread 0x7f2275677700 (LWP 2681)):
    #0 pthread_cond_timedwait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:216
    #1 0x00007f228a4ba79e in wait (this=, mutex=0x29a2690, time=30000) at thread/qwaitcondition_unix.cpp:86
    #2 QWaitCondition::wait (this=, mutex=0x29a2690, time=30000) at thread/qwaitcondition_unix.cpp:160
    #3 0x00007f228a4af218 in QThreadPoolThread::run (this=0x29a22d0) at concurrent/qthreadpool.cpp:140
    #4 0x00007f228a4ba175 in QThreadPrivate::start (arg=0x29a22d0) at thread/qthread_unix.cpp:320
    #5 0x00007f22840e2d8c in start_thread (arg=0x7f2275677700) at pthread_create.c:304
    #6 0x00007f2284b1f04d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112
    #7 0x0000000000000000 in ?? ()

    Thread 1 (Thread 0x7f228b411780 (LWP 2678)):
    [KCrash Handler]
    #6 0x00007f2284a6cd05 in raise (sig=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:64
    #7 0x00007f2284a70ab6 in abort () at abort.c:92
    #8 0x00007f22853236dd in __gnu_cxx::__verbose_terminate_handler() () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
    #9 0x00007f2285321926 in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
    #10 0x00007f2285321953 in std::terminate() () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
    #11 0x00007f2285321aa6 in __cxa_rethrow () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
    #12 0x00007f228a5a0bb2 in QEventLoop::exec (this=0x7fffbfac9660, flags=...) at kernel/qeventloop.cpp:214
    #13 0x00007f228a5a4ecb in QCoreApplication::exec () at kernel/qcoreapplication.cpp:1008
    #14 0x00007f228af51d36 in Akonadi::ResourceBase::init (r=0x26bd480) at ../../akonadi/resourcebase.cpp:393
    #15 0x0000000000409d36 in init (argc=, argv=) at /usr/include/KDE/Akonadi/../../akonadi/resourcebase.h:188
    #16 main (argc=, argv=) at /home/henning/develop/akonadi-google/calendar/calendarresource.cpp:411

    This is the output from the console where I started Akonadi with akonadictl:

    akonadi_googlecalendar_resource_8(2678)/libakonadi Akonadi::CollectionSync::Private::createRemoteNode: Collection ' "Google Calendar" ' does not have a remote identifier - skipping
    reply received, 25 items fetched
    akonadi_googlecalendar_resource_8(2678)/libkdepim Akonadi::PluginLoader::scan: missing or empty X-KDE-ClassName value in " "/usr/share/kde4/apps/akonadi/plugins/serializer/akonadi_serializer_kcal.desktop" "

    akonadi_googlecalendar_resource_8(2678)/libkdepim Akonadi::PluginLoader::scan: missing or empty X-KDE-ClassName value in " "/usr/share/kde4/apps/akonadi/plugins/serializer/akonadi_serializer_microblog.desktop" "

    akonadi_googlecalendar_resource_8(2678)/libkdepim Akonadi::PluginLoader::scan: missing or empty X-KDE-ClassName value in " "/usr/share/kde4/apps/akonadi/plugins/serializer/akonadi_serializer_mail.desktop" "

    akonadi_googlecalendar_resource_8(2678)/libkdepim Akonadi::PluginLoader::scan: missing or empty X-KDE-ClassName value in " "/usr/share/kde4/apps/akonadi/plugins/serializer/akonadi_serializer_contactgroup.desktop" "

    akonadi_googlecalendar_resource_8(2678)/libkdepim Akonadi::PluginLoader::scan: missing or empty X-KDE-ClassName value in " "/usr/share/kde4/apps/akonadi/plugins/serializer/akonadi_serializer_addressee.desktop" "

    akonadi_googlecalendar_resource_8(2678)/libkdepim Akonadi::PluginLoader::scan: missing or empty X-KDE-ClassName value in " "/usr/share/kde4/apps/akonadi/plugins/serializer/akonadi_serializer_bookmark.desktop" "

    Qt has caught an exception thrown from an event handler. Throwing
    exceptions from an event handler is not supported in Qt. You must
    reimplement QApplication::notify() and catch all exceptions there.

    terminate called after throwing an instance of 'Akonadi::PayloadException'
    what(): Akonadi::PayloadException: Wrong payload type (requested: sp(0); present: sp(2)
    KCrash: Application 'akonadi_googlecalendar_resource' crashing...
    KCrash: Attempting to start /usr/lib/kde4/libexec/drkonqi from kdeinit
    sock_file=/home/henning/.kde/socket-henning-desktop/kdeinit4__0
    Lost connection to resource "org.freedesktop.Akonadi.Resource.akonadi_googlecalendar_resource_8" , discarding cached interface
    void Akonadi::NotificationSource::serviceUnregistered(const QString&) Notification source "akonadi_googlecalendar_resource_8" now serving: ()
    void Akonadi::NotificationSource::unsubscribe() "akonadi_googlecalendar_resource_8"
    AgentManager::removeAgentInstance: calling instance->quit()
    PreprocessorManager::unregisterInstance( "akonadi_googlecalendar_resource_8" )
    QProcess: Destroyed while process is still running.
    ProcessControl: Application '/usr/bin/akonadi_googlecalendar_resource' stopped unexpected (Process crashed)

    • Thanks for the feedback. I did some googling, and the problem seem to be same as here for example.
      KDE can’t find a serializer (a convertor from C++ objects to a data stream that can be stored in Akonadi database) for KCalCore and therefor it crashes. I tested the resource on clean Kubuntu 11.04 setup in VirtualBox and I can confirm that in the default installation, the KCalCore serializer is missing in KDE4.7. Please report this in Kubuntu buzilla, there’s nothing I can do about it.

      PS: please use <pre> tag for backtraces and logs next time, it’s easier to read :) Thanks!

      • Thomas McGuire

        I had the same problem when I wrote the Facebook resource.
        In the end, I added a CMake switch that let me toggle between using the old KCal and the new KCalCore. Using old KCal is the default, since that even works for newer versions of Akonadi and kdepimlibs, as KCal and KCalCore are just different APIs for the same data format.

        • Henning Rogge

          If its a problem between an old and a new API version, what is the long-term solution? Just using the old API sounds like a problem waiting to be happen.

          • Thomas’ solution is the best possible: provide support for both APIs and a CMake switch.

            Anyway your problem is caused by a bug in Kubuntu’s packaging: if they provide kdepimlibs with KCalCore API, they should provide KCalCore Akonadi serializer as well.

    • I just pushed an update with support for KCal API. Run CMake with -DUSE_KCAL=TRUE switch and libKGoogle and the Calendar Resource should be both using KCal API (you will see a lot of deprecation warnings during the compilation, but that’s OK).

      • Henning Rogge

        Yes, it works! Thank you very much…

        the only thing I noticed with a quick test is that “whole day” events in the calendar seem to be mapped to two days in KOrganizer.

        I am using a “read only” Akonadi resource, so its not that bad, but maybe its something you want to look into.

        • the only thing I noticed with a quick test is that “whole day” events in the calendar seem to be mapped to two days in KOrganizer.

          Yeah, there are some inconsistencies in what is a start date and end date for all-day events in KOrganizer and on Google. Plus Google seems to use different rules for single and recurrent events. Thanks for report.

  2. venky80

    Hey Progdan,
    You had mentioned that there were 2 projects for google-akonadi resource and those will be merged as one in future. Is this the result of that? Have you guys unified your efforts?
    Awesome project and thanks.
    Venky

    • Hi,
      actually nope. This is all my work. The “merging”, if it happens, would probably in the sense that the other resource would switch from libgcal to libKGoogle. But before that can happen, there is a looong way to go for libKGoogle.

  3. Serafean

    Hi, Great Work! Now I”m looking forward to the Nepomuk PIMO representation of contacts : facebook resource + google resource + personal contacts = contacts in triplicate. Messy.
    I also found an issue : create an event in KOrganizer, then go modify the duration in the google web interface, sync the google resource => the event appears twice, and only the newly appeared one syncs (the original can be deleted, nothing happens).

    • I knew about this issue but I wasn’t sure what could cause it. At the end, the data parsers were to blame :) Thank you, fixed in git now!

      • Serafean

        Yes, thank you! Another issue has come up (git 24-08) : deleting a calendar entry crashes plasma-desktop… just create an entry, and then delete it, watch plasma-desktop disappear (no backtrace). It doesn’t happen without the calendar widget loaded (either loaded by itself on desktop or through the digital clock). Tried with the normal iCal resource, there it works fine.

  4. KenP

    I just wanted to ask if there’s any traction with Akonadi/Kontact+MS Exchange resources?

    Thanks.

    • Sorry, this is not the right place to ask :) Try kdepim mailing lists or IRC, they may know if someone’s working on this.
      The general problem with MS Exchange is that you need an Exchange server to develop against and for stand-alone free-time developers it could be a problem to get one. And for what I know from Evolution, it’s not really fun to hack anything for Exchange :D

  5. Rino

    Hi!
    First of all, congratulation for your work and thanks a lot for it! I didn’t tried it yet, but I hope to do it soon ;)
    But to be honest, I believe there is a little user interface mistake in the two picture you posted: I can’t stop thinking the ok-button is unnecessary?

    I would suggest you the reading of this post:
    http://agateau.wordpress.com/2011/01/21/common-user-interface-mistakes-in-kde-applications-part-3-message-boxes/
    for me it was very helpfull.

    Rino

    • Rino

      Errata corrige: “unnecessary” not “unnecessary?”.
      My phrase wasn’t a question… :P
      bye,
      rino

    • Hehe, I’m really bad at designing user interface. I’ve fixed it in git by putting there only ‘Close’ button, since the actions are already committed after using the ‘Authenticate’ and ‘Revoke tokens’ buttons, so you don’t need (can’t) to confirm or cancel anything, you can just leave :) Thanks.

      And thanks for the link, looks interesting. I haven’t read it yet though, so I just hope the change I made conforms with what’s written there :)

  6. martin

    Hi progdan,
    first I want to thank you for your great work. I used to use akonadi-googledata before, but it kept bugging me recently, so I decided to give your implementation a try and it seems to work fine so far :)

    However, I had to adjust some settings in the CMake files (patch is attached inline). I’m using Fedora 15 with KDE 4.7 installed. Dunno why it didn’t compile out of the box.

    Keep up the good work!

    diff –git calendar/CMakeLists.txt calendar/CMakeLists.txt
    index 8e50168..7e09f1e 100644
    — calendar/CMakeLists.txt
    +++ calendar/CMakeLists.txt
    @@ -37,7 +37,7 @@ kde4_add_executable(akonadi_googlecalendar_resource RUN_UNINSTALLED ${calendarre

    target_link_libraries(akonadi_googlecalendar_resource
    ${KDEPIMLIBS_AKONADI_LIBS}
    - ${KDEPIMLIBS_KCALCORE_LIBS}
    + ${KCALENDAR_LIBRARIES}
    ${KDEPIMLIBS_KMIME_LIBS}
    ${QT_QTCORE_LIBRARY}
    ${QT_QTGUI_LIBRARY}
    diff –git contacts/CMakeLists.txt contacts/CMakeLists.txt
    index aa4f8ca..7208e44 100644
    — contacts/CMakeLists.txt
    +++ contacts/CMakeLists.txt
    @@ -40,6 +40,7 @@ target_link_libraries(akonadi_googlecontacts_resource
    ${KDEPIMLIBS_KABC_LIBS}
    ${QT_QTCORE_LIBRARY}
    ${QT_QTDBUS_LIBRARY}
    + ${QT_QTNETWORK_LIBRARY}
    ${KDE4_KDECORE_LIBS}
    kgoogle
    )

    • Looks like Christophe already committed it :) Thanks for the patch, it was an obvious mistake in my CMakeLists. I’m actually not sure how it’s possible that it compiled on my system.

  7. Alex

    Hi progdan, thank you for your efforts, you did the great job!
    There is a few things that prevents me to use it instead the google web interface:
    1. Missed notification settings, namely I can’t set email/SMS notification for the event.
    2. When an event is created trough the google web interface and its duration was set to “full day”, korganizer display it like this — http://img84.imageshack.us/img84/4214/278u.png (displays that the duration is two days instead of one). Yes, it’s not a problem if use only korganizer, but in case with shared calendars I can’t force all users to use korganizer or even better to say to change their OS :)
    3. Seems that korganizer still doesn’t know anything about new events, created through the web interface. I should press F5 or update calendar resource through its context menu, but perhaps there is a way to register a kind of callback on google? To know when event has been added/changed/etc.

    Best,
    Alex

    • Thanks for your interest, Alex.

      ad 1) Indeed, setting an email reminder seems to be broken
      ad 2) Yop, known issue, Google can’t make it’s mind how to define length of all-day events. I need to look into this.
      ad 3) There’s no ‘auto refresh’ atm, but will be soon :)

      PS: please, please use our shiny new bugzilla product for reports next time. Thank you!

    • Hi,
      feel free to push it there :) I’m not an Ubuntu user and I don’t have a slightest idea how PPA works, so I won’t be of much help here…

  8. Markus

    On Debian the QJSON library cannot be found with capital QJSON_LIBRARIES but only with small qjson_LIBRARIES. There was some discussion on that on the qjson
    mailing list.

    Would you mind adding the following to git?


    diff --git a/libkgoogle/CMakeLists.txt b/libkgoogle/CMakeLists.txt
    index afe039c..7097f6e 100644
    --- a/libkgoogle/CMakeLists.txt
    +++ b/libkgoogle/CMakeLists.txt
    @@ -18,6 +18,7 @@ target_link_libraries(kgoogle
    ${KDE4_KDECORE_LIBS}
    ${KDE4_KDEWEBKIT_LIBS}
    ${KDE4_KDEUI_LIBS}
    + ${qjson_LIBRARIES}
    ${QJSON_LIBRARIES}
    ${QT_QTWEBKIT_LIBRARY}
    ${QT_QTNETWORK_LIBRARY}

    Additionally, I changed all KDateTime::RFC3339Date to KDateTime::ISODate so that it compiles on Debian KDE 4.6.5. Is it possible to have a compile time switch based on the KDE version?

    Markus

    • Hi,
      the qjson-thing seems ok, thanks for patch.

      First I need to check if KDateTime will parse correctly all dates as ISODate instead of RFC3339Date (Google API docs explicitly state that dates are in RFC3339 format). I think KDateTime will properly parse fromString(), but I’m not sure if toString() would return valid date that would be accepted by Google API.
      If there would be problem, I’d probably provide custom implementation of the RFC3339 format.

      I’ll try to look at this as soon as possible ( = until the end of the week, I hope :) ) and let you know.

  9. BartOtten

    The new way of authentication is much better than the old one. Thanks for that!

    Bug: It seems to du/triplicate my contacts. Identical information appears in Akonadi. Any idea what goes wrong?

    Thanks for all the effort :D

  10. BartOtten

    Hehe, saw that report on my bughunt but did not connect it to contacts. I am no Git-expert let alone compiling stuff from it, but it seems to be resolved. Gotta wait untill my repo refreshes.

  11. Robert

    Nice project. I just compiled it from your git master branch.

    Calendar and Tasks seems to work very nicely, but the contacts part is crashing every time after sync. :(

    Application: Google Contacts of type Google Contacts (akonadi_googlecontacts_resource), signal: Aborted
    [Current thread is 1 (Thread 0x7ff291151760 (LWP 4341))]

    Thread 3 (Thread 0x7ff27d0f8700 (LWP 4343)):
    #0 0x00007ff29027011f in __pthread_mutex_unlock_usercnt () from /lib64/libpthread.so.0
    #1 0x00007ff28bc188a9 in ?? () from /usr/lib64/libglib-2.0.so.0
    #2 0x00007ff28bc18f59 in g_main_context_iteration () from /usr/lib64/libglib-2.0.so.0
    #3 0x00007ff29060e3a6 in QEventDispatcherGlib::processEvents (this=0x6f2670, flags=) at kernel/qeventdispatcher_glib.cpp:424
    #4 0x00007ff2905e2882 in QEventLoop::processEvents (this=, flags=…) at kernel/qeventloop.cpp:149
    #5 0x00007ff2905e2a7f in QEventLoop::exec (this=0x7ff27d0f7e20, flags=…) at kernel/qeventloop.cpp:201
    #6 0x00007ff2904fa5cf in QThread::exec (this=) at thread/qthread.cpp:498
    #7 0x00007ff2904fd015 in QThreadPrivate::start (arg=0x7c03f0) at thread/qthread_unix.cpp:331
    #8 0x00007ff29026cf05 in start_thread () from /lib64/libpthread.so.0
    #9 0x00007ff28d3cf53d in clone () from /lib64/libc.so.6

    Thread 2 (Thread 0x7ff27bc71700 (LWP 4344)):
    #0 0x00007ff2902711eb in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
    #1 0x00007ff2904fd41e in wait (time=30000, this=0x907df0) at thread/qwaitcondition_unix.cpp:86
    #2 QWaitCondition::wait (this=, mutex=0x8226f0, time=30000) at thread/qwaitcondition_unix.cpp:160
    #3 0x00007ff2904f1e8f in QThreadPoolThread::run (this=0×825610) at concurrent/qthreadpool.cpp:140
    #4 0x00007ff2904fd015 in QThreadPrivate::start (arg=0×825610) at thread/qthread_unix.cpp:331
    #5 0x00007ff29026cf05 in start_thread () from /lib64/libpthread.so.0
    #6 0x00007ff28d3cf53d in clone () from /lib64/libc.so.6

    Thread 1 (Thread 0x7ff291151760 (LWP 4341)):
    [KCrash Handler]
    #6 0x00007ff28d32ad95 in raise () from /lib64/libc.so.6
    #7 0x00007ff28d32c2ab in abort () from /lib64/libc.so.6
    #8 0x00007ff28dbb0f2d in __gnu_cxx::__verbose_terminate_handler() () from /usr/lib64/libstdc++.so.6
    #9 0x00007ff28dbaf0d6 in ?? () from /usr/lib64/libstdc++.so.6
    #10 0x00007ff28dbaf103 in std::terminate() () from /usr/lib64/libstdc++.so.6
    #11 0x00007ff28dbaf246 in __cxa_rethrow () from /usr/lib64/libstdc++.so.6
    #12 0x00007ff2905e2bb6 in QEventLoop::exec (this=, flags=) at kernel/qeventloop.cpp:214
    #13 0x00007ff2905e6c47 in QCoreApplication::exec () at kernel/qcoreapplication.cpp:1064
    #14 0x00007ff290c2d8f7 in Akonadi::ResourceBase::init (r=0x78e1a0) at /usr/src/debug/kdepimlibs-4.7.2/akonadi/resourcebase.cpp:393
    #15 0x000000000040f996 in Akonadi::ResourceBase::init (argc=, argv=) at /usr/include/KDE/Akonadi/../../akonadi/resourcebase.h:188
    #16 0x00007ff28d31723d in __libc_start_main () from /lib64/libc.so.6
    #17 0x000000000040a541 in _start () at ../sysdeps/x86_64/elf/start.S:113

    • Thanks. I’m aware of this crash (happens to me as well sometimes), but exams in college have higher priority atm :)

      If you want to get informed when the issue is fixed, submit the backtrace to bugzilla (I think anyone didn’t report it yet), I’ll close it when it’s done.

  12. pholthau

    Hey,

    first of all thanks for your effort!
    I cannot get the resources to work because everytime I try to authenticate with my google account there’s the following error:

    Parsing token page failed!

    Maybe you can help me with that.

    Cheers
    Patrick

  13. Bernhard

    Thank you for your great work!

    I have a small problem: If I try to add a shared calendar from another person i get an “unknown error”. Is this a known bug?

    Cheers

  14. txutxifel

    I have this bug when i try to configure a account:

    Parsing token page failed!

    Any idea?

    Thankf for your effort

  15. Igor

    I tested the last version of libkgapi today and problem with parsinf token is still with us.

    What information send to you for resolving the problem?

    p.s. Sorry for my english.

Comments are closed.