Email Threading in KMail: Your Help is Needed!

Hi KMail users,

we have a little favor to ask from you Smilie: :-) On the KDE PIM Sprint we discussed how to improve email threading in KMail by using Akonadi to store the information, so that KMail does not have to compute it every time. This would make opening a folder almost instant, all threads would be reconstructed immediately and it would massively improve CPU and memory consumption (so it’s totally something worth helping us with Smilie: ;-)) More details on what else we discussed and implemented will follow in another blog post tomorrow.

To implement the threading caching, we need to know, whether in these days it still makes sense to support threading by Subject. It’s used as a fallback when grouping by standardized email headers (In-Reply-To, References) are missing, which used to be a case with buggy email clients years ago, but hopefully is better now, so we could drop it, which would massively simplify the algorithms.

So we would like you to disable threading by Subject, observe how much it breaks your threading (and potentially your workflow) and report back to us. To disable it, go to View ->Message List ->Aggregation -> Configure. There go to Groups & Threading tab and in Threading combobox select Perfect and by Reference. If the combo boxes are disabled, you have to click Clone Aggregation to clone the default settings, and use the clone.

View->Message List->Aggregation->Configure...

View->Message List->Aggregation->Configure…

Aggregation Configuration

Aggregation Configuration

If removing threading by subject would break threading and workflow for too many users, we will keep the settings and we will try to figure out another way to do it.

So please configure your KMails, and let us know in comments below this post, on IRC, kde-pim mailing list or through any other communication means (just please try to avoid using smoke signals and pigeons Smilie: ;-))

Thank you for your help!

Category(s): Akonadi, KDE
Tags: , , ,

64 Responses to Email Threading in KMail: Your Help is Needed!

    Robert Kratky says:

    Works just fine for me. Get rid of it.

    Petr Svoboda says:

    No problem here with disabled threading by subject. Thank you for your work!

  1. Broke my threads here.

    Stefan Vater says:

    Broke some of my threads Smilie: :-(

  2. It breaks threads of bug reports in github among others.

  3. Broke some long threads.

  4. For mailing list traffic, it works fine.

    For some bug reporting, it breaks.

    For my inbox and personal communications, it often breaks. My replies are in the sent box, not in the inbox (e.g.) and this seems to cause issues … and that is between people using the latest Kontact/KMail even.

  5. Breaks threads sent by Microsoft Outlook 14.0

    Here are the interesting headers (corporate email, I can’t share everything) in the answers :

    References:
    In-Reply-To:
    Thread-Index: AQNrwSVzvMLodwFzw8xq3V7+H17T7Jbw/Jdg

    And in the first mail of the thread (Outlook 14 too) :
    Thread-Index: Ac7iF8qpP7YIwk6mQo64U8xUAZeSVw==

    No idea how to use them, it’s base64 of a blob :/

  6. Works here

  7. Probably/Hopefully you are already investigating this. I really would like to see the full thread at once (received & sent mails). This also addresses the issue raised by Aaron.

    • I BCC every email to myself too which sort of gives this feature.

    • Right now you can select both “INBOX” and “SENT” folders at the same time in KMail and you will get the full conversation.

      In future I would like to see a complete subtree of the particular conversation in the mail reader window when reading emails. This should be much easier to do with Akonadi, than now from KMail.

  8. Sounds fine but what sounds a “nice to have” idea to me is that if you come across an email that isn;t in the correct (or any) thread, you could manually move it into the thread and this is remembered.

    • +1
      … and out of threads, too!

        Jani-Matti Hätinen says:

        Yes please.

        Support for manual threading would be awesome.

        Especially in a normal corporate environment most people are absolute dolts when it comes to mail threading and subject lines. Everyone knows about the 8 month old threads with hundreds of messages and dozens of completely unrelated subjects as well as the other extreme of a single conversation divided into dozens of threads with the super informative subject lines of Re: and Fwd:. Would be great if Kmail would support fixing those.

  9. It breaks quite some threads for me…

    Some guy from the internets says:

    Hello, I use KMail but I don’t know what is ” threading by Subject,”

    For helping you, first you have to help me, hehe sorry!

  10. Well it would say it is broken for quite some time. I have a folder for Bugzilla mails. If you don’t get the first mail which opens the bug and you subscribe later. The mails aren’t threaded at all …

    Here is an example of KDE Bugzilla-Mails:

    http://min.us/i/blMHsNY3qsoN1

  11. You should post it to the KDE forums as well.

  12. Github threads break if the thread leader is too old.

    Note that they all seem to have the same reference, so in theory kmail should be able to find the thread leader (or fake one) without looking at the subject. The thread would look different (very flat), but that would arguably be more correct, and in any cas not break my “see all messages from this thread in one place” workflow.

  13. You mean you won’t accept RFC6214 compliant datagrams?
    I Don’t see any issues here, but most of my mails are mailing lists.

    • RFC6214 seems to be about adoption of IPv6, we are not probably going to support that Smilie: ;-)

      I know there are some RFCs about how to do threading, we will definitely look into them, but I can’t promise 100% exact and full implementation.

  14. Threading without subject completely breaks threads for me. I’m on a GroupWise server (not for much longer Google willing), and every single thread is gone after removing subject.

  15. It breaks some threads. Most threads affected are from bug reports or ticket systems. Some are normal replies from Thunderbird with correct references in them (so I think it could be a bit better even without subject threading).

  16. My default since email offered threading is by subject. I use it at work and it helps my search greatly as most often I need to search by subject or keywords therein. Please keep this option, even if its not the default. Thank you.

  17. I haven’t been using this for ages since it often threaded unrelated messages together.

  18. Just posting to say there was a warning on the page:

    Warning: Missing argument 2 for wpdb::prepare(), called in /srv/http/www/p/progdan.cz/www.progdan.cz/wp-content/themes/grey-opaque/functions.php on line 2536 and defined in /srv/http/www/p/progdan.cz/www.progdan.cz/wp-includes/wp-db.php on line 992

  19. Disclaimer: I am extremely biased, as I was the original implementer of subject threading in KMail. It was one of my first KDE contributions, about 10 years ago. Coming from mutt, which did it right, I missed it in KMail and foolishly attempted to fix it. It was way harder than I thought, but very educational.

    That said, I still believe it is useful and quite necessary, mainly because some very widely used mailers like MS Outlook get threading headers wrong and also because the mechanisms that are needed for it are also used to implement threading by References headers. This allows threads to be sorted properly that have messages missing, like your own replies. As Aaron points out that’s very frequent with people who have sent mail folders.

    KMail has the second best threading in the world, I think, second only to mutt because that is faster (easier to do with a much simpler interface). I for one would hate to see it regress Smilie: :).

    Laurent Pointal says:

    Break some of threads.

    Looks like its threads where there is a “hole” in the References: list. By example i can see that a reply of mine, for which the email is not in the “In” box (but in my “Sent” box), make the reply’s reply beginning a new thread.

    Hope this help.
    A+
    Laurent.

  20. I think I never used it. At least I’m not using it now. And, as already has been said, threading by subject tends to group unrelated mails and makes them unfindable. Thus for me it would be okay if you kick it out.

  21. It works for me. The ticket systems will be affected maybe..

  22. I love you! I think you may have just *fixed* my workflow. “by subject” was probably the reason KMail kept grouping multiple, chronologically quite separate conversations with the same subject in the same thread here, which drove me insane and never happened in other clients. I wish I had known about this sooner.

    Dominik Kalinowski says:

    Broke my all threads.

    Olaf the Lost Viking says:

    It works fine for me without threading by subject (only very few broken threads, not worth mentioning). If the memory hunger goes down it sounds like a nice price to pay for _me_. I did NOT check on bug reports though, as I am using the webinterfaces.

    The problem with the seperate “Sent” and “INBOX” mails can be prevented by simply storing your own mails in the same folders as the messages from other “participants”. For me, this seems to be much more usable than an unsorted “Sent”-folder. So I have a BCC in all my mails with me as the recepient. This leads to all mails going through my filters on the server and in the end sitting in the same folder as the original mail or future answers (independently of the client I used (KMail, K9Mail, web). Like that, I do always have full threads will all relevant mails.

  23. Everything it’s ok… For now!

    Rajesh G Iyer says:

    Found no problem, works fine. Smilie: :)

  24. It works fine for me.

    Regarding the separate sent box issue – wouldn’t this not be a problem if Akonadi is doing the threading? Because it would be able to construct the threads over all folders?

  25. I actually would like threading per people. See the list of people who last contacted me, descending. Click on a name to see the list of emails from that person.

    Sergey Salnikov says:

    It breaks some of my threads. It’s difficult to find any regularity because e-mails in those threads are from many different people.

  26. Works like a charm.

    Andreas Kuhl says:

    Breaks some threads for me, please keep it.

    Zoran Dimovski says:

    I have a large amount of mail and everything seems to be fine. Don’t have problem with the threading.

  27. I just disabled it and it barely removes any threads on a almost 100% gmail powered inbox, as everybody else uses gmail to send me messages. However, reading the many people that do get broken threads – for instance due Outlook – I believe its still necessary to keep it as Kmail has indeed the best threading in the world. I would use no other client just because of the way mails are threaded.

    However, there should be some way to prevent fulltext indexes or another reason why you can’t simplify the implementation and move it into Akonadi. There must be a good trade-off here to keep both.

    Jonathan Marten says:

    As others have mentioned, there are problems with threading where the thread starter message is not available, especially Bugzilla mails where the user has joined the bug late. If subject threading were retained in some form and worked in this case, it would be most useful. See bugs 129973 and 279115.

  28. It fixes replys to bulk mails for me but also breaks some threads, but not all.
    Mostly of the same person which I don’t like anyway.

  29. A bit off-topic, in case you didn’t already know about it – message threading by jwz.

    Michal Hlavac says:

    I think, that mails from yahoo doesn’t work

  30. Works fine, but the more concerning issue is the frequent hanging on folders :/
    There are numerous threads around kde forums etc and some are quite dated. Still that seems to have no solution.

    Bob Waddington says:

    Really bad news for me.

    gmail correspondence threads were lost. Two replies to the same email (from my kmail) from the same person within a few minutes of each other were treated differently.

  31. It breaks the aggregation a bit, but I can live with it.

  32. It broke threading for me on an email which I believe was sent from gmail’s web interface.
    Relevant header: In-Reply-To:

    I’m in favour of adding support for manually combining emails into threads, esp. if it uses the Filter mechanism (in addition to just drag and drop).

    Happy Heyoka says:

    Toggling between the two flavours of “Activity by Date” breaks threading for me without including Subject.
    People above suggested “all gmail” based threads worked – but I have some that didn’t…

    I could live with that if the benefit outweighed the cost of keeping ‘Subject’.

    As an aside, please consider the idea that maybe part of the problem is UX of driving the whole aggregation thing.

    One example: you go to “Configure…” in the “Aggregation” sub-menu, clone a new item, change it, and exit the dialog box. The new form of aggregation is _not_ now the selected method of aggregation… you have to go back to the “Aggregation” sub-menu and select it. I would argue a user makes a new form of aggregation in order to _use it right away_ far more often than not.

    Also, maybe a non-modal live “aggregation controller” – so the currently displayed message list changes appearance as you change the method (eg: like a live filter preview in a photo editor).

    My point – fantastic functionality is of no use if ux is very complicated.

    anyway, I know, code or stfu Smilie: :-)

  33. Sorry, but that’s a bad idea; breaks lots of threads Smilie: :-(
    Why would you want to remove a working feature in the first place ?

  34. The setting does not break a single thread for me. Don’t need threading by subject.

    Simon Slater says:

    Looks fine here.

  35. Breaks thread for mail sent from our RT ( http://www.bestpractical.com/rt/ ) installation. Threading is pretty useful there Smilie: :(

  36. Breaks quite a few “threads” for me. I quote “threads” because quite often I delete or archive the less interesting mails in a thread besides not having my own emails in the inbox, so many “threads” in my inbox are very incomplete.