Email Threading in KMail: Your Help is Needed!

Standard

Hi KMail users,

we have a little favor to ask from you :-) On the KDE PIM Sprint we discussed how to improve email threading in KMail by using Akonadi to store the information, so that KMail does not have to compute it every time. This would make opening a folder almost instant, all threads would be reconstructed immediately and it would massively improve CPU and memory consumption (so it’s totally something worth helping us with ;-)) More details on what else we discussed and implemented will follow in another blog post tomorrow.

To implement the threading caching, we need to know, whether in these days it still makes sense to support threading by Subject. It’s used as a fallback when grouping by standardized email headers (In-Reply-To, References) are missing, which used to be a case with buggy email clients years ago, but hopefully is better now, so we could drop it, which would massively simplify the algorithms.

So we would like you to disable threading by Subject, observe how much it breaks your threading (and potentially your workflow) and report back to us. To disable it, go to View ->Message List ->Aggregation -> Configure. There go to Groups & Threading tab and in Threading combobox select Perfect and by Reference. If the combo boxes are disabled, you have to click Clone Aggregation to clone the default settings, and use the clone.

View->Message List->Aggregation->Configure...

View->Message List->Aggregation->Configure…

Aggregation Configuration

Aggregation Configuration

If removing threading by subject would break threading and workflow for too many users, we will keep the settings and we will try to figure out another way to do it.

So please configure your KMails, and let us know in comments below this post, on IRC, kde-pim mailing list or through any other communication means (just please try to avoid using smoke signals and pigeons ;-))

Thank you for your help!

64 thoughts on “Email Threading in KMail: Your Help is Needed!

  1. Aaron Seigo

    For mailing list traffic, it works fine.

    For some bug reporting, it breaks.

    For my inbox and personal communications, it often breaks. My replies are in the sent box, not in the inbox (e.g.) and this seems to cause issues … and that is between people using the latest Kontact/KMail even.

  2. Pierre

    Breaks threads sent by Microsoft Outlook 14.0

    Here are the interesting headers (corporate email, I can’t share everything) in the answers :

    References:
    In-Reply-To:
    Thread-Index: AQNrwSVzvMLodwFzw8xq3V7+H17T7Jbw/Jdg

    And in the first mail of the thread (Outlook 14 too) :
    Thread-Index: Ac7iF8qpP7YIwk6mQo64U8xUAZeSVw==

    No idea how to use them, it’s base64 of a blob :/

  3. Robert

    Probably/Hopefully you are already investigating this. I really would like to see the full thread at once (received & sent mails). This also addresses the issue raised by Aaron.

    • Right now you can select both “INBOX” and “SENT” folders at the same time in KMail and you will get the full conversation.

      In future I would like to see a complete subtree of the particular conversation in the mail reader window when reading emails. This should be much easier to do with Akonadi, than now from KMail.

  4. Ian

    Sounds fine but what sounds a “nice to have” idea to me is that if you come across an email that isn;t in the correct (or any) thread, you could manually move it into the thread and this is remembered.

      • Jani-Matti Hätinen

        Yes please.

        Support for manual threading would be awesome.

        Especially in a normal corporate environment most people are absolute dolts when it comes to mail threading and subject lines. Everyone knows about the 8 month old threads with hundreds of messages and dozens of completely unrelated subjects as well as the other extreme of a single conversation divided into dozens of threads with the super informative subject lines of Re: and Fwd:. Would be great if Kmail would support fixing those.

  5. Some guy from the internets

    Hello, I use KMail but I don’t know what is ” threading by Subject,”

    For helping you, first you have to help me, hehe sorry!

  6. Andreas

    Well it would say it is broken for quite some time. I have a folder for Bugzilla mails. If you don’t get the first mail which opens the bug and you subscribe later. The mails aren’t threaded at all …

    Here is an example of KDE Bugzilla-Mails:

    http://min.us/i/blMHsNY3qsoN1

  7. moltonel

    Github threads break if the thread leader is too old.

    Note that they all seem to have the same reference, so in theory kmail should be able to find the thread leader (or fake one) without looking at the subject. The thread would look different (very flat), but that would arguably be more correct, and in any cas not break my “see all messages from this thread in one place” workflow.

  8. Serafean

    You mean you won’t accept RFC6214 compliant datagrams?
    I Don’t see any issues here, but most of my mails are mailing lists.

    • RFC6214 seems to be about adoption of IPv6, we are not probably going to support that ;-)

      I know there are some RFCs about how to do threading, we will definitely look into them, but I can’t promise 100% exact and full implementation.

  9. Shayne

    Threading without subject completely breaks threads for me. I’m on a GroupWise server (not for much longer Google willing), and every single thread is gone after removing subject.

  10. Tim

    It breaks some threads. Most threads affected are from bug reports or ticket systems. Some are normal replies from Thunderbird with correct references in them (so I think it could be a bit better even without subject threading).

  11. Kanwar

    My default since email offered threading is by subject. I use it at work and it helps my search greatly as most often I need to search by subject or keywords therein. Please keep this option, even if its not the default. Thank you.

  12. Just posting to say there was a warning on the page:

    Warning: Missing argument 2 for wpdb::prepare(), called in /srv/http/www/p/progdan.cz/www.progdan.cz/wp-content/themes/grey-opaque/functions.php on line 2536 and defined in /srv/http/www/p/progdan.cz/www.progdan.cz/wp-includes/wp-db.php on line 992

  13. Till Adam

    Disclaimer: I am extremely biased, as I was the original implementer of subject threading in KMail. It was one of my first KDE contributions, about 10 years ago. Coming from mutt, which did it right, I missed it in KMail and foolishly attempted to fix it. It was way harder than I thought, but very educational.

    That said, I still believe it is useful and quite necessary, mainly because some very widely used mailers like MS Outlook get threading headers wrong and also because the mechanisms that are needed for it are also used to implement threading by References headers. This allows threads to be sorted properly that have messages missing, like your own replies. As Aaron points out that’s very frequent with people who have sent mail folders.

    KMail has the second best threading in the world, I think, second only to mutt because that is faster (easier to do with a much simpler interface). I for one would hate to see it regress :).

  14. Laurent Pointal

    Break some of threads.

    Looks like its threads where there is a “hole” in the References: list. By example i can see that a reply of mine, for which the email is not in the “In” box (but in my “Sent” box), make the reply’s reply beginning a new thread.

    Hope this help.
    A+
    Laurent.

  15. Knut

    I think I never used it. At least I’m not using it now. And, as already has been said, threading by subject tends to group unrelated mails and makes them unfindable. Thus for me it would be okay if you kick it out.

  16. Eike Hein

    I love you! I think you may have just *fixed* my workflow. “by subject” was probably the reason KMail kept grouping multiple, chronologically quite separate conversations with the same subject in the same thread here, which drove me insane and never happened in other clients. I wish I had known about this sooner.

  17. Olaf the Lost Viking

    It works fine for me without threading by subject (only very few broken threads, not worth mentioning). If the memory hunger goes down it sounds like a nice price to pay for _me_. I did NOT check on bug reports though, as I am using the webinterfaces.

    The problem with the seperate “Sent” and “INBOX” mails can be prevented by simply storing your own mails in the same folders as the messages from other “participants”. For me, this seems to be much more usable than an unsorted “Sent”-folder. So I have a BCC in all my mails with me as the recepient. This leads to all mails going through my filters on the server and in the end sitting in the same folder as the original mail or future answers (independently of the client I used (KMail, K9Mail, web). Like that, I do always have full threads will all relevant mails.

  18. David

    It works fine for me.

    Regarding the separate sent box issue – wouldn’t this not be a problem if Akonadi is doing the threading? Because it would be able to construct the threads over all folders?

  19. Mat

    I actually would like threading per people. See the list of people who last contacted me, descending. Click on a name to see the list of emails from that person.

  20. Sergey Salnikov

    It breaks some of my threads. It’s difficult to find any regularity because e-mails in those threads are from many different people.

  21. Zoran Dimovski

    I have a large amount of mail and everything seems to be fine. Don’t have problem with the threading.

  22. I just disabled it and it barely removes any threads on a almost 100% gmail powered inbox, as everybody else uses gmail to send me messages. However, reading the many people that do get broken threads – for instance due Outlook – I believe its still necessary to keep it as Kmail has indeed the best threading in the world. I would use no other client just because of the way mails are threaded.

    However, there should be some way to prevent fulltext indexes or another reason why you can’t simplify the implementation and move it into Akonadi. There must be a good trade-off here to keep both.

  23. Jonathan Marten

    As others have mentioned, there are problems with threading where the thread starter message is not available, especially Bugzilla mails where the user has joined the bug late. If subject threading were retained in some form and worked in this case, it would be most useful. See bugs 129973 and 279115.

  24. Fabian

    It fixes replys to bulk mails for me but also breaks some threads, but not all.
    Mostly of the same person which I don’t like anyway.

  25. Works fine, but the more concerning issue is the frequent hanging on folders :/
    There are numerous threads around kde forums etc and some are quite dated. Still that seems to have no solution.

  26. Bob Waddington

    Really bad news for me.

    gmail correspondence threads were lost. Two replies to the same email (from my kmail) from the same person within a few minutes of each other were treated differently.

  27. R

    It broke threading for me on an email which I believe was sent from gmail’s web interface.
    Relevant header: In-Reply-To:

    I’m in favour of adding support for manually combining emails into threads, esp. if it uses the Filter mechanism (in addition to just drag and drop).

  28. Happy Heyoka

    Toggling between the two flavours of “Activity by Date” breaks threading for me without including Subject.
    People above suggested “all gmail” based threads worked – but I have some that didn’t…

    I could live with that if the benefit outweighed the cost of keeping ‘Subject’.

    As an aside, please consider the idea that maybe part of the problem is UX of driving the whole aggregation thing.

    One example: you go to “Configure…” in the “Aggregation” sub-menu, clone a new item, change it, and exit the dialog box. The new form of aggregation is _not_ now the selected method of aggregation… you have to go back to the “Aggregation” sub-menu and select it. I would argue a user makes a new form of aggregation in order to _use it right away_ far more often than not.

    Also, maybe a non-modal live “aggregation controller” – so the currently displayed message list changes appearance as you change the method (eg: like a live filter preview in a photo editor).

    My point – fantastic functionality is of no use if ux is very complicated.

    anyway, I know, code or stfu :-)

  29. Morinehtar

    Sorry, but that’s a bad idea; breaks lots of threads :-(
    Why would you want to remove a working feature in the first place ?

  30. dhardy

    Breaks quite a few “threads” for me. I quote “threads” because quite often I delete or archive the less interesting mails in a thread besides not having my own emails in the inbox, so many “threads” in my inbox are very incomplete.

Comments are closed.