Implemented thread merging

It can happen that thread messages are not delivered in order, which
means we will have to merge threads once all messages are available.

1 files changed, 283 insertions, 0 deletions

diff --git a/tests/threaddata/thread9 b/tests/threaddata/thread9
new file mode 100644
index 0000000..c17e7fd
--- /dev/null
+++ b/tests/threaddata/thread9
@@ -0,0 +1,283 @@
+Return-Path: <kde-community-bounces@kde.org>
+Received: from imapb010.mykolab.com ([unix socket])
+         by imapb010.mykolab.com (Cyrus 2.5.10-49-g2e214b4-Kolab-2.5.10-8.1.el7.kolab_14) with LMTPA;
+         Thu, 17 Aug 2017 18:30:41 +0200
+X-Sieve: CMU Sieve 2.4
+Received: from int-mx001.mykolab.com (unknown [10.9.13.1])
+        by imapb010.mykolab.com (Postfix) with ESMTPS id 92811150C3DE0
+        for <christian@mailqueue.ch>; Thu, 17 Aug 2017 18:30:41 +0200 (CEST)
+Received: from mx.kolabnow.com (unknown [10.9.4.3])
+        by int-mx001.mykolab.com (Postfix) with ESMTPS id 7973B11D
+        for <christian@mailqueue.ch>; Thu, 17 Aug 2017 18:30:41 +0200 (CEST)
+X-Virus-Scanned: amavisd-new at mykolab.com
+Authentication-Results: ext-mx-in003.mykolab.com (amavisd-new);
+        dkim=pass (1024-bit key) header.d=kde.org header.b=q4j4OOKP;
+        dkim=fail (2048-bit key) reason="fail (message has been altered)"
+        header.d=gmail.com header.b=DJRXq7Se
+X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0
+Received: from forward1-smtp.messagingengine.com (forward1-smtp.messagingengine.com [66.111.4.223])
+        by ext-mx-in003.mykolab.com (Postfix) with ESMTPS id 3E01E292D
+        for <christian@mailqueue.ch>; Thu, 17 Aug 2017 18:30:29 +0200 (CEST)
+Received: from mailredirect.nyi.internal (imap36.nyi.internal [10.202.2.86])
+        by mailforward.nyi.internal (Postfix) with ESMTP id 89BC31A23
+        for <christian@mailqueue.ch>; Thu, 17 Aug 2017 12:30:28 -0400 (EDT)
+Received: by mailredirect.nyi.internal (Postfix, from userid 501)
+        id 6EC348E9B6; Thu, 17 Aug 2017 12:30:28 -0400 (EDT)
+Received: from compute1.internal (compute1.nyi.internal [10.202.2.41])
+         by sloti36d2t28 (Cyrus fastmail-fmjessie44745-15358-git-fastmail-15358) with LMTPA;
+         Thu, 17 Aug 2017 12:30:28 -0400
+X-Cyrus-Session-Id: sloti36d2t28-3239059-1502987428-2-12164706007640762698
+X-Sieve: CMU Sieve 3.0
+X-Spam-known-sender: no
+X-Orig-Spam-score: 0.0
+X-Spam-hits: BAYES_00 -1.9, HTML_MESSAGE 0.001, RCVD_IN_DNSWL_MED -2.3,
+  RP_MATCHES_RCVD -0.001, SPF_PASS -0.001, LANGUAGES en, BAYES_USED global,
+  SA_VERSION 3.4.0
+X-Spam-source: IP='46.4.96.248', Host='postbox.kde.org', Country='DE', FromHeader='org',
+  MailFrom='org'
+X-Spam-charsets: plain='UTF-8', html='UTF-8'
+X-Resolved-to: chrigi_1@fastmail.fm
+X-Delivered-to: chrigi_1@fastmail.fm
+X-Mail-from: kde-community-bounces@kde.org
+Received: from mx1 ([10.202.2.200])
+  by compute1.internal (LMTPProxy); Thu, 17 Aug 2017 12:30:28 -0400
+Authentication-Results: mx1.messagingengine.com;
+    dkim=fail (message has been altered; 2048-bit rsa key sha256) header.d=gmail.com header.i=@gmail.com header.b=DJRXq7Se;
+    dkim=pass (1024-bit rsa key sha256) header.d=kde.org header.i=@kde.org header.b=q4j4OOKP;
+    dmarc=none (p=none;has-list-id=yes) header.from=kde.org;
+    spf=pass smtp.mailfrom=kde-community-bounces@kde.org smtp.helo=postbox.kde.org;
+    x-google-dkim=fail (message has been altered; 2048-bit rsa key) header.d=1e100.net header.i=@1e100.net header.b=U7Pdj/LB
+Received-SPF: pass
+    (kde.org: 46.4.96.248 is authorized to use 'kde-community-bounces@kde.org' in 'mfrom' identity (mechanism 'mx' matched))
+    receiver=mx1.messagingengine.com;
+    identity=mailfrom;
+    envelope-from="kde-community-bounces@kde.org";
+    helo=postbox.kde.org;
+    client-ip=46.4.96.248
+Received: from postbox.kde.org (postbox.kde.org [46.4.96.248])
+        (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
+        (No client certificate requested)
+        by mx1.messagingengine.com (Postfix) with ESMTPS
+        for <chrigi_1@fastmail.fm>; Thu, 17 Aug 2017 12:30:27 -0400 (EDT)
+DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=kde.org; s=default;
+        t=1502987424; bh=P5C3gzzCdP/NQgV0POjgD3g9Hpun4leANxLktFzWpbo=;
+        h=In-Reply-To:References:From:Date:Subject:To:Reply-To:List-Id:
+         List-Unsubscribe:List-Archive:List-Post:List-Help:List-Subscribe:
+         From;
+        b=q4j4OOKPbM6MLGfTlM1WlnmVrh2PfQSKPYcLEoHUjBwoiu+oacbJb5cxmPkadvddx
+         MIYmJyog8F4NCNZCIi5vzNkit8vaUJgHws3pk+0uIFo9SdOBkFBfTXSGsDBWB2AdL5
+         wryEwxZKOqEDcECpTNEEmQykU3MYwLBw7sD+KJjY=
+X-Original-To: kde-community@kde.org
+X-Remote-Delivered-To: kde-community@localhost.kde.org
+Received-SPF: Pass (sender SPF authorized) identity=mailfrom;
+ client-ip=2607:f8b0:4003:c06::232; helo=mail-oi0-x232.google.com;
+ envelope-from=kexipl@gmail.com; receiver=kde-community@kde.org 
+Authentication-Results: postbox.kde.org; dkim=pass
+ reason="2048-bit key; unprotected key"
+ header.d=gmail.com header.i=@gmail.com header.b=DJRXq7Se;
+ dkim-adsp=pass; dkim-atps=neutral
+Received: from mail-oi0-x232.google.com (mail-oi0-x232.google.com
+ [IPv6:2607:f8b0:4003:c06::232])
+ by postbox.kde.org (Postfix) with ESMTPS id A4F67A014D
+ for <kde-community@kde.org>; Thu, 17 Aug 2017 16:30:09 +0000 (UTC)
+Received: by mail-oi0-x232.google.com with SMTP id f11so71799456oic.0
+ for <kde-community@kde.org>; Thu, 17 Aug 2017 09:30:09 -0700 (PDT)
+DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
+ h=mime-version:sender:in-reply-to:references:from:date:message-id
+ :subject:to; bh=P5C3gzzCdP/NQgV0POjgD3g9Hpun4leANxLktFzWpbo=;
+ b=DJRXq7SewhEpbavrA6kFk0TPbF9526gl1WpH2O4R7hpuM5tJVqLoT4b53UfmZyGeDw
+ pSdW599ZTY3WLsK29IZ5buua1TgJeSLgN+PWKfTJAFW7qAZaJo6pRIpqSgETEEk/BdMc
+ KtqYdBD/IkwUVx5LAuQikyNn1HrKbti/tbc/YiI23f5TRxfIQZb7DOvOaAi1bZO8jEFq
+ 5EHEVcrjvIR2S4HHWxen9rZvGIotVN3womdK8b0t+Wx+Kt0qv06px9jNF0mTqLKhCJAz
+ los9Tpv/7RI0JiQyfPzl7kMQjU3i/pyA1u6b6t69ALfUQcjv25NcwhSaQbWIi9DN8rLg
+ Lc7g==
+X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
+ d=1e100.net; s=20161025;
+ h=x-gm-message-state:mime-version:sender:in-reply-to:references:from
+ :date:message-id:subject:to;
+ bh=P5C3gzzCdP/NQgV0POjgD3g9Hpun4leANxLktFzWpbo=;
+ b=U7Pdj/LBlwGbtXVpnpJudVk938b0fKTIRhK+Lc9IfB9zZhXh3FGwX5kWSUObEOT3hX
+ BXRk0cWvJYHpuGUnLXx/Wjen3j6283GHvDvPhfagyTZbGJohkRMEkxLwFh4ZsJ87M71t
+ pqLLayDjqDHj5jVuko5TDPTtRL8mjzPM7r0DQKu1GYkYtiNLE5JlGR9OsqK8ZH78Wkf8
+ PWUT2BD+mkOE03gFEYpTA0oQW1iwv+wN5xySzaUOlBVfxUUx69EOLnFuRthkQHXcnlGG
+ rchW44D/eiSVU7JWK1Tk2IKNK+ERiq2/zftSmKRzpbwfv6D8De0PZOJPyi89kS9t2I2L
+ ar9g==
+X-Gm-Message-State: AHYfb5g28N+JHV5G6R5j0X0hpMFpCnu/TuLNw/idrsMKyvGOUXdbQiIn
+ whIAqO9js0sL5H92k3yqqJIGhuicWA==
+X-Received: by 10.202.108.130 with SMTP id h124mr8149045oic.289.1502987407944; 
+ Thu, 17 Aug 2017 09:30:07 -0700 (PDT)
+MIME-Version: 1.0
+Received: by 10.182.45.227 with HTTP; Thu, 17 Aug 2017 09:29:27 -0700 (PDT)
+In-Reply-To: <5A696707-744C-4035-A8FA-CA83EE8691D6@kde.org>
+References: <2048912.XfIJe3ZSdj@vkpc5> <2990543.KVDkBByYO0@minixfox>
+ <CACpu024EH1OeDqwL94QK33eq4sCGjKjwedcQDR_PWjprBevzfg@mail.gmail.com>
+ <5231282.Ch11jfsTMl@lenovo> <B95D7D2A-23A9-4245-AAC0-8A3FAE330090@kde.org>
+ <5A696707-744C-4035-A8FA-CA83EE8691D6@kde.org>
+From: Jaroslaw Staniek <staniek@kde.org>
+Date: Thu, 17 Aug 2017 18:29:27 +0200
+X-Google-Sender-Auth: LxL4QEJfN3UTITM2I0VbgyX7420
+Message-ID: <CAOj7QQ0WsTCfu9hoq+DTKMGTW=+KObo7HG2_YU1QZ6eOGwQbAQ@mail.gmail.com>
+Subject: Re: Telemetry Policy
+To: informing about and discussing non-technical community topics
+ <kde-community@kde.org>
+Content-Type: multipart/alternative; boundary="001a1142e7548d73010556f58604"
+X-BeenThere: kde-community@kde.org
+X-Mailman-Version: 2.1.16
+Precedence: list
+Reply-To: informing about and discussing non-technical community topics
+ <kde-community@kde.org>
+List-Id: informing about and discussing non-technical community topics
+ <kde-community.kde.org>
+List-Unsubscribe: <https://mail.kde.org/mailman/options/kde-community>,
+ <mailto:kde-community-request@kde.org?subject=unsubscribe>
+List-Archive: <http://mail.kde.org/pipermail/kde-community/>
+List-Post: <mailto:kde-community@kde.org>
+List-Help: <mailto:kde-community-request@kde.org?subject=help>
+List-Subscribe: <https://mail.kde.org/mailman/listinfo/kde-community>,
+ <mailto:kde-community-request@kde.org?subject=subscribe>
+Errors-To: kde-community-bounces@kde.org
+Sender: "kde-community" <kde-community-bounces@kde.org>
+
+--001a1142e7548d73010556f58604
+Content-Type: text/plain; charset="UTF-8"
+Content-Transfer-Encoding: quoted-printable
+
+On 17 August 2017 at 18:20, Thomas Pfeiffer <thomas.pfeiffer@kde.org> wrote=
+:
+
+>
+> On 17. Aug 2017, at 17:38, Mirko Boehm - KDE <mirko@kde.org> wrote:
+>
+> Hi,
+>
+> On 17. Aug 2017, at 01:46, Thomas Pfeiffer <thomas.pfeiffer@kde.org>
+> wrote:
+>
+> Hi Valorie,
+> Even if opt-out for some data is legally and even morally fine, it does n=
+ot
+>
+> align with the values we communicate to our users:
+> Unlike Mozilla's Mission, our Vision mentions privacy explicitly, and we'=
+re
+>
+> striving to make privacy our USP.
+>
+>
+> We seem to assume a contradiction between telemetry and privacy. I believ=
+e
+> this is a knee-jerk reaction. We can implement telemetry in a way that
+> privacy is not violated. In fact, I would say that it follows from our
+> vision that we should do this.
+>
+>
+> The problem is: I expect users to have the same knee-jerk reaction. I
+> don=E2=80=99t see us being able to explain to users that actually their p=
+rivacy is
+> perfectly safe before they freak out.
+> Privacy-minded Free Software users have freaked out in the past over
+> things which objectively speaking were not a huge deal.
+> It=E2=80=99s emotion more than rational arguments
+>
+>
+=E2=80=8BIt's hard to argue here or generalize to all app's communities. Kr=
+ita
+community for example is different than gcc community in these aspects.
+
+--=20
+regards, Jaroslaw Staniek
+
+KDE:
+: A world-wide network of software engineers, artists, writers, translators
+: and facilitators committed to Free Software development - http://kde.org
+Calligra Suite:
+: A graphic art and office suite - http://calligra.org
+Kexi:
+: A visual database apps builder - http://calligra.org/kexi
+Qt Certified Specialist:
+: http://www.linkedin.com/in/jstaniek
+
+--001a1142e7548d73010556f58604
+Content-Type: text/html; charset="UTF-8"
+Content-Transfer-Encoding: quoted-printable
+
+<div dir=3D"ltr"><div class=3D"gmail_default" style=3D"font-family:monospac=
+e,monospace;font-size:small"><br></div><div class=3D"gmail_extra"><br><div =
+class=3D"gmail_quote">On 17 August 2017 at 18:20, Thomas Pfeiffer <span dir=
+=3D"ltr">&lt;<a href=3D"mailto:thomas.pfeiffer@kde.org" target=3D"_blank">t=
+homas.pfeiffer@kde.org</a>&gt;</span> wrote:<br><blockquote class=3D"gmail_=
+quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1=
+ex"><div style=3D"word-wrap:break-word"><br><div><span class=3D""><blockquo=
+te type=3D"cite"><div>On 17. Aug 2017, at 17:38, Mirko Boehm - KDE &lt;<a h=
+ref=3D"mailto:mirko@kde.org" target=3D"_blank">mirko@kde.org</a>&gt; wrote:=
+</div><br class=3D"m_-9083494930210199564Apple-interchange-newline"><div><d=
+iv style=3D"word-wrap:break-word">Hi,=C2=A0<div><br><div><blockquote type=
+=3D"cite"><div>On 17. Aug 2017, at 01:46, Thomas Pfeiffer &lt;<a href=3D"ma=
+ilto:thomas.pfeiffer@kde.org" target=3D"_blank">thomas.pfeiffer@kde.org</a>=
+&gt; wrote:</div><br class=3D"m_-9083494930210199564Apple-interchange-newli=
+ne"><div><span style=3D"font-family:Menlo-Regular;font-size:11px;font-style=
+:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;t=
+ext-align:start;text-indent:0px;text-transform:none;white-space:normal;word=
+-spacing:0px;float:none;display:inline!important">Hi Valorie,</span><br sty=
+le=3D"font-family:Menlo-Regular;font-size:11px;font-style:normal;font-varia=
+nt-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;te=
+xt-indent:0px;text-transform:none;white-space:normal;word-spacing:0px"><spa=
+n style=3D"font-family:Menlo-Regular;font-size:11px;font-style:normal;font-=
+variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:sta=
+rt;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;=
+float:none;display:inline!important">Even if opt-out for some data is legal=
+ly and even morally fine, it does not<span class=3D"m_-9083494930210199564A=
+pple-converted-space">=C2=A0</span></span><br style=3D"font-family:Menlo-Re=
+gular;font-size:11px;font-style:normal;font-variant-caps:normal;font-weight=
+:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transfo=
+rm:none;white-space:normal;word-spacing:0px"><span style=3D"font-family:Men=
+lo-Regular;font-size:11px;font-style:normal;font-variant-caps:normal;font-w=
+eight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-tr=
+ansform:none;white-space:normal;word-spacing:0px;float:none;display:inline!=
+important">align with the values we communicate to our users:</span><br sty=
+le=3D"font-family:Menlo-Regular;font-size:11px;font-style:normal;font-varia=
+nt-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;te=
+xt-indent:0px;text-transform:none;white-space:normal;word-spacing:0px"><spa=
+n style=3D"font-family:Menlo-Regular;font-size:11px;font-style:normal;font-=
+variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:sta=
+rt;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;=
+float:none;display:inline!important">Unlike Mozilla&#39;s Mission, our Visi=
+on mentions privacy explicitly, and we&#39;re<span class=3D"m_-908349493021=
+0199564Apple-converted-space">=C2=A0</span></span><br style=3D"font-family:=
+Menlo-Regular;font-size:11px;font-style:normal;font-variant-caps:normal;fon=
+t-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text=
+-transform:none;white-space:normal;word-spacing:0px"><span style=3D"font-fa=
+mily:Menlo-Regular;font-size:11px;font-style:normal;font-variant-caps:norma=
+l;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px=
+;text-transform:none;white-space:normal;word-spacing:0px;float:none;display=
+:inline!important">striving to make privacy our USP.</span><br style=3D"fon=
+t-family:Menlo-Regular;font-size:11px;font-style:normal;font-variant-caps:n=
+ormal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent=
+:0px;text-transform:none;white-space:normal;word-spacing:0px"></div></block=
+quote><br></div><div>We seem to assume a contradiction between telemetry an=
+d privacy. I believe this is a knee-jerk reaction. We can implement telemet=
+ry in a way that privacy is not violated. In fact, I would say that it foll=
+ows from our vision that we should do this.</div><div><br></div></div></div=
+></div></blockquote><div><br></div></span>The problem is: I expect users to=
+ have the same knee-jerk reaction. I don=E2=80=99t see us being able to exp=
+lain to users that actually their privacy is perfectly safe before they fre=
+ak out.</div><div>Privacy-minded Free Software users have freaked out in th=
+e past over things which objectively speaking were not a huge deal.</div><d=
+iv>It=E2=80=99s emotion more than rational arguments</div><br></div></block=
+quote></div><br><div class=3D"gmail_default" style=3D"font-family:monospace=
+,monospace;font-size:small">=E2=80=8BIt&#39;s hard to argue here or general=
+ize to all app&#39;s communities. Krita community for example is different =
+than gcc community in these aspects.</div><div><br></div>-- <br><div class=
+=3D"gmail_signature" data-smartmail=3D"gmail_signature">regards, Jaroslaw S=
+taniek<br><br>KDE:<br>: A world-wide network of software engineers, artists=
+, writers, translators<br>: and facilitators committed to Free Software dev=
+elopment - <a href=3D"http://kde.org" target=3D"_blank">http://kde.org</a><=
+br>Calligra Suite:<br>: A graphic art and office suite - <a href=3D"http://=
+calligra.org" target=3D"_blank">http://calligra.org</a><br>Kexi:<br>: A vis=
+ual database apps builder - <a href=3D"http://calligra.org/kexi" target=3D"=
+_blank">http://calligra.org/kexi</a><br>Qt Certified Specialist:<br>: <a hr=
+ef=3D"http://www.linkedin.com/in/jstaniek" target=3D"_blank">http://www.lin=
+kedin.com/in/jstaniek</a></div>
+</div></div>
+
+--001a1142e7548d73010556f58604--