Bug in parsing email body

For one specific message, the Email Message Parser is corrupting the message body. I will paste the raw email content below (can't see how to send as an attachment).

It's just a plain text email (Content-Type: text/plain; charset="big5").

If I decode the base64 body, the message looks ok, and is also displayed ok in Thunderbird email client. However the output from function Decode only contains the first 40 characters of the body as they should be - after this it's a load of garbled characters.


From [email protected] Mon Mar 19 00:35:02 2018
Received: from mail-vk0-f41.google.com ([209.85.213.41]:42303)
by s.orgtwo.com with esmtps (TLSv1.2:ECDHE-RSA-AES128-GCM-SHA256:128)
(Exim 4.89_1)
(envelope-from <[email protected]>)
id 1exilR-0000ss-Vx
for [email protected]; Mon, 19 Mar 2018 00:35:02 +0000
Received: by mail-vk0-f41.google.com with SMTP id y127so9381286vky.9
for <[email protected]>; Sun, 18 Mar 2018 17:34:41 -0700 (PDT)
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20161025;
h=x-gm-message-state:delivered-to:envelope-to:delivery-date
:dkim-signature:from:to:subject:thread-topic:thread-index:date
:message-id:references:in-reply-to:accept-language:content-language
:spamdiagnosticoutput:spamdiagnosticmetadata
:content-transfer-encoding:mime-version;
bh=E4pQwxHc5m8xkbktsR5qOhnAiABltwXSRt8Q7wzGwP0=;
b=rjglObHm8abzf/Q2DZ8erOFfbwfpu0ToI7RRhqn2ONE/ugIExH7Q1VLDnY+0NmKx1c
p/cVPt0y0FTTlywnAyUZTadD8bPcdRKcwdhGOFKoC9fsz+q4mYARApZSc5Ru/YEwj+kK
P9qN5poty+bgBGpRAqEdD7YbJTufERwWUQ17PxYCd/Z3hH5ALq4nEB6CiUhDynC+AAYs
2W5qu36vmGquQSCtMuxrfqdSW7KCVYeq+ysnaItbChDS8faEQAvBCJq1RwbDjyjOx9fw
05hiKhESEbpsFkxQdD4Q5mLIHzKRzXeGm+SC8XoMVo2k/BhJreekc4IjZsKDjSDXP1W4
bjTw==
X-Gm-Message-State: AElRT7GZc6md3LoM5SQCfGaRfEDU6gDOiyZUaDzFe9k31TnLHLBP26jW
AT9TnmuViVA7cBBEXUeulyvde6k1JXLe1udAwXaf7S5S5JwVRN4=
X-Received: by 10.31.173.14 with SMTP id w14mr6889845vke.74.1521419660593;
Sun, 18 Mar 2018 17:34:20 -0700 (PDT)
X-Forwarded-To: [email protected]
X-Forwarded-For: [email protected] [email protected]
Delivered-To: [email protected]
Received: by 10.176.9.65 with SMTP id c1csp1069175uah;
Sun, 18 Mar 2018 17:34:19 -0700 (PDT)
X-Google-Smtp-Source: AG47ELsLqdDlR9zHqEN2KDGTOnkQTQTGEYyqA0tKoFYxFfZGvlLfSiHbPyhyi+BOKRJQ3sEiDisgtGb3d2c=
X-Received: by 10.31.157.6 with SMTP id g6mr6313220vke.114.1521419659218;
Sun, 18 Mar 2018 17:34:19 -0700 (PDT)
Authentication-Results: mx.google.com;
spf=pass (google.com: domain of [email protected] designates 40.92.6.37 as permitted sender) [email protected];
dkim=pass [email protected] header.s=selector1 header.b=E0EtYs1A
Received-SPF: pass (google.com: domain of [email protected] designates 40.92.6.37 as permitted sender) client-ip=40.92.6.37;
Received: by 10.31.6.143 with POP3 id 137mf12305142vkg.0;
Sun, 18 Mar 2018 17:34:19 -0700 (PDT)
X-Gmail-Fetch-Info: [email protected] 1 mail.groundology.com 995 [email protected]
Received: from s.orgtwo.com
by s.orgtwo.com with LMTP id qD3sLXIFr1r6DAAAkaw/ug
; Mon, 19 Mar 2018 00:33:54 +0000
Received: from mail-oln040092006037.outbound.protection.outlook.com ([40.92.6.37]:6137 helo=NAM03-BY2-obe.outbound.protection.outlook.com)
by s.orgtwo.com with esmtps (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256)
(Exim 4.89_1)
(envelope-from <[email protected]>)
id 1exikM-0000qI-I5
for [email protected]; Mon, 19 Mar 2018 00:33:54 +0000
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hotmail.com;
s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
bh=E4pQwxHc5m8xkbktsR5qOhnAiABltwXSRt8Q7wzGwP0=;
b=E0EtYs1AJZ0nlsyYZUkzTUeXzAmFBdqDm1rlQ/UTzDfrXTfaiSJhV80o6aWD+o+UtXjuFT8+FR1y2yrMkqaNYpGn6JCqfaHdFyNorMK/zEMUoO/xa+1GFm8flVdUFSVWqq/csX1a+h+irbOubL2oERTglvhz9sZVjOgvhobXZz5sap59tdZaCusC72QmGm9HCse2CyLrmsNdYsdfB+tC0NSSRymLSlZcUhlG3v/RbqzXAhUnTAZRMrJ0CqKTcVvD1B//09KVsu+qXQOLWF7EM7RchiTfKfBb4u2oOM2416bKe1OCeTYzBlcQxae9QUs5RAiquLKynS91CCV6X8tO3g==
Received: from BY2NAM03FT025.eop-NAM03.prod.protection.outlook.com
(10.152.84.60) by BY2NAM03HT176.eop-NAM03.prod.protection.outlook.com
(10.152.85.195) with Microsoft SMTP Server (version=TLS1_2,
cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.567.18; Mon, 19 Mar
2018 00:33:12 +0000
Received: from CY4PR17MB0982.namprd17.prod.outlook.com (10.152.84.56) by
BY2NAM03FT025.mail.protection.outlook.com (10.152.84.232) with Microsoft SMTP
Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id
15.20.567.18 via Frontend Transport; Mon, 19 Mar 2018 00:33:12 +0000
Received: from CY4PR17MB0982.namprd17.prod.outlook.com
([fe80::b406:80ea:3558:df02]) by CY4PR17MB0982.namprd17.prod.outlook.com
([fe80::b406:80ea:3558:df02%14]) with mapi id 15.20.0588.016; Mon, 19 Mar
2018 00:33:12 +0000
From: Lawrence Yeung <[email protected]>
To: Groundology <[email protected]>
Subject: Re: Your order confirmation
Thread-Topic: Your order confirmation
Thread-Index: AQHTvtRnVht7fRkcK0Oemt4S38A2XKPWtg8z
Date: Mon, 19 Mar 2018 00:33:11 +0000
Message-ID: <CY4PR17MB0982C4222CC87B88E0B72490F5D40@CY4PR17MB0982.namprd17.prod.outlook.com>
References: <[email protected]>
In-Reply-To: <[email protected]>
Accept-Language: en-US
Content-Language: zh-TW
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-incomingtopheadermarker: OriginalChecksum:0449E72958C33850178804A961CB596B99AF31DC8D9E6B4FA280A550F304FCD6;UpperCasedChecksum:71E351BD3CDFFDB2ABAF17091E1760B733BA1D2072D2772C6F9C79976BB459F9;SizeAsReceived:6969;Count:46
x-ms-exchange-messagesentrepresentingtype: 1
x-tmn: [4F5P6lIt3BVobJyHa9g4NcMCW7Mqq8C9]
x-ms-publictraffictype: Email
x-microsoft-exchange-diagnostics: 1;BY2NAM03HT176;6:rfgzQxhjrHkEtIlg2XAffjy8TSd1NPt62mmZGQW3IpcJbvKTW91YxZ/t8g1MfuCs24hoVdbw/P7GcaZjzUTcRKIoUMXrgSoFHyuDN72BEy20422sMY3Td2oZD3u3/e7yptxCzrf1FQR6iYcQClrE/95Qqdn6RVbRKkXqgqpTN9Wv4EaBusGWxy/80jbMn6dj8YlT3DNHlBqfJxwQdu8BRddV5rRkxgTJmuaPRxNHrzloHTacI2OYatIB2JvbLjGw9VusN2Mzgv0It2vGv4ZBIGNVXYxj9UyEsreqp5zjnSBeoDQrdL4gh2Zk/Tj99lYGVGvHqheSiT4MDaJjkxLgJlrxRaZg7SIEZ0dNLzCwayI=;5:1EWcFbZnEn0El7ZAgPuECGY4wLmcI1nQg57dTbrg7UmUHNIGyb2wX9+UrM2/tfv2AcuybPc0hZVR3GMyo0Qgzjs2IC3Ey9PXTwWQX416XwXVRuqmp9a/moRYE8FWKZP0eTZeMPuoYYUA6k0jTO3sYK8Dl2tWNqXb/mId+d2XiGc=;24:mJ+eVf4G+lCb1KSfUafUfRTspkctVTPyGSFPTEZ9q4mhUFgT83CbR0RmpJGvZx8Un1yQghqTltJsjOQ2m95LdxbbREVQy1DpFHNLqRBMEUw=;7:BTdF3bbWBw29V9kH5lZYrQf+EcFALVrtG6qLnNTJ9QCB4ZV23vEpyQqM6oSqrwBRpCXCZh6w7ajDtSHexFFGsMd4Msf6crgYmLV92ayPvw530uA+QX3ZDOsIDY97mmcfbqMHt5rRH+eilHTlLq/OdS0k/2OT/vDv2OXxgNwmaTe/DQg3QRiqhSuSw10oDEwCjhk+ZvvkmR6kNS5B8yvNpu7xB7Ua2oioDg33u92T2GvokkqtPd2bhAuYmeZcPygU
x-incomingheadercount: 46
x-eopattributedmessage: 0
x-microsoft-antispam: UriScan:;BCL:0;PCL:0;RULEID:(7020095)(201702061078)(5061506573)(5061507331)(1603103135)(2017031320274)(2017031324274)(2017031323274)(2017031322404)(1601125374)(1603101448)(1701031045);SRVR:BY2NAM03HT176;
x-ms-traffictypediagnostic: BY2NAM03HT176:
x-ms-office365-filtering-correlation-id: 95d714ee-1e6f-41ac-122e-08d58d30ff18
x-exchange-antispam-report-cfa-test: BCL:0;PCL:0;RULEID:(444000031);SRVR:BY2NAM03HT176;BCL:0;PCL:0;RULEID:;SRVR:BY2NAM03HT176;
x-forefront-prvs: 06167FAD59
x-forefront-antispam-report: SFV:NSPM;SFS:(7070007)(98901004);DIR:OUT;SFP:1901;SCL:1;SRVR:BY2NAM03HT176;H:CY4PR17MB0982.namprd17.prod.outlook.com;FPR:;SPF:None;LANG:;
x-microsoft-antispam-message-info: J5QXbNlxQz2EFU+qrGS/kbocY86cOfCpNr68D5OHUAA6ByrpbswpaeX0zH6pmkaZOrqx3gRR91P3XNRn9szNh7uqeP+4HGjA31Ctbm2FET0bmeRznuoD56qcLrE3X6F0uyPbYzz0pJ8Dat+Gej4RyNwcOG4FSzQbY9FzPF4UsvkjJbT9PQFLIc+HdOrxO3O5
spamdiagnosticoutput: 1:99
spamdiagnosticmetadata: NSPM
Content-Type: text/plain; charset="big5"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: hotmail.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 95d714ee-1e6f-41ac-122e-08d58d30ff18
X-MS-Exchange-CrossTenant-originalarrivaltime: 19 Mar 2018 00:33:11.8476
(UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Internet
X-MS-Exchange-CrossTenant-id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY2NAM03HT176

SGksDQoNClBscyBsZXQgbWUga25vdyB0aGUgdHJhY2tpbmcgbnVtYmVyIHdoZW4gaXQgd2FzIGNy
ZWF0ZWQuICBUaHggYSBsb3QuDQoNClJlZ2FyZHMsDQpMYXdyZW5jZQ0KDQqxcafaqrogaVBob25l
ILbHsGUNCg0KPiBHcm91bmRvbG9neSA8Y29udGFjdEBncm91bmRvbG9neS5jb20+IKnzIDIwMTim
fjOk6zE5pOkgMDA6MTYgvGe5RKFHDQo+IA0KPiBEZWFyIGxhd3JlbmNlIHlldW5nLA0KPiANCj4g
VGhhbmsgeW91IGZvciB5b3VyIG9yZGVyIG9mOg0KPiBORVcgUGx1c2ggUGFkIChVSykNCj4gRXh0
ZW5zaW9uIGNvcmQgLSAzbQ0KPiBFYXJ0aCBjb25uZWN0aW9uIHBsdWcgKFVLKSB4Mg0KPiBEZWxp
dmVyeSBieSBSb3lhbCBNYWlsIFRyYWNrZWQgNDgNCj4gDQo+IERlbGl2ZXJ5IGFkZHJlc3M6DQo+
IGxhd3JlbmNlIHlldW5nDQo+IEJ1eWlwcGVlIGMvbyBUcmFja2JheQ0KPiA1MDEgTWV0cm9wbGV4
IEJ1c2luZXNzIFBhcmsNCj4gQnJvYWR3YXksICBNMDAzODk1MzENCj4gU2FsZm9yZA0KPiBHcmVh
dGVyIE1hbmNoZXN0ZXINCj4gTTUwIDJVRQ0KPiBVbml0ZWQgS2luZ2RvbQ0KPiANCj4gUGhvbmU6
IDE2MSA4ODQgMTI0Ng0KPiANCj4gWW91ciBpbnZvaWNlIGlzIGF0dGFjaGVkLg0KPiANCj4gSWYg
eW91IG5lZWQgdG8gbWFrZSBhbnkgY2hhbmdlcyB0byB5b3VyIG9yZGVyLCBwbGVhc2UgY2xpY2sg
dGhlIGxpbmsgYmVsb3c6DQo+IGh0dHBzOi8vd3d3Lm9yZ3R3by5jb20vZ3JvdW5kb2xvZ3lsdGQv
Y2hhbmdlb3JkZXIucGhwP3NhbGVpZD0yNzY3NiZpbnZpZD0zMzU2NyZlbWFpbD1sYXdyZW5jZTA2
MjQlNDBnbWFpbC5jb20NCj4gDQo+IE5vdGUgdGhhdCBjaGFuZ2VzIG1heSBub3QgYmUgcG9zc2li
bGUgb25jZSB5b3VyIG9yZGVyIGhhcyBiZWVuIHBhY2tlZCwgc28gZG8gcGxlYXNlIGNoZWNrIHlv
dXIgb3JkZXIgbm93Lg0KPiANCj4gVGhhbmsgeW91IGZvciB5b3VyIGN1c3RvbS4NCj4gDQo+IEtp
bmQgUmVnYXJkcywNCj4gVGhlIEdyb3VuZG9sb2d5IHRlYW0NCj4gPEludm9pY2UucGRmPg0K

Came across another message which triggers the same bug. The problem occurs with plain text (single part) base64 encoding messages.

The problem is the base64 encoded block being split at random means that it can't be decoded properly.

Increasing $message_buffer_length larger than the whole message makes the problem go away.

Let me know if you need more sample emails that trigger this bug.

Sorry for the delay. Can you send me that message via email?

Hi Manuel,

I sent the raw email data to you by email. With this particular email, the bug is triggered when $message_buffer_length has the default value of 8000.

Seb