Decoding large attachments

Ex. message with 7.4MB mp3 file as an attachment:

Parsing this message gave me an:
Fatal error: Maximum execution time of 300 seconds exceeded in /var/www/..path.../mime_parser/mime_parser.php on line 855 (this can be different a little. i added chmod(path,0777) operation after a file where body data is going to be written is opened/created).

resource limits are pretty high:

max_execution_time (Maximum time in seconds a script is allowed to run before it is terminated) = 300 sec
max_input_time (Maximum time in seconds a script is allowed to receive input data) = 180 sec
memory_limit (Maximum of amount of memory in bytes that a script is allowed to allocate) = 100Mbytes

I get my messages from mysql database (dbmail v2.2).
So in the end i have a variable with full message and then pass it to mime_parser instance.

the largest mail i tested and got (slowly, but anyway) an output array was 6.6MB. I believe i could make execution time larger, but... i seek another solution. We have a pretty good and modern server here, so hardware should not be an issue.

Any suggestions, may be?

And i forgot to mention! Good work, Manuel! I'm sure that this class has a bright future. One of the things i like is that it does not use any PEAR library :)

I suggest that you save the message body parts to disk. Use the SaveBody parameter to specify a directory where the body parts are saved. That parameter is commented in the example script.

That will make the class use much less memory and hopefully use less CPU as with large messages it will be concatenating and reallocating memory a lot.

Also setting the mbox parameter to 0 will speedup parsing a lot, at least 4 times with large messages.

Our dev server config:

Apache/2.0.55 (Ubuntu) PHP/5.1.6 mod_ssl/2.0.55 OpenSSL/0.9.8b

I have tried different combinationsand and came to:

$this->mime->mbox = 0;
$this->mime->decode_bodies = 1;

$parameters['SkipBody'] = 1;
$parameters['SaveBody'] = CONF_TMP.$msg_id,
$parameters['Data'] = $msg_raw; //complete message is held in var

but execution time is still about the same.

before this class i've been using mime decoder in PEAR library and it works relatively well, but some mail gets parsed incorectly. Ex. recognition of inline image attachments is not very good.

i don't have a clue what to do next... to shorten the execution time...

fatal error concerning execution time points to this if statement:

if(GetType($line_break=strpos($this->body_buffer, $break="\r\n", $position))!='integer' && GetType($line_break=strpos($this->body_buffer, $break="\n", $position))!='integer' && GetType($line_break=strpos($this->body_buffer, $break="\r", $position))!='integer')
{
if($position > 0)
{
if(!$this->body_parser->Parse(substr($this->body_buffer, 0, $position), 0)) return($this->SetError($this->body_parser->error));
if(!$this->QueueBodyParts()) return(0);
}
$this->body_buffer = substr($this->body_buffer, $position);
$this->body_buffer_position = 0;
$this->body_offset += $position;
return(1);

}

The original design of the class was to make it feasible to parse arbitrarily large files without consuming excessive memory. I am sure that loading the whole message in memory and parse it a single string would be much faster, but than it may exceed your configured memory limits.

The class was also made a bit tolerant to different types of line breaks and eventual malformed messages. That is probably why you had problems with the PEAR class.

I am sure the parsing performance can be improved as I have not yet had the time and profile the class. If you have the time to do it and figure what code segments are causing most of the delays, that would help determining what is worth investing on further optimization.