PHP Memory Allocation Blues

As a Systems Administrator and Programmer, I’ve determined it’s nearly impossible to properly determine the system requirements for a PHP based application under a given load. Short of using a special PHP module to profile the application it’s very hard to figure out where memory goes, and because code is loaded and unloaded as it’s requested it’s very hard to predict memory utilization at any given time.

For example, one request may only need a very small subset of code, and only need, say 4 MB of memory. In fact, a high percentage of requests probably do use a very small amount of memory. Great, that means we can run a lot of PHP threads and handle requests quickly.

But say your app is like WordPress, and the public facing side only uses a small amount of memory for the front end pages, but the administration system requires a significant amount more memory. Well, because code is loaded on the fly, every request will hit the disk and load up code and as it runs it will expand its memory footprint until the code is finished and then it will flush itself out again. This is great if you’re running a ton of applications on the same server that don’t get a lot of usage, because memory can be shared easily between applications as they’re used. But, if you’re looking to host a single high performance application, the disk I/O and wild memory fluctuations can be a real problem. For example, if you have an application that normally uses 16 MB of memory to render pages and you have 200 MB of memory available to run applications, one might permit as much as 10-12 PHP Processes. However, if some code loads and allocates 100 or more MB of memory and you have several PHP threads using 20 MB or so memory usage can far exceed the 200 MB available and your system will have to begin paging. Also, since each request loads and unloads all code, memory can be fluctuating in the neighborhood of hundreds of MB per second, so code that gets flushed to the page file can be immediately requisitioned again to close or perform some other function. This is referred to “Thrashing” and is usually considered an edge case.

In Java, for example, the server administrator can set a “heap size” for each application. This then allows the application and web server to work together to allow the application to use an appropriate amount of memory. In .net, IIS runs the web application in a process and automatically balances the memory available with the application and data size. This even works together well with other applications that are sensitive to memory pressure like SQL Server to ensure the best performance available for a given amount of memory and disk bandwidth.

Rails seems to have the best of both worlds. One can use Mod_Rails to use a more php-like hosting environment (but still different and better) for multiple applications on one server, or some form of a mongrel cluster for high performance applications.

If anyone’s interested I can do a more technical evaluation of the other platforms hosting environments, and the strength / weaknesses of both.