OpenX Blog

The Need for Speed

Tags: Developers

by radek.maciaszek on June 20th, 2007

As Scott previously mentioned one of our development tasks was to tweak every possible part of the Openads delivery engine to increase the performance. This article explains how we made our PHP application 50x faster.

Myself and another Senior Developer, Matteo Beccati, decided to work on this task. When we first started the latest Openads version was slower from his older brother by an order of magnitude. By the end of the exercise, our changes improved the speed of the delivery files by up to 50 times. It was very exciting to hit new performance record each day of the development cycle.

Don’t believe it when Matteo tries to convince you that we’re magicians. The truth is that we just followed a few simple rules described below.

* Rule number one: Forget about so-called “common sense” and trust only a profiler. PHP has quirks and sulks and there are things you need to know in order to pacify this beast. We have tried many different ideas. Some of them seemed to make sense at a first glance but then turned out to be a complete performance disaster. On the other hand some changes which we would never suspect to be important improved performance a lot. Fortunately, the profiler gave us a helping hand here and provided us with hints so that we knew what to focus on to improve the speed.

* Rule number two: Use the minimalist approach. This rule seems to be obvious but surprisingly often programmers do things in a complex way. We threw away a lot of code. For a start we replaced PEAR::Cache_Lite with our custom simple caching system. As a result the speed improved.. ten times.

* Rule number three: Optimize for real life scenarios. We focused on the speed for the most demanding users. For example, we used opcode caching all the time to ensure that we don’t try to improve things which are already improved by an accelerator itself.

Many people like the colorful and good-looking profiler KCacheGrind (used with XDebug). I need to say that although it looks nice, we chose the APD. All the performance tests analyzed with the KCacheGrind were very unreliable as each time we generated the report, it was different.

The statistics were very interesting and we learned some things about PHP from them. It seems that when we started more than 80% of the delivery time was wasted just on including files. In other words, PHP didn’t do anything useful in that time. Reducing the number of included files gave us such a good performance boost that we even decided to write a delivery merger tool to put all the code from included files together into a single file. Using a more technical term: we wrote a custom Java Ant task which automatically executes after every commit as a SVN post-commit hook and generates delivery files.

Another good example of a PHP “quirk” is the way PHP handles constants. It was one of the major factors affecting performance. Just removing all the constants allowed us to improve the performance by almost 2x (we left one constant to be precise). More useful tips and detailed reports can be found on our wiki page.

After all that fun, our Openads 2.3 release, even with all the new features, is now faster than Openads 2.0 (known previously as PhpAdsNew).

Community updates

14 Comments »

nice post - looking forward to using 2.3…

from the wiki page:

Real User System secs/ cumm %Time (excl/cumm) (excl/cumm) (excl/cumm) Calls call s/call Memory Usage Name -------------------------------------------------------------------------------------- 28.1 0.00 0.00 0.00 0.00 0.00 0.00 2 0.0000 0.0000 0 parse_ini_file

looks like caching the config would be a big win?

Comment by justin — June 21, 2007 @ 3:17 am

Actually we tried this idea. The difference between parsing ini file with “parse_ini_file()” and caching it as a PHP array is something like 1% only. See rule number 1 and a comment about common sense.
It still could be improved by about 14% though. As you probably noticed there are two calls to parse_ini_file and this could (and will) be lowered to only one.

Comment by radek.maciaszek — June 21, 2007 @ 9:14 am

Hi!

This is avery interesting blogpost. We did very similar things for our site http://de.alinki.com/ with very similar results. We also wrote our own cachehandler to speed things up. Due to the fact that our sites usually do not change for a very long time, we were able to get another performance boost in delivery by bypassing the whole application when a cache is found. Caching and Delivering gzip-compressed contents also improved performance because transfered data became much smaller and http-connections are closed faster. Most Browsers accept gzip compressed contents so we have a compressed and a uncompressed version in our cache and simply pass the right one through.

We’ll definitively try your “merger tool”-idea for our application. And we’ll also try to strip off whitespaces and comments to reduce pasing times.

Comment by Tobias Schwarz — June 21, 2007 @ 9:47 am

Hi Tobias In fact I din’t notice any gain in speed after removing whitespaces and comments. The only reason we removed them was to reduce the size of merged files.

Comment by radek.maciaszek — June 21, 2007 @ 10:25 am

Actually we tried this idea. The difference between parsing ini file with “parse_ini_file()” and caching it as a PHP array is something like 1% only.

interesting. how do you cache it between requests? in memory or to a file?

Comment by justin — June 21, 2007 @ 12:04 pm

Hi Justin. We only used files for caching but as I’m typing this Matteo just added a new ticket for it (https://developer.openads.org/ticket/658) to our track. We are thinking about using shm and memcached.

Comment by radek.maciaszek — June 21, 2007 @ 2:00 pm