Performance discussion

First we will have a look at just some points regarding performance and later on see some real-life figures.

Object orientation vs. performance

There are false rumours about various technologies having low performance. Java is considered slow even though many heavy-traffic sites run on Java and could never be run using standard PHP. In complex string handling, for instance, Java wins PHP about 20:1. The "poor performance" is a probably a myth caused by slowly loading applets. General OO is said to be slow as well. There was a time when OO was immature and the hardware not powerful enough - in the late 1980's. Some PHP web applications have a lousy performance. Funny enough all of the ones we have seen have been written in the old procedural manner. No OO. No templates. In all cases the problems have been caused by unprofessional or lazy programmers who have misused the language or totally neglected even the slightest brainwork in database handling.

A real-life example

A piece of procedural PHP software suffered from timeouts and memory overflows. The developer had used a "dbQuery" class in a loop that updated 100.000+ database records.The instance of the object got the SQL query in the constructor which then ran the query immediately and stored the result set. There was no way of updating the query and rerunning it - the object was useless after the query/fetch . So it had to be recreated 100.000+ times. The result was a huge CPU load and a horrendous memory consumption.It had nothing to do with OO, it was just a a good example of total misunderstanding of the OO paradigm and using an object like a function. 

There was also no way to extend the class to a healthy one, either, because the class had been installed in the heart of the system. Dozens of scripts would have needed a rewrite to get rid of it. The damage done the solution was to set the PHP memory limit from 64MB to 256MB and  then to 512MB. It has presumably passed the 1GB limit by now. This all for one single update script run one user at a time once a day.

Our experiences

Having used OO, Smarty classes and Smarty based templating for about eight years as of writhing this we have yet to see a situation where objects and templating would be the problem. A colleague running a very popular heavy metal site written using Smarty templates (and without caching) confirms that he has never had any problems yet.

This very tutorial site is running on a shared server in a XEN VPS environment with Alternative PHP Cache  installed. Feel free to have a look at the tail of HTML source code any time. The full rendering cycle is measured and only the inclusion of the tiny class CmsModule has been left out because it is included in the auto_prepend_file part. It's inclusion would have a non-measurable effect.

Some real life tests

First of all we have some figures about the factors influencing our CMS.  To do that we have tested the loading of this page using different configurations:

• All features on, RSS fetched from BBC : 0.072...0.081 sec
• As above bur RSS fetched from local file : 0.051...0.064 sec
• As above but visitor logging to database removed: 0.032...0.034 sec

Please note that those times include loading all classes and handling the following compiled templates

• Main template
• Cascading menu
• Event calendar view
• Blog list creation
• RSS list creation
• Dynamic page creation
• Visitor logging (except in the last one)

The results prove at least some of our  beliefs: The less I/O the better the performance. The system runs very smoothly and the memory use is about 35% lower than with some of the competitors who are using on-the-fly -parse templates. This must be the result of template compilation in SkoopCMS. Parsing is very expensive CPU- and memory-wise. More on this later...

Had we made use of  the Smarty intelligent cache functionality and disabled compiler checks in the production environment the performance would have been better still. And once again: please bear in mind that each disk write, including the database write operations costs more in time  than you have anticipated, especially in a shared environment.

That said the fastest thing you can get are static HTML files but only seldom are they a viable alternative.

Comparing a few 'light-weight' systems

We did a very quick comparison with three systems. One of them was the very nice and lightweight IonizeCms . The second one was the infamous Joomla. The last one was our humble SkoopCMS. Due to popular demand we also measured the performance of Drupal.  We compared the size of compiled code in APC cache, overall memory use and response time. Knowing that IonizeCms is based on the CodeIgniter MVC framework we anticipated a very low consumption of system resources and very fast rendering. Having had a look at Joomla's architecture we were certain about considerable use of CPU and memory. The results really surprised us even though they are not fully comparable.

Compiled Bytecode (all classes involved)

This is something that is for real: compiled bytecode will very probably have a direct effect on performance:

The biggest consumer blocks of SkoopCMS are:

The highlights for Drupal are

The highlights for Joomla are

The biggest bytecode consumers in IonizeCMS are 

Response times and memory usage

We want to stress that the following figures are NOT fully comparable because the content is not the same. IonizeCMS displays the default front page of default installation and Joomla its default demo 'Australian parks'  page. The first (Drupal) figure is the default empty site page with no created content. The second Drupal is the result with just one tiny article displayed.  SkoopCMS displays its own front page with the menu module and dynamic page module - a bit more than the rest of the gang, actually. 

The HTML column tells the number of HTML lines, KB the total size and the last column the render time from the start of the index.php to the spot when everything has been output to the browser. The MAXMEM shows the maximum memory usage in KB.

We think we have an explanation to the difference with Ionize. When rendering the template Ionize parses the template every time whereas SkoopCMS gets pure PHP code which is cached by APC. The Ionize approach cannot use APC bytecode cache the same way and on-the-fly parsing is a heavy task. That is why it also uses about one megabyte more during each request. Ionize bytecode itself, however does reside in the APC cache exactly like the bytecode of SkoopCMS. Sorry to say there is no easy way to compile the Ionize templates to cacheable PHP.

On the other hand the size of the bytecode is pretty much in line with the CPU/Memory figures.

As for Joomla and Drupal we cannot find any explanations or excuses. The pages are utterly simple with almost nothing to process. It may well be that the system loads a bunch of unnecessary classes (as implied by the memory use) and can be optimized one way or another. Off-the-shelf installations behave this way, anyhow.

After adding three modules (event calendar, RSS feed, site news) the numbers for Skoop got drastically worse :)

As one can see the performance has not been affected much with only 18ms added to the processing time. This number of modules is often more than enough for simple sites. The effect of adding more complex modules like product catalogues and shopping carts will mostly depend on the design of the database as well as the number of database writes. Database operations will, of course, have the same effect on all systems no matter how they have been designed code-wise.

Tuning the cache further would have cut down the time. However, we are not tuning any of the systems here and only comparing  'out of the box' installations.

CPU Time - what about it?

This is something that will probably make a difference as well. We believe that the overall load of a server consists of CPU use and I/O use and their combined effect on the request lifetime. A high  number of  concurrent unfinished requests will result in  higher the memory usage which in turn may  at some point lead to paging and a slowdown of the whole system. CPU usage can, of course, be taken into account using the getrusage() function but estimating its effect on the performance can be very difficult. We believe that the formula that gives a rough estimate about the resource use is simply:

(script running time) * (number of concurrent requests) * (memory use)

Why? Because most of the time will in most cases be spent waiting for an I/O operation to finish (database read, file read, log write) the CPU only plays a mininmal part in the game. The more concurrent requests there are the more memory the system will use and which may result in paging and degraded performance. There is no doubt, however, that CPU time has a role in the game but definitely not the main one.

We firmly believe that the key to success is optimized I/O which means the use of a sound database, caching and as little disk traffic as possible.

Well, eventually we did have a look at the CPU times for the fastest performer and the slowest one but could not find anything conclusive. There was too much fluctuation from one call to another and the differences between the systems were not very big compared to the script time. The CPU time only seems to be a few per cent of the total script time:

Some other systems hastily tested

The server memory use of all systems we have seen is from 1600MB (Plain CodeIgniter, 150 lines of HTML) up to ~20 MB (Plain WordPress, a whooping 220 lines of HTML - pun intended).

Well, the writer of BetterWP (an optimized WordPress version) says: "... Some websites use about 61MB memory and make about 126 queries to the database...I myself never use more than 30MB memory and 40 queries for a single page (except for some specific situations or specific pages, and FYI, BetterWP.net only uses an average of 16MB memory and 25 queries per page)...." The little word "only" tells it all ;)

Conclusive thoughts

We want to stress that the performance of IonizeCMS is very good for normal sites, even mid-busy ones and the same goes for most systems like Joomla and Drupal: they are used in pretty heavy sites, anyhow!  Being used by  ~3% of websites Joomla simply cannot be too bad. 

That said we cannot help being hilarious about the performance of our humble CMS :)  Did we hear anyone repeating the phrase about Smarty being bulky and slow ?-)

"But why use templates, anyhow?"

Comments like "PHP is already a templating language"  and "Smarty produces an unnecessary overhead" are common. The way PHP was originally used was kind-of templating. After the early days PHP has evolved to a proper object oriented programming language. 

Templates are used in many languages, including the web ones. Microsoft IIS has the ASP templates coupled with modules written in C# or some other .NET based language. Java has JSP templates which compile to Java servlet bytecode. The ancient Microsoft Macro Assemblers made use of a  templating language as well, the macros being compiled to Assembler code and in turn to machine code. So why on earth should PHP not have them? What is the point of the people opposing this idea? When compiled properly they are just as fast as native PHP.

That said we agree with the performance problems of non-compiled templates which may indeed be a bottleneck.  And we would like to see where the Smarty 'bottleneck' and the Smarty 'overhead' can be found in the numbers above.

The statements against Smarty can be easily proved wrong - at least performance-wise. Well, one can always hide behind the old meaningless proverb "There is no accounting for taste". In the case of crusaders agains Smarty this proverb probably means deeply rooted attitudes and reluctance or incapability to learn new things. They are windmills that are very hard to fight against.