CF8 vs Railo: Speed Comparison

On CF-Talk, there's been an ongoing thread about Railo's blazing performance. Railo has always been known for its blazing speed and I'm glad people are finally taking notice.
David Low posted an interesting speed comparison between CF8 & Railo. David's tests show Railo clearly outperforming CF8 and by a healthy margin.

This must've really set some things in motion because Adobe decided to run their own performance comparison. Adobe's Damon Cooper posted the following performance stats on CF-Talk today and I have to say I was impressed & surprised by CF8's results:

“OVERVIEW: In all load tests here, we see ColdFusion 8 outperform Railo

Testing Scenarios:
All tests were run with 100 Virtual users on Windows 2003 Sp2 Server with dual 3.60GHz CPU's and 2GB RAM.

Because we do not support TomCat running on Beta virtualization software ( Apple Bootcamp ) the following tests were run on a fully supported server platform.

The following tests mirror some of the tests that were posted on your blog.

Test 1. Simple CFM page outputting a string.

  • CF 8: Requests per Second: 800.94
  • Railo: Requests per Second: 721.22

Test 2: Simple MySQL query returning 241 records.

  • CF 8: Requests per Second: 279.15
  • Railo: Requests per Second: 187.22

Test 3: Simple page with 5 cfincludes:

  • CF 8: Requests per Second: 633.33
  • Railo: Requests per Second: 631.43

I'm not sure why in your testing you are seeing CF 8 take one second longer for queries to MySQL. Railo ships with an older MySQL driver ( MySQL Connector/J 3.0.1 ). CF 8 ships with MySQL Connector 5.0.5. In our testing, CF 8 and the newer driver were substantially faster when running queries.”

These results show that ColdFusion 8 outperforms Railo! That was a real stunner especially in light of David's results.

This looks like its going to turn into an interesting debate.