Thursday, March 4, 2010

Flame Post: Performance 1.2.6 vs 1.3.0-RC1 - Bad news.

Hello,


I am not a professional benchmarker, but I decided to make some
comparison between 1.2.6 and 1.3.0-RC1 tags. So, I downloaded them,
and created the following file:


cat app/views/pages/home.ctp
hello world.


And changed nothing else. Just created the file, I ran Apache's AB
several times, I tried to pick the best numbers among all tests done
on my server and I obtained these numbers:

# /usr/local/apache/bin/ab -A"******:*******" -n 1000 -c 100
http://serverbenchmark/cakephp-1.2.6
This is ApacheBench, Version 2.0.41-dev <$Revision: 1.121.2.12 $>
apache-2.0
Copyright (c) 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Copyright (c) 2006 The Apache Software Foundation, http://www.apache.org/

Benchmarking serverbenchmark (be patient)
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Completed 600 requests
Completed 700 requests
Completed 800 requests
Completed 900 requests
Finished 1000 requests


Server Software: Apache/2.0.63
Server Hostname: serverbenchmark
Server Port: 80

Document Path: /cakephp-1.2.6
Document Length: 359 bytes

Concurrency Level: 100
Time taken for tests: 7.615287 seconds
Complete requests: 1000
Failed requests: 0
Write errors: 0
Non-2xx responses: 1000
Total transferred: 629000 bytes
HTML transferred: 359000 bytes
Requests per second: 131.31 [#/sec] (mean)
Time per request: 761.529 [ms] (mean)
Time per request: 7.615 [ms] (mean, across all concurrent
requests)
Transfer rate: 80.63 [Kbytes/sec] received

Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 2.0 0 17
Processing: 3 695 819.2 101 3245
Waiting: 1 695 819.3 100 3244
Total: 7 696 819.6 101 3248

Percentage of the requests served within a certain time (ms)
50% 101
66% 1013
75% 1029
80% 1044
90% 2052
95% 2244
98% 3234
99% 3241
100% 3248 (longest request)

# /usr/local/apache/bin/ab -A"******:*******" -n 1000 -c 100
http://serverbenchmark/cakephp-1.3.0-RC1/
This is ApacheBench, Version 2.0.41-dev <$Revision: 1.121.2.12 $>
apache-2.0
Copyright (c) 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Copyright (c) 2006 The Apache Software Foundation, http://www.apache.org/

Benchmarking serverbenchmark (be patient)
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Completed 600 requests
Completed 700 requests
Completed 800 requests
Completed 900 requests
Finished 1000 requests


Server Software: Apache/2.0.63
Server Hostname: serverbenchmark
Server Port: 80

Document Path: /cakephp-1.3.0-RC1/
Document Length: 1111 bytes

Concurrency Level: 100
Time taken for tests: 14.150790 seconds
Complete requests: 1000
Failed requests: 101
(Connect: 0, Length: 101, Exceptions: 0)
Write errors: 0
Total transferred: 1505381 bytes
HTML transferred: 1116446 bytes
Requests per second: 70.67 [#/sec] (mean)
Time per request: 1415.079 [ms] (mean)
Time per request: 14.151 [ms] (mean, across all concurrent
requests)
Transfer rate: 103.88 [Kbytes/sec] received

Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 7 21.2 0 73
Processing: 95 1361 517.6 1275 3105
Waiting: 82 1324 533.1 1232 3105
Total: 122 1368 516.4 1276 3105

Percentage of the requests served within a certain time (ms)
50% 1276
66% 1354
75% 1396
80% 1466
90% 2224
95% 2475
98% 3063
99% 3073
100% 3105 (longest request)


Well the 1.2.6 tag out performs the 1.3.0-RC1 on the main number
(IMHO): requests per second.

Flame Question: Shouldn't 1.3.0-RC1 release perform better than 1.2.6?


Should anyone point any flaw in this benchmark, pleas tell me, so I
can run the tests correctly.


Dérico Filho

Check out the new CakePHP Questions site http://cakeqs.org and help others with their CakePHP related questions.

You received this message because you are subscribed to the Google Groups "CakePHP" group.
To post to this group, send email to cake-php@googlegroups.com
To unsubscribe from this group, send email to
cake-php+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/cake-php?hl=en

No comments: