poor AB bench results

[lubosdz]

Hi,

not sure where to ask - so just shortly here:

I did apache bench on yii3 (demo) against yii2 (basic app) and got extremly poor results for yii3:

• yii3 - only about 0.18 page per sec, 1 request takes about 5-6 secs
• yii2 - OK, about 140 pages per sec

Using latest version of yii3 dev (composer update), PHP 7.4.1.

bench results here (must have switched to -n 1 due to extremly slow response):

Yii3

d:\wamp\bin\apache\apache2.4.37\bin>ab -n 1 -c 1 127.0.0.1/yii3/yii-demo/public/index.php
This is ApacheBench, Version 2.3 <$Revision: 1843412 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking 127.0.0.1 (be patient).....done


Server Software:        Apache/2.4.37
Server Hostname:        127.0.0.1
Server Port:            80

Document Path:          /yii3/yii-demo/public/index.php
Document Length:        43 bytes

Concurrency Level:      1
Time taken for tests:   5.987 seconds
Complete requests:      1
Failed requests:        0
Non-2xx responses:      1
Total transferred:      344 bytes
HTML transferred:       43 bytes
Requests per second:    0.18 [#/sec] (mean)                    <- why? edit: found reason: 404 timeout
Time per request:       5986.504 [ms] (mean)
Time per request:       5986.504 [ms] (mean, across all concurrent requests)
Transfer rate:          0.06 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.0      0       0
Processing:  5987 5987   0.0   5987    5987
Waiting:      687  687   0.0    687     687
Total:       5987 5987   0.0   5987    5987

Is there anything special to configure for yii3 demo CLI environment .. ?
In HTTP mode yii3 demo works all OK.

---

As for yii2 results are OK on the same setup:

Yii2

d:\wamp\bin\apache\apache2.4.37\bin>ab -n 1000 -c 1 127.0.0.1/yii2/basic/web/index.php
This is ApacheBench, Version 2.3 <$Revision: 1843412 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking 127.0.0.1 (be patient)
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Completed 600 requests
Completed 700 requests
Completed 800 requests
Completed 900 requests
Completed 1000 requests
Finished 1000 requests


Server Software:        Apache/2.4.37
Server Hostname:        127.0.0.1
Server Port:            80

Document Path:          /yii2/basic/web/index.php
Document Length:        65901 bytes

Concurrency Level:      1
Time taken for tests:   6.571 seconds
Complete requests:      1000
Failed requests:        0
Non-2xx responses:      1000
Total transferred:      66116000 bytes
HTML transferred:       65901000 bytes
Requests per second:    152.20 [#/sec] (mean)
Time per request:       6.571 [ms] (mean)
Time per request:       6.571 [ms] (mean, across all concurrent requests)
Transfer rate:          9826.70 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   1.3      0      10
Processing:     0    6   4.7      9      16
Waiting:        0    6   4.7      7      16
Total:          0    6   4.8     10      16

Percentage of the requests served within a certain time (ms)
  50%     10
  66%     10
  75%     10
  80%     10
  90%     10
  95%     13
  98%     15
  99%     15
 100%     16 (longest request)

[samdark]

Is there anything special to configure for yii3 demo CLI environment .. ?
In HTTP mode yii3 demo works all OK.

What do you mean by "HTTP mode" and "CLI environment"?

[yiiliveext]

At least yii2-basic doesn't have db queries on page load.

[lubosdz]

HTTP mode = browsing in a browser
CLI mode = running command line shell (shellscript in windows), console

OK - will check what config is loaded in CLI and check DB connection.

Edit: slightly updated results for yii2 after running composer update.

[lubosdz]

After investigation I found:

• Yii3 demo public/index.php - does not use DB connection. Even if it would, there is already pre-configured DB (for demo blog) using local SQLite, which does not require credentials. So DB connection in console is not an issue here.
• there is exception throw in both console/web mode No definition for FastRoute\RouteParser at CompositeContainer.php line 32 . Silently ignored by configuration. This is not the reason, just wondering whether this is correct. Exceptions should not control request workflow.
• returned document length 43 bytes - it simply returns 404 not found after 7 secs of timeout. But I cannot figure out what file cannot be found :-( Path to public/index.php is correct.

I am wondering - does AB bench in yii3-demo work well for anyone else?

Aside notes:

• yii3 is much-much harder to track down issues than yii2. Learning curve is significantly steeper. Some parts feel overengineered in yii3. Even config files implement some kind of business logic.
• there is lot of room for performance improvements - see profiling screen bellow.
• memory usage shows 3-4x more than in yii2 (cca 1.5 MB against 4.5 MB). Actually while debugging yii3-demo I measured 22 MB memory usage (peak).

[samdark]

@lubosdz top performance eaters are https://github.com/yiisoft/yii-demo/blob/master/public/index.php#L15

[samdark]

Also we haven't worked on performance yet focusing on design mostly.

[yiiliveext]

After investigation I found:

• Yii3 demo public/index.php - does not use DB connection. Even if it would, there is already pre-configured DB (for demo blog) using local SQLite, which does not require credentials. So DB connection in console is not an issue here.

You should take into account the Cycle ORM initialization. Try to disable config rebuild and Cycle ORM.

[samdark]

Another thing is that debugger is turned on collecting ~500kb of data each request. https://github.com/yiisoft/yii-demo/blob/master/config/params.php#L8

[shurik2k5]

Try to disable this: https://github.com/yiisoft/yii-demo/blob/a7595dbffb89a236c89e0e23a904eefbf47e6a5f/public/index.php#L15

[samdark]

Anyway, it should not take 6! seconds to respond and you can see it when running web app. There must be something that makes ab unhappy/fail.

[terabytesoftw]

Yii3 (yiisoft/yii-demo) AB benchmark http://demo.yiipeople.site:

• disable rebuild container.
• disable yii-debug.

[root@localhost php]# ab -n 1000 -c 1 http://demo.yiipeople.site/
This is ApacheBench, Version 2.3 <$Revision: 1430300 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking demo.yiipeople.site (be patient)
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Completed 600 requests
Completed 700 requests
Completed 800 requests
Completed 900 requests
Completed 1000 requests
Finished 1000 requests


Server Software:        LiteSpeed
Server Hostname:        demo.yiipeople.site
Server Port:            80

Document Path:          /
Document Length:        4627 bytes

Concurrency Level:      1
Time taken for tests:   4.580 seconds
Complete requests:      1000
Failed requests:        109
   (Connect: 0, Receive: 0, Length: 109, Exceptions: 0)
Write errors:           0
Total transferred:      4914881 bytes
HTML transferred:       4626881 bytes
Requests per second:    218.33 [#/sec] (mean)
Time per request:       4.580 [ms] (mean)
Time per request:       4.580 [ms] (mean, across all concurrent requests)
Transfer rate:          1047.92 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.0      0       0
Processing:     4    4   0.4      4      10
Waiting:        4    4   0.4      4       9
Total:          4    5   0.4      4      10

Percentage of the requests served within a certain time (ms)
  50%      4
  66%      5
  75%      5
  80%      5
  90%      5
  95%      5
  98%      6
  99%      7
 100%     10 (longest request)

[terabytesoftw]

Yii3 (yiisoft/app) AB benchmark http://yiipeople.site:

• disable rebuild container.
• disable yii-debug.

[root@localhost php]# ab -n 1000 -c 1 http://yiipeople.site/
This is ApacheBench, Version 2.3 <$Revision: 1430300 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking yiipeople.site (be patient)
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Completed 600 requests
Completed 700 requests
Completed 800 requests
Completed 900 requests
Completed 1000 requests
Finished 1000 requests


Server Software:        LiteSpeed
Server Hostname:        yiipeople.site
Server Port:            80

Document Path:          /
Document Length:        4162 bytes

Concurrency Level:      1
Time taken for tests:   3.752 seconds
Complete requests:      1000
Failed requests:        0
Write errors:           0
Total transferred:      4445000 bytes
HTML transferred:       4162000 bytes
Requests per second:    266.49 [#/sec] (mean)
Time per request:       3.752 [ms] (mean)
Time per request:       3.752 [ms] (mean, across all concurrent requests)
Transfer rate:          1156.80 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.0      0       0
Processing:     3    4   0.6      4      13
Waiting:        3    3   0.6      3      12
Total:          3    4   0.6      4      13

Percentage of the requests served within a certain time (ms)
  50%      4
  66%      4
  75%      4
  80%      4
  90%      4
  95%      4
  98%      5
  99%      5
 100%     13 (longest request)

[terabytesoftw]

Yii2 (yiisoft/app-basic) AB benchmark http://basic.yiipeople.site/index.php:

[root@localhost php]# ab -n 1000 -c 1 http://basic.yiipeople.site/index.php
This is ApacheBench, Version 2.3 <$Revision: 1430300 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking basic.yiipeople.site (be patient)
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Completed 600 requests
Completed 700 requests
Completed 800 requests
Completed 900 requests
Completed 1000 requests
Finished 1000 requests


Server Software:        LiteSpeed
Server Hostname:        basic.yiipeople.site
Server Port:            80

Document Path:          /index.php
Document Length:        4107 bytes

Concurrency Level:      1
Time taken for tests:   4.391 seconds
Complete requests:      1000
Failed requests:        0
Write errors:           0
Total transferred:      4647000 bytes
HTML transferred:       4107000 bytes
Requests per second:    227.75 [#/sec] (mean)
Time per request:       4.391 [ms] (mean)
Time per request:       4.391 [ms] (mean, across all concurrent requests)
Transfer rate:          1033.57 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.0      0       0
Processing:     3    4   8.5      3     130
Waiting:        3    4   8.5      3     130
Total:          3    4   8.5      3     130

Percentage of the requests served within a certain time (ms)
  50%      3
  66%      3
  75%      3
  80%      4
  90%      4
  95%      4
  98%      5
  99%     68
 100%    130 (longest request)

[samdark]

Interesting results, @terabytesoftw. Have you turned off YII_DEBUG in Yii 2?

[terabytesoftw]

Interesting results, @terabytesoftw. Have you turned off YII_DEBUG in Yii 2?

Yes.

[samdark]

Then, despite not optimizing Yii 3 yet, it gives 266 rps vs 227 rps with Yii 2. That's unexpected but exciting.

[lubosdz]

Thanx for advices.
Disabling Builder::rebuild() in index.php and yii-debug mode in config reduced memory usage from 4,5 MB to cca 1.5 MB - nice. But now the page rendering timer shows the number of secs since session start instead of page generation time e.g. Time: 402.6691 s. Memory: 1.2735 mb..

The 404 timeout in ab yii3-demo - still cannot figure out. But if ab it works on http://demo.yiipeople.site/ (linux) then my guess it could be related to windows 10 e.g. permissions or something. Will take deeper look into it next days.

EDIT:
I am getting suspicious about .htaccess in yii3-demo config because:

• when I access http://localhost/yii3/yii-demo/public/index.php - shows error We were unable to find the page /index.php. - this would explain 404 timeout in apache benchmark.
• when I access http://localhost/yii3/yii-demo/public/ - (without specifying index.php) everything works OK

In yii2-basic accessing http://localhost/yii2/yii-basic/web/index.php (with or without index.php) works both OK.

Though I dont see anything wrong in .htaccess.
I played with it e.g. adding Options +FollowSymLinks and mod_deflate - did not help so far.

[alexey-egoshin]

Guys, we very strong wait Yii3 and it must be faster than Yii2 of course! )

[rossaddison]

Working nicely. xdebug in apache's php.ini disabled as mentioned in your suggestions. yii-debug and api disabled. Significant speed increase.

Alternating between the following in config/params.php especially the first or former option ie. PhpFileSchemaProvider has increased performance significantly since syncing is mainly for building the database. The database is relatively secure so this first option is my preferred choice.

           \Yiisoft\Yii\Cycle\Schema\Provider\PhpFileSchemaProvider::class => [
               'mode' => \Yiisoft\Yii\Cycle\Schema\Provider\PhpFileSchemaProvider::MODE_READ_AND_WRITE,
               'file' => 'runtime/schema.php',
            ],

            \Yiisoft\Yii\Cycle\Schema\Provider\FromConveyorSchemaProvider::class => [
                'generators' => [
                     Cycle\Schema\Generator\SyncTables::class, // sync table changes to database
                ],
            ],

[kmergen]

I have the same issue ca. 5 sec per Request. but in chrome dev tools i have nearly the same good load results than in yii2-basic-app. (Load finish in ca. 100 ms, DomContentLoaded ca. 65 ms)
I run yii3 application template on windows 11 with xampp and php 8.1 and apache 2.4.56.

Tried with env prod , debug false, without bootstrap(), comment out all unnecessary stuff, but always the same bad results.

[samdark]

That should be something about how ab is measuring time. We need to look into it but since real app responds in a timely manner to the browser, that's not a critical thing.