buffered IO performance

February 11, 2007

Next to the raw-IO performance which is important for heavy, static file transfers the buffered IO performance is more interesting for sites which have a small set of static files which can be kept in the fs-caches.

As we are using hot-caches for this benchmark the “lightness” of the server becomes important. The less syscalls it has to do, the better.

The test-case is made up of 100MByte of files in the size of 10MByte and 100kByte.

Benchmark

100kByte

100MByte of 100kBytes files served from the hot caches:

lighttpd
backendMByte/sreq/suser + sys
writev82.20802.7190%
linux-sendfile70.27686.3256%
gthread-aio75.39736.2398%
posix-aio73.10713.8898%
linux-aio-sendfile31.32305.9035%
others
Apache 2.2.4 (event)70.28686.3860%
LiteSpeed 3.0rc270.20685.6550%
  • linux-aio-sendfile is loosing most of its performance as it has to use O_DIRECT to operation which always is a unbuffered read.
  • Apache, LiteSpeed and linux-sendfile are using the same syscall: sendfile() and end up with the same performance values
  • gthread-aio and posix-aio perform better than sendfile()
  • write() performs better than the threaded AIO and sendfile() I can’t explain that right now :)

10MByte

100MByte of 10MBytes files served from the hot caches. The benchmark command has been changed as in the other benchmarks:

$ http_load -verbose -timeout 40 -parallel 100 -fetches 500 http-load.10M.urls-100M

http_load is doing a hard cut when we are using the -seconds option and we might lose some MByte/s due to incomplete transfers.

lighttpd
backendMByte/sreq/suser + sys
writev82.208.7680%
linux-sendfile53.955.6540%
gthread-aio83.028.6690%
posix-aio82.318.6093%
linux-aio-sendfile70.177.3560%
others
Apache 2.2.4 (event)50.925.3340%
LiteSpeed 3.0rc255.585.8040%
  • all the sendfile() implementations seem to have the same performance problem.
  • writve() and the threaded AIO backends utilize the network as expected
  • linux-aio-sendfile is faster as the buffered sendfile() even if it has to read everything from disk … strange