In this post, I compare Kourier with QHttpServer regarding performance and compliance with HTTP syntax.
To make things more interesting, I use TLS-encrypted connections for testing performance. As QHttpServer does not use the concept of workers, for both servers, I limit HTTP request processing to a single thread.
Both servers use TLS 1.2 with the ECDHE-ECDSA-AES128-GCM-SHA256 cipher.
All Docker images used in this benchmark can be easily built using the build script contained in Kourier’s repository.
I run the servers with the following commands:
# Kourier. The server listens on port 3275
# and uses a single thread to process incoming requests.
docker run --rm -d --network host kourier-bench:kourier -a 127.0.0.1 -p 3275 --worker-count=1 --enable-tls --request-timeout=20 --idle-timeout=60
# QHttpServer. The server listens on port 5275
# and uses a single thread to process incoming requests.
docker run --rm --network host -d kourier-bench:qhttpserver -a 127.0.0.1 -p 5275 --enable-tls
I use the command shown below to test the performance:
# PORT is 3275 for Kourier, or 5275 for QHttpServer.
docker run --rm --network host -it kourier-bench:wrk -c 512 -d 15 -t 6 --latency https://localhost:PORT/hello -s /wrk/pipeline.lua -- 256
I use the following command to test compliance with HTTP syntax (for this test to work, the servers must be started without the --enable-tls command-line option):
# PORT is 3275 for Kourier, or 5275 for QHttpServer.
# Servers should be started without the --enable-tls command line option.
docker run --rm --network host -it kourier-bench:http-compliance -a 127.0.0.1 -p PORT
The test certificates and private key used for TLS encryption are located at Src/Tests/Resources/Benchmarks/certs in the Kourier repository. They are copied to the Docker images when the images are built. You can use the Src/Tests/Resources/Benchmarks/build_all.sh script to create all Docker images used on benchmarks.
Results 🔗
# Testing Kourier server at 127.0.0.1:3275
glauco@ldh:~$ sudo docker run --rm --network host -it kourier-bench:wrk -c 512 -d 15 -t 6 --latency https://localhost:3275/hello -s /wrk/pipeline.lua -- 256
Running 15s test @ https://localhost:3275/hello
6 threads and 512 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 26.61ms 17.53ms 148.54ms 38.63%
Req/Sec 383.20k 82.12k 647.68k 67.58%
Latency Distribution
50% 54.71ms
75% 0.00us
90% 0.00us
99% 0.00us
33782628 requests in 15.09s, 3.30GB read
Requests/sec: 2238158.06
Transfer/sec: 224.12MB
# Testing QHttpServer server at 127.0.0.1:5275
glauco@ldh:~$ sudo docker run --rm --network host -it kourier-bench:wrk -c 512 -d 15 -t 6 --latency https://localhost:5275/hello -s /wrk/pipeline.lua -- 256
Running 15s test @ https://localhost:5275/hello
6 threads and 512 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 856.07ms 502.41ms 1.93s 56.34%
Req/Sec 11.92k 3.98k 50.26k 89.16%
Latency Distribution
50% 872.91ms
75% 1.35s
90% 1.66s
99% 0.00us
1052160 requests in 15.09s, 51.17MB read
Requests/sec: 69704.77
Transfer/sec: 3.39MB
# Testing Kourier server compliance at 127.0.0.1:3275
glauco@ldh:~$ sudo docker run --rm --network host -it kourier-bench:http-compliance -a 127.0.0.1 -p 3275
Testing HTTP compliance...
Finished testing HTTP compliance.
175 requests with invalid URL absolute path detected.
0 requests with invalid URL absolute path undetected.
175 requests with invalid URL query detected.
0 requests with invalid URL query undetected.
178 requests with invalid header name detected.
0 requests with invalid header name undetected.
30 requests with invalid header value detected.
0 requests with invalid header value undetected.
61692 requests with invalid pct-encoded hex digits in URL absolute path detected.
0 requests with invalid pct-encoded hex digits URL absolute path undetected.
61692 requests with invalid pct-encoded hex digits URL query detected.
0 requests with invalid pct-encoded hex digits URL query undetected.
# Testing QHttpServer server compliance at 127.0.0.1:5275
glauco@ldh:~$ sudo docker run --rm --network host -it kourier-bench:http-compliance -a 127.0.0.1 -p 5275
Testing HTTP compliance...
Finished testing HTTP compliance.
1 requests with invalid URL absolute path detected.
174 requests with invalid URL absolute path undetected.
1 requests with invalid URL query detected.
174 requests with invalid URL query undetected.
0 requests with invalid header name detected.
178 requests with invalid header name undetected.
0 requests with invalid header value detected.
30 requests with invalid header value undetected.
765 requests with invalid pct-encoded hex digits in URL absolute path detected.
60927 requests with invalid pct-encoded hex digits URL absolute path undetected.
765 requests with invalid pct-encoded hex digits URL query detected.
60927 requests with invalid pct-encoded hex digits URL query undetected.
Conclusion 🔗
Kourier is the next level of network-based communication. It is on another league regarding performance, compliance, and memory consumption. Kourier leaves everything else in the dust, including enterprise network appliances.
Creating the fastest server ever requires much more than a stellar HTTP parser. From ring buffers to socket programming, to a custom timer implementation. I implemented it all and open-sourced all of it alongside Kourier.
I developed Kourier with strict and demanding requirements, where all possible behaviours are comprehensively verified in specifications written in the Gherkin style. To this end, I created Spectator, a test framework that I also open-sourced with Kourier. You can check all files ending in spec.cpp in the Kourier repository to see how meticulously tested Kourier is. There is a stark difference in testing rigor between Kourier and other frameworks.
Kourier can empower the next generation of network appliances/solutions, making businesses that rely on them run at a fraction of their infrastructure cost and in a much more HTTP-compliant way.
You can contact me if your Business is not compatible with the requirements of the AGPL and you want to license Kourier under alternative terms.