Benchmarks are out of date against Rust redis-rs

[BijanVan]

It seems benchmarks are out of date at least against Rust redis-rs.

boost::redis:
https://godbolt.org/z/MYTqW5MY8

Rust redis-rs
https://godbolt.org/z/v5dxsYnoE

In an EC2 machine:

[nihohit]

I'd argue that the benchmarks are just plain wrong - tasks don't run concurrently, but serially. Why bother with an async interface if nothing is concurrent?

[mzimbres]

Hi @BijanVan , as far as I can see you are not running the same benchmark as I did (see the benchmarks folder).

That sad, those results are almost four years old and I agree an update is needed. In fact, I am working on it at the moment on my https://github.com/mzimbres/redis-cli-comp repository, which is still work in progress and therefore I haven't advertised it yet.

Redis-rs seems to have improved a lot, but the performance difference between boost-redis and redis-rs seems to persist to this day. After seeing your message I have rerun the benchmarks from my new repo and got the following results

• Commands per second

Client	Megahertz
boost_redis_co	6578
boost_redis_cb	8207
redis_rs	1470
go_redis	1581

• Cpu usage

Client	%usr	%system	%Total
boost_redis_co	92.00	7.33	99.33
boost_redis_cb	92.43	4.86	97.29
redis_rs	199.72	84.28	284.00
go_redis	211.25	140.83	352.08

• Resource usage

Type	threads	fd-nr
boost_redis_co	2	7
boost_redis_cb	2	7
redis_rs	21	10
go_redis	30	1006

The general feeling is that rueidis comes 3rd in terms of performance after boost-redis and rueidis, the latter is not included above because of a problem in my script.

I agree with @nihohit in the sense that your benchmarks do not seem to measure a useful quantity from the practical point of view, I think it will be dominated by the unavoidable latency i.e. write/ read socket, Redis processing time etc. Let me know if they represent a practical use case and we can review your benchmark code and perhaps optimize boost-redis for it.

For reference: redis-rs/redis-rs#637

[nihohit]

@mzimbres redis-rs/redis-rs#637 is completely out of date, and that connection type was deprecated and removed in prior versions, so I'm not sure what version you're using for your benchmark. Use the MultiplexedConnection without wrapping it with an Arc or a Mutex and you'll get a threadsafe multiplexed connection that doesn't need 21 threads for good perf :). You can follow this example.

[nihohit]

I'll be happy to help, if you get bogged down with how to use it for the benchmark :)

[mzimbres]

@nihohit The link referred to the old benchmarks, the new ones do use the multiplexed connection https://github.com/mzimbres/redis-cli-comp/blob/573b551656c9cd91f25b8df90fafcae1325ec46a/apps/redis-rs/src/main.rs#L41 and the latest redis-rs. You can review it and let me know if it can be improved. Thanks.

.