Performance proposal with ch-go

[xzkostyan]

Hi.

Reading and writing data packets is really expensive in any language. If you really need some connection inspection (I suppose you do) you should decode at least information packets, except data packet.

The problem is that data packet doesn't always has block size: only if it's singed. You can try to read compressed data block without decoding it with ch-go. By default clickhouse-client uses lz4 compression with singing by cityhash. Client sets compression method byte right before the query body. For quick & dirty you can assume it's on.

Layout of compressed by LZ4 data package is quite simple:

• 128-bit cityhash
• compression byte
• 32-bit size of decompressed data packet with header
• compressed data packet

So in case of LZ4 compression you should read following amount of bytes to fetch data block without decoding it:

size with header - 1 (compression byte) - 4 (decompressed size).

[mga-chka]

thanks, for the info.
Indeed we need to inspect packets in order to cache them and, for some (like the client hello) we need to modify them.
We also need to micmic the behavior of clickhouse in case we need to return to the client a cached query.

The idea we currently have in mind is to use ch-go for all the packets (to ease the management of backward compatibility) excepts the big ones (data from query results and data from "insert into"). For these ones, we would read the stream "manually" just to get the size of the block (which means for some of them reading partially the full block, but it would still be faster than what ch-go does).

[xzkostyan]

There is one corner case: the signing of data blocks is optional right now. AFAIK, there is no size of data block case of no compression and signing.

I guess we need to propose addition of the data block size in ClickHouse server or refuse such connections in tcp proxy.

[mga-chka]

Thanks for info. I worked in the past (like 3 years ago) on the blocks to implement a custom compression. If it hasn't changed, I was thinking about reading the block by "hopping" every time we can. For example in the case of a column of strings, we would read the length of a string then jump to the next string to get its length and so on. It's not efficient but should be better than ch-go that reads/deserializes everything. I'm just worried about the extra-complexity.

Another trick I have in mind would be to force the LZ4 connection between clickhouse and chproxy no matter what the client specifies. Then, depending on what the client specified we would either:

• decompress the block
• keep it has LZ4
• replace LZ4 with ZSTD
  so that the client gets the compression he asked for

Adding the block size in the block would be indeed a good solution but we would still need to handle the "legacy" version of clickhouse because I'm expecting most of the companies a version. That being said, if you open an issue on clickhouse, we will upvote it (and if you don't, we might create it because it will ease the native protocol implementations).