Interested in a concurrent bloom filter interface?

[jberryman]

Two questions:

1. Would you be interested in merging a thread-safe IO interface to bloom filters into this package if I put one together? It would add a dependency at least on atomic-primops.
2. It might be that the cache-efficient bloom filter variant would be a more appropriate though, in which case would you mind if I forked your package or otherwise re-used internals for a cache-efficient, concurrent bloomfilter package? And how would you like to be credited in that case?

Thanks!

[dcoutts]

@jberryman I see you subsequently wrote the unagi-bloomfilter concurrent Bloom filter.

It may be interesting to compare the performance of that with the new cache-efficient implementation in bloomfilter-blocked. Note however that it uses whole cache lines for buckets/blocks. This is better for the FPR, but it would complicate a safe concurrent implementation of the insert operation, since CPUs cannot do whole-cache-line-atomic operations.

[jberryman]

Oh interesting, good to see a new library in this space! unagi-bloomfilter depends on my hashabler library which unfortunately I got burned out on 80% of the way through a rewrite; in the first versions I didn't properly understand what makes for a principled Hashable instance (it needs to be an injective mapping to serialized bytes, and prefix code is the obvious way to get there), and then I never got around to TH deriving for those instances. So I can't really recommend anyone try to use either library, sadly.

Maybe I will pick up hashabler again someday. It would be neat to have a class that could be used both for serialization (for deserialization) and for hashing, would allow for libraries like unordered-containers to swap out the hash function for one specified by user (maybe even via backpack, keeping the same API?), and it's very fast when everything gets inlined.