Skip to content

pierrec/go-cuckoofilter

Repository files navigation

godoc Build Status

Package cuckoofilter implements the Cuckoo Filter algorithm for approximated set-membership queries. See https://www.cs.cmu.edu/~dga/papers/cuckoo-conext2014.pdf for the theory.

Six types are defined depending on the use cases:

Filter Type Name Input Type memory usage (per item) error rate (%)
Filter64S uint64 1 byte ~11
Filter64M uint64 2 bytes ~1
Filter64L uint64 8 bytes ~0.003
FilterS []byte 1 byte ~11
FilterM []byte 2 bytes ~1
FilterL []byte 8 bytes ~0.003

Filter64* types are much faster then their siblings, see the benchmarks below.

The following benchmarks were performed on randomly generated items of sizes up to 16 bytes.

Benchmarks for FilterL:

Benchmark Name # tests time/op bits/op allocations/op
Benchmark64InsertL-4 2000000 1283 ns/op 4 B/op 0 allocs/op
Benchmark64HasL-4 30000000 39.4 ns/op 0 B/op 0 allocs/op
BenchmarkInsertL-4 1000000 1060 ns/op 8 B/op 0 allocs/op
BenchmarkHasL-4 20000000 101 ns/op 0 B/op 0 allocs/op

Benchmarks for FilterM:

Benchmark Name # tests time/op bits/op allocations/op
Benchmark64InsertM-4 2000000 2152 ns/op 1 B/op 0 allocs/op
Benchmark64HasM-4 50000000 22.6 ns/op 0 B/op 0 allocs/op
BenchmarkInsertM-4 2000000 1691 ns/op 1 B/op 0 allocs/op
BenchmarkHasM-4 20000000 79.0 ns/op 0 B/op 0 allocs/op

Benchmarks for FilterS:

Benchmark Name # tests time/op bits/op allocations/op
Benchmark64InsertS-4 1000000 1077 ns/op 1 B/op 0 allocs/op
Benchmark64HasS-4 100000000 16.4 ns/op 0 B/op 0 allocs/op
BenchmarkInsertS-4 1000000 1004 ns/op 1 B/op 0 allocs/op
BenchmarkHasS-4 20000000 65.6 ns/op 0 B/op 0 allocs/op

Benchmarks for github.com/seiflotfy/cuckoofilter:

Benchmark Name # tests time/op bits/op allocations/op
BenchmarkExt1Insert-4 1000000 4950 ns/op 17 B/op 1 allocs/op
BenchmarkExt1Has-4 10000000 175 ns/op 16 B/op 1 allocs/op

Benchmarks for github.com/tylertreat/BoomFilters:

Benchmark Name # tests time/op bits/op allocations/op
BenchmarkExt2Insert-4 1 1656973329 ns/op 1006633088 B/op 8388613 allocs/op
BenchmarkExt2Has-4 3000000 455 ns/op 32 B/op 2 allocs/op

About

Package cuckoofilter implements the Cuckoo Filter algorithm for approximated set-membership queries.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages