Skip to content

Commit 89635bd

Browse files
Nicolas De Carlifacebook-github-bot
authored andcommitted
Add rapidhash family of functions
Summary: We've seen high cpu usage on these two hashing algorithms: std::_Hash_bytes and multifeed/common/Hash.h::hashBytesImpl For the record, std::_Hash_bytes compiles to ~60 instructions on aarch64 and ~100 instructions on AMD64: https://godbolt.org/z/xeoqf1aaE hashBytesImpl compiles to slightly over 100 instructions on aarch64 and slightly over 160 instructions on AMD64: https://godbolt.org/z/bTroqGE7o The diff adds three new hash functions: rapidhash, rapidhashMicro and rapidhashNano RapidhashNano is designed for situations where keeping a small code size is a top priority. Clang-19 compiles it to less than 100 instructions without stack usage, both on x86-64 and aarch64. The fastest for sizes up to 48 bytes, but may be considerably slower for larger inputs. RapidhashMicro is designed for situations where cache misses make a noticeable performance detriment. Clang-19 compiles it to ~140 instructions without stack usage, both on x86-64 and aarch64. Faster for sizes up to 512 bytes, just 15%-20% slower for inputs above 1kb. rapidhash provides formidable speed across all input sizes Clang-19 compiles it to ~185 instructions, both on x86-64 and aarch64. Benchmark results on BGM: P1826606121, and Grace: P1826591223 On AMD64, RapidhashNano should be strictly better than both std::_Hash_bytes and hashBytesImpl On aarch64, std::_Hash_bytes compiles to fewer instructions. RapidhashNano should still be faster in most situations, given its much higher throughput. It should also be strictly better than hashBytesImpl In many situations, RapidhashMicro should be a better choice, due to its higher throughput. This diff allows us to analyze workloads on a case by case basis. rapidhash seems to be the fastest high-quality hash function for aarch64 systems. It may still find usage on large-input cases. Folly's benchmark results have been updated to include runs from Bergamo and Neoverse-V2 Differential Revision: D66326393
1 parent 7a74f81 commit 89635bd

File tree

8 files changed

+1271
-95
lines changed

8 files changed

+1271
-95
lines changed

folly/BUCK

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3184,6 +3184,20 @@ non_fbcode_target(
31843184
],
31853185
)
31863186

3187+
non_fbcode_target(
3188+
_kind = folly_xplat_library,
3189+
name = "external_rapidhash",
3190+
feature = triage_InfrastructureSupermoduleOptou,
3191+
raw_headers = [
3192+
"external/rapidhash/rapidhash.h",
3193+
],
3194+
exported_deps = [
3195+
":bits",
3196+
":cpp_attributes",
3197+
":portability",
3198+
],
3199+
)
3200+
31873201
# A basic pieces of folly
31883202
# Should only require glog and boost
31893203
non_fbcode_target(
@@ -4335,6 +4349,18 @@ non_fbcode_target(
43354349
],
43364350
)
43374351

4352+
non_fbcode_target(
4353+
_kind = folly_xplat_library,
4354+
name = "hash_rapidhash",
4355+
feature = triage_InfrastructureSupermoduleOptou,
4356+
raw_headers = [
4357+
"hash/rapidhash.h",
4358+
],
4359+
deps = [
4360+
":external_rapidhash",
4361+
],
4362+
)
4363+
43384364
non_fbcode_target(
43394365
_kind = folly_xplat_library,
43404366
name = "logging_init_weak",

folly/external/rapidhash/BUCK

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
load("@fbcode_macros//build_defs:build_file_migration.bzl", "fbcode_target")
2+
load("@fbcode_macros//build_defs:cpp_library.bzl", "cpp_library")
3+
4+
oncall("fbcode_entropy_wardens_folly")
5+
6+
fbcode_target(
7+
_kind = cpp_library,
8+
name = "rapidhash",
9+
headers = ["rapidhash.h"],
10+
exported_deps = [
11+
"//folly:cpp_attributes",
12+
"//folly/lang:bits",
13+
],
14+
)

0 commit comments

Comments
 (0)