siphash24: force inlining of the round

gcc does not inline sipround() at -O2: its size estimate lands just over
the default max-inline-insns-auto budget, so nm on libsystemd-shared.so
shows a local sipround symbol and each of the ten call sites pays a
call/ret plus a reload and store of the state through the pointer, around
six rounds of ~14 ALU ops. clang inlines it on its own, and gcc does too
at -O3 or with --param max-inline-insns-auto=30, so this is not a missed
optimization to report upstream but the inliner's size heuristic doing
what it is designed to do. The documented way to overrule it for one
function is always_inline.

The SipHash reference implementation this file is derived from inlines
the round manually, by writing it as a SIPROUND macro.

AMD Ryzen AI 7 350, gcc 16.2.1, -O2 -fstack-protector-strong
-fno-strict-aliasing, median of five runs:

    input     before      after     speedup
      8 B    61.3 ns     9.3 ns       6.63x
     16 B    54.6 ns    10.7 ns       5.11x
     24 B    50.0 ns    12.6 ns       3.98x
     40 B    49.4 ns    16.4 ns       3.01x
     64 B    53.2 ns    22.1 ns       2.41x
    128 B    67.8 ns    36.9 ns       1.84x
    512 B   157.7 ns   128.1 ns       1.23x
   4096 B   992.5 ns   964.5 ns       1.03x

The win is concentrated on short inputs because the bulk loop in
siphash24_compress() was the one place gcc did inline the round; the
prologue, the tail and all six rounds of siphash24_finalize() were not.
Short inputs are also what we overwhelmingly hash: hashmap and set keys,
unit names, paths, journal field names and data objects, bus match
strings. clang is unaffected.

Resulting hash values are unchanged; verified identical over every input
length from 0 to 128 bytes, and test-siphash24, test-hash-funcs,
test-hashmap and test-set pass.
This commit is contained in:
Armaan Sandhu
2026-09-23 16:30:50 +02:00
committed by Luca Boccassi
parent efe5e4d271
commit 31bf94f87d
2 changed files with 7 additions and 2 deletions
+6 -2
View File
@@ -26,13 +26,17 @@
#include "string-util.h"
#include "unaligned.h"
static uint64_t rotate_left(uint64_t x, uint8_t b) {
/* The round is small enough that it is worth inlining at every call site, but large enough that gcc's
* -O2 inlining budget declines to do so on its own, leaving a call/ret and a reload of the state around
* six rounds of ~14 ALU ops. clang inlines it either way, and the SipHash reference implementation this
* file is derived from inlines the round manually, by writing it as a macro. */
static inline _always_inline_ uint64_t rotate_left(uint64_t x, uint8_t b) {
assert(b < 64);
return (x << b) | (x >> (64 - b));
}
static void sipround(struct siphash *state) {
static inline _always_inline_ void sipround(struct siphash *state) {
assert(state);
state->v0 += state->v1;
+1
View File
@@ -76,6 +76,7 @@
#define _align_(x) __attribute__((__aligned__(x)))
#define _alignas_(x) __attribute__((__aligned__(alignof(x))))
#define _alignptr_ __attribute__((__aligned__(sizeof(void *))))
#define _always_inline_ __attribute__((__always_inline__))
#define _cleanup_(x) __attribute__((__cleanup__(x)))
#define _const_ __attribute__((__const__))
#define _deprecated_ __attribute__((__deprecated__))