Cryptographic Hash Functions

Definition: One-way mathematical functions that map arbitrary-length input data to a fixed-size output (a digest), designed so the mapping is deterministic, hard to reverse, and hard to collide.

How It Works

A cryptographic hash function takes any input, a single byte or a multi-gigabyte file, and produces a fixed-length digest (128, 256, or 512 bits depending on the algorithm). Four properties define whether a hash function is cryptographically useful:

Deterministic: the same input always produces the same digest.
Fast to compute: hashing even large inputs is computationally cheap (this is a feature for integrity checks, and a liability for password hashing, addressed separately below).
Pre-image resistant: given a digest, it’s computationally infeasible to find any input that produces it.
Collision resistant: it’s computationally infeasible to find two different inputs that produce the same digest.
Avalanche effect: flipping a single input bit changes roughly half the output bits, so similar inputs produce wildly different, uncorrelated digests.

General-purpose hashing (file integrity, Git object IDs, checksums) uses fast algorithms like SHA-256 or BLAKE3. Password storage inverts the speed requirement: it deliberately uses slow, memory-hard algorithms like bcrypt, scrypt, or Argon2, so that brute-forcing a stolen password database is expensive even at scale.

Under the Hood

Merkle-Damgård construction: SHA-1 and SHA-256 process input in fixed-size blocks (512 bits for SHA-256), running each block through a compression function that mixes it with the running state from the previous block, chaining until the final block produces the digest. This structure is why length-extension attacks are possible against raw SHA-256: an attacker who knows H(secret || message) can compute H(secret || message || padding || extra) without knowing secret, because they can resume the internal state from the known digest. HMAC exists specifically to close this gap by hashing the key in twice with different padding.
Sponge construction: SHA-3 (Keccak) instead absorbs input into a large internal state and squeezes output out of it, which structurally avoids the length-extension weakness that Merkle-Damgård designs have.
Salting: a unique random value stored alongside each password hash, concatenated with the password before hashing. It defeats precomputed rainbow tables because an attacker would need a separate table per salt, and it ensures two users with the same password get different stored hashes.
Work factor / memory-hardness: Argon2id’s cost parameters (time, memory, parallelism) are tunable so that as hardware gets faster, the algorithm can be reconfigured to keep brute-force attempts expensive; memory-hardness specifically resists GPU and ASIC acceleration, which parallelize raw compute far more easily than they parallelize memory bandwidth.

One Compression Round of SHA-256

SHA-256 initializes eight 32-bit working variables a through h from fixed constants (the first 32 bits of the fractional parts of the square roots of the first eight primes), and processes each 512-bit block through 64 rounds. Each round:

Derives a 32-bit message schedule word W[t] for the round, either directly from the input block (rounds 0-15) or by mixing earlier schedule words together (rounds 16-63).
Computes T1 = h + Σ1(e) + Ch(e,f,g) + K[t] + W[t], where Σ1 is a fixed bitwise rotation-and-XOR function of e, Ch(e,f,g) is a bitwise “choose” function ((e AND f) XOR (NOT e AND g)), and K[t] is a round-specific constant.
Computes T2 = Σ0(a) + Maj(a,b,c), where Maj(a,b,c) is a bitwise majority function (each output bit is whichever value, 0 or 1, appears in at least two of the three inputs).
Shifts the working variables down one position: h=g, g=f, f=e, e=d+T1, d=c, c=b, b=a, then sets a=T1+T2.

After 64 rounds, each working variable is added back into the running hash state (H0..H7), and the next 512-bit block is processed the same way, chaining forward until the final block produces the 256-bit digest. The Ch and Maj functions provide non-linearity, and the Σ rotation functions spread each input bit’s influence across the whole state quickly, which together is what produces the avalanche effect.

Hashing Pipeline and the Avalanche Effect

Any length of input collapses to the same fixed-length digest, and changing a single input bit should flip roughly half the output bits, with no visible relationship between the two digests:

For example, hashing hello and hellp (one bit different in the last character) with SHA-256 produces 2cf24dba5fb0a30e... and fd06de3ec78de6a6..., digests that share no visible structure despite the near-identical input. That’s the avalanche effect working as designed: it’s what makes the digest useless for guessing anything about nearby inputs.

Why It Matters

Verifies data integrity: a downloaded file’s SHA-256 checksum confirms it wasn’t corrupted or tampered with in transit.
Underpins digital signatures, which sign a message’s hash rather than the message itself, making signing fast regardless of message size.
Is the only defensible way to store passwords: a breached database of properly hashed and salted passwords forces attackers into slow, per-password brute force instead of instant plaintext exposure.
Powers content-addressed systems (Git, IPFS, blockchain) where the hash of the data serves as its unique, tamper-evident identifier.
Enables efficient large-scale deduplication and integrity verification of huge datasets, since comparing two short digests is far cheaper than comparing the full underlying content byte for byte.

Common Pitfalls

Using fast general-purpose hashes (MD5, SHA-1, plain SHA-256) for password storage instead of memory-hard algorithms like Argon2id or bcrypt
Reusing a single global salt (or no salt) across all users, which reduces salting to nearly the same weakness as having none
Assuming a hash provides confidentiality; a hash reveals nothing about the input by itself, but a low-entropy input (a 4-digit PIN) can simply be brute-forced regardless of hashing
Continuing to use MD5 or SHA-1 for integrity or signature purposes after practical collision attacks were demonstrated against both
Truncating a strong hash’s output to save space, which reduces its effective collision resistance far below the design margin
Using a raw hash (SHA256(key || message)) as a message authentication code instead of HMAC, which is vulnerable to length-extension attacks against Merkle-Damgård constructions like SHA-256
Comparing digests with a non-constant-time comparison in security-sensitive code, which can leak timing information about how many leading bytes matched
Leaving a legacy, weakly hashed authentication path active alongside a properly upgraded one, since attackers only need the weakest reachable path to recover a reused password

Comparison

	MD5	SHA-1	SHA-256	BLAKE3	bcrypt / Argon2id
Digest size	128-bit	160-bit	256-bit	256-bit (variable)	Variable
Speed	Very fast	Fast	Fast	Very fast, parallelizable	Deliberately slow, memory-hard
Primary use	Legacy checksums	Legacy signatures, historic Git	General integrity, TLS, Bitcoin	General integrity	Password storage
Status	Broken, collisions practical since 2004	Broken, collisions demonstrated (SHAttered, 2017)	Secure, industry standard	Secure, modern alternative to SHA-2	Secure, purpose-built for passwords
Construction	Merkle-Damgård	Merkle-Damgård	Merkle-Damgård	Merkle tree of parallel chunks	Custom memory-hard design
Compression rounds	64	80	64	7 per chunk	Tunable via time parameter
Length-extension vulnerable	Yes	Yes	Yes	No	N/A, not a general-purpose MAC

Example

Git identifies every commit, tree, and blob by the SHA-1 hash of its content (migrating toward SHA-256 in newer repository formats); two files with identical content always produce the identical object ID, which is how Git deduplicates storage. On the password side, a properly implemented login system stores Argon2id(password, unique_salt) rather than the password itself, so a database breach doesn’t hand over usable credentials directly. openssl dgst -sha256 file.iso is the standard command-line way to verify a downloaded ISO’s integrity against a publisher’s published checksum, and sha256sum -c checksums.txt batch-verifies a whole directory of downloaded files against a publisher-supplied manifest in one pass.

Real-World Case Study

LinkedIn’s 2012 breach exposed roughly 6.5 million password hashes that had been hashed with unsalted SHA-1. Because there was no salt, identical passwords across different accounts produced identical hashes, and because SHA-1 is fast by design, attackers could run large precomputed dictionaries and brute-force attempts against the entire leaked set at once rather than one password at a time. Most of the hashes were cracked within days of the leak becoming public. The incident is now a standard reference case for why password storage needs both a slow, memory-hard algorithm and a unique per-user salt, not general-purpose speed-optimized hashing.

The 2015 Ashley Madison breach is a useful counterexample of the same lesson from the other direction. The site’s main password hashes used bcrypt correctly, with a proper per-user salt and a reasonable work factor, and remained largely uncracked at scale as a result. But researchers found a separate legacy field storing MD5(lowercase(username) || password) left over from an old authentication system that had never been fully removed. That fast, unsalted secondary hash was crackable at scale, and because most users reused the same password across both fields, cracking the weak legacy hash recovered the same password bcrypt was supposedly protecting. The incident shows that a single unhashed or weakly hashed leftover code path can undermine an otherwise correct implementation elsewhere in the same system.

Algorithm Families

MD family (MD5): 128-bit output, broken. Still seen in non-security contexts like deduplication or checksums where collision resistance doesn’t matter.
SHA-1: 160-bit output, broken for collision resistance since 2017, but pre-image resistance hasn’t been practically broken, so some legacy systems still treat it as acceptable for non-signature uses.
SHA-2 family (SHA-256, SHA-384, SHA-512): Merkle-Damgård construction, the current industry-standard general-purpose family, used throughout TLS, Bitcoin, and code signing.
SHA-3 (Keccak): sponge construction, standardized as a structurally independent backup to SHA-2 rather than a replacement, since a future break in Merkle-Damgård designs wouldn’t affect it.
BLAKE2/BLAKE3: designed for speed, often faster than SHA-2 and SHA-3 in software, with BLAKE3 adding native parallelism via a Merkle tree structure.
Argon2, bcrypt, scrypt: purpose-built for password hashing, deliberately slow and memory-hard, not used for general-purpose integrity checks.
KMAC/cSHAKE: SHA-3-derived constructions standardized by NIST for keyed hashing and customizable-output hashing, an alternative to HMAC that takes advantage of the sponge construction directly instead of nesting a Merkle-Damgård hash twice.

History

1990s: MD5 (1992) and SHA-1 (1995) became the dominant general-purpose hashes, widely used in TLS, code signing, and version control.
2004: Xiaoyun Wang’s team published practical collision attacks against MD5, ending its use for any security-relevant purpose within a few years.
2005–2017: theoretical weaknesses in SHA-1 were published in 2005; Google and CWI Amsterdam demonstrated a practical collision (SHAttered) in 2017, forcing browsers, CAs, and Git’s ecosystem to accelerate migration away from it.
2001–2015: NIST standardized SHA-2 (SHA-256/SHA-512) in 2001 as the successor family, then ran an open competition that selected Keccak as SHA-3 in 2015, specifically to have a structurally different backup algorithm in case SHA-2 was ever broken.
2015: Argon2 won the Password Hashing Competition, becoming the current recommended default for new password storage over older options like bcrypt and scrypt.
2020: BLAKE3 was released, combining a Merkle tree structure with a reduced-round BLAKE2-derived compression function to enable multi-core and SIMD parallelism without weakening security margins.

FAQ

Can a hash be decrypted? No, hashing isn’t encryption and isn’t reversible by design. “Cracking” a hash means guessing inputs and hashing them until one matches, not mathematically inverting the function.

Why not just use SHA-256 for passwords, since it’s secure? SHA-256 is secure for integrity but far too fast for password storage: modern GPUs compute billions of SHA-256 hashes per second, making brute force against short or common passwords practical. Argon2id is deliberately slow and memory-hard to make that brute force expensive.

What’s the difference between a hash and a checksum like CRC32? A checksum like CRC32 is designed to catch accidental corruption, not intentional tampering, and is trivial to forge. A cryptographic hash is designed so an adversary can’t deliberately construct a colliding input.

Do longer digests always mean more security? Not directly, security depends on the algorithm’s design, not just output length. SHA3-256 and SHA-256 have the same digest size but different internal structures and different resistance to specific attack classes like length extension.

Why does HMAC need two passes of the hash function instead of one? A naive H(key || message) is vulnerable to length extension on Merkle-Damgård hashes: an attacker who knows the digest can compute the hash of message || extra without knowing the key. HMAC’s nested construction, H(key_outer || H(key_inner || message)), breaks the attacker’s ability to resume the internal state, closing that gap regardless of which underlying hash is used.

What’s the difference between a salt and a pepper? A salt is unique per password and stored alongside the hash, defeating rainbow tables and cross-user pattern matching. A pepper is a single secret value shared across all password hashes and kept separately from the database, usually in application config or a secrets manager, so a database-only breach still leaves an attacker missing one ingredient needed to brute-force any hash.

Referenced by

Dig deeper

Spot an error in this note?