Shared Dictionaries — When Browsers Only Download What Actually Changed

Posted on: 4/22/2026 9:12:24 AM

The Problem: Why Traditional Compression Falls Short

Every time you deploy a new version of your JavaScript bundle — even changing a single line — the entire compressed file must be transmitted from scratch. Gzip and Brotli compress well against the original, but they have no idea that the browser already has 99% of the identical content from the previous load.

With modern development patterns — CI/CD deploying multiple times daily, AI-assisted coding accelerating ship velocity — this problem compounds rapidly. Ship 10 small changes per day and you've effectively opted out of caching 10 times.

Gzip → Brotli → Zstandard: A History of Web Compression

Gzip — The 30-Year Standard

Born in 1992, using the DEFLATE algorithm with a 32 KB window. Gzip remains the default for most web servers due to universal compatibility. However, its 32 KB dictionary window means it can't leverage repeating patterns that are far apart in large files.

Brotli — Google's Leap Forward

Released in 2015, Brotli ships with a built-in dictionary containing common web patterns (HTML tags, CSS properties, JavaScript keywords). The result: 10-20% better compression than Gzip for web content, with a window up to 16 MB. By 2026, Brotli is supported by 98%+ of browsers.

Zstandard (Zstd) — Speed Meets Efficiency

Developed by Facebook and standardized via RFC 8878, Zstandard achieves compression ratios nearly matching Brotli but significantly faster — 42% faster compression at equivalent ratios. Zstd excels with custom dictionaries because its architecture was designed from the ground up for dictionary-based compression.

Shared Dictionaries: Delta Compression for HTTP

The idea is simple yet powerful: instead of compressing from zero, the server uses the old version the client already cached as a dictionary. The result? Only the diff is transmitted — exactly how git stores changes between commits.

sequenceDiagram
    participant B as Browser
    participant S as Server

    Note over B,S: First Load
    B->>S: GET /js/app.bundle.js
    S->>B: 200 OK (92 KB Brotli)
Use-As-Dictionary: match="/js/app.bundle.js"
    Note over B: Cache file + mark as dictionary

    Note over B,S: Subsequent Load (file changed)
    B->>S: GET /js/app.bundle.js
Available-Dictionary: :hash-abc:
Accept-Encoding: br, zstd, dcb, dcz
    S->>B: 200 OK (2.6 KB dcz)
Only the diff is sent!
    Note over B: Decompress using cached dictionary

New HTTP Headers

Use-As-Dictionary — The server sends this header with the response, telling the browser: "Keep this file because it'll be useful later." It contains a match pattern for the URLs this dictionary applies to.

HTTP/1.1 200 OK
Content-Type: application/javascript
Content-Encoding: br
Use-As-Dictionary: match="/js/app.bundle.js"

Available-Dictionary — The browser sends this header in subsequent requests, including the SHA-256 hash of the cached dictionary, telling the server: "Here's what I already have."

GET /js/app.bundle.js HTTP/1.1
Available-Dictionary: :pZGm1Av0IEBKARczz7exkNYsZb8LzaMrV7J32a2fCG4=:
Accept-Encoding: br, zstd, dcb, dcz

dcb / dcz — Two new Content-Encodings:

• dcb: Dictionary-Compressed Brotli
• dcz: Dictionary-Compressed Zstandard

Real-World Performance: Impressive Numbers

Case Study: YouTube Desktop

Case Study: Amazon Product Pages

Server Implementation

Node.js with Zstandard Dictionary

const zlib = require('zlib');
const fs = require('fs');
const crypto = require('crypto');

// Create dictionary from old version
const oldBundle = fs.readFileSync('dist/app.v1.js');
const newBundle = fs.readFileSync('dist/app.v2.js');

// Compress WITHOUT dictionary
const normalCompressed = zlib.zstdCompressSync(newBundle);
console.log(`Standard Zstd: ${normalCompressed.length} bytes`);

// Compress WITH dictionary (old version as dictionary)
const dictCompressed = zlib.zstdCompressSync(newBundle, {
  dictionary: oldBundle
});
console.log(`Zstd + dictionary: ${dictCompressed.length} bytes`);

// Calculate hash for Available-Dictionary header
const dictHash = crypto
  .createHash('sha256')
  .update(oldBundle)
  .digest('base64');
console.log(`Dictionary hash: :${dictHash}:`);

Express.js Middleware

const express = require('express');
const app = express();

// Dictionary version storage
const dictionaryStore = new Map();

app.get('/js/:file', (req, res) => {
  const filePath = `dist/${req.params.file}`;
  const content = fs.readFileSync(filePath);
  const dictHeader = req.headers['available-dictionary'];
  const acceptEnc = req.headers['accept-encoding'] || '';

  if (dictHeader && acceptEnc.includes('dcz')) {
    // Client has a dictionary — send delta only
    const dictHash = dictHeader.replace(/:/g, '');
    const dictionary = dictionaryStore.get(dictHash);

    if (dictionary) {
      const delta = zlib.zstdCompressSync(content, { dictionary });
      res.set('Content-Encoding', 'dcz');
      res.set('Vary', 'Accept-Encoding, Available-Dictionary');
      return res.send(delta);
    }
  }

  // Fallback: standard compression + mark as dictionary for next time
  const compressed = zlib.brotliCompressSync(content);
  const hash = crypto.createHash('sha256').update(content).digest('base64');

  dictionaryStore.set(hash, content);

  res.set('Content-Encoding', 'br');
  res.set('Use-As-Dictionary', `match="/js/${req.params.file}"`);
  res.set('Vary', 'Accept-Encoding, Available-Dictionary');
  res.send(compressed);
});

Nginx Configuration

# Enable Zstd compression
zstd on;
zstd_comp_level 3;
zstd_types application/javascript application/json text/css;

# Forward Shared Dictionary headers
proxy_set_header Available-Dictionary $http_available_dictionary;
proxy_pass_header Use-As-Dictionary;

# Vary header for correct caching
add_header Vary "Accept-Encoding, Available-Dictionary" always;

Cloudflare Shared Dictionaries: 3-Phase Roadmap

graph LR
    subgraph Phase1["Phase 1: Passthrough (Beta 04/2026)"]
        A1[Origin manages dictionaries] --> A2[Cloudflare forwards headers]
        A2 --> A3[Extended cache keys
Vary: Available-Dictionary]
    end
    subgraph Phase2["Phase 2: Managed Dictionaries"]
        B1[Admin configures rules] --> B2[Edge injects headers]
        B2 --> B3[Delta compress at edge]
    end
    subgraph Phase3["Phase 3: Auto Detection"]
        C1[Cloudflare detects
versioned URL patterns] --> C2[Auto-stores old versions]
        C2 --> C3[Auto delta compress]
    end
    Phase1 --> Phase2 --> Phase3

    style A1 fill:#f8f9fa,stroke:#e94560,color:#2c3e50
    style A2 fill:#f8f9fa,stroke:#e94560,color:#2c3e50
    style A3 fill:#f8f9fa,stroke:#e94560,color:#2c3e50
    style B1 fill:#e94560,stroke:#fff,color:#fff
    style B2 fill:#e94560,stroke:#fff,color:#fff
    style B3 fill:#e94560,stroke:#fff,color:#fff
    style C1 fill:#2c3e50,stroke:#fff,color:#fff
    style C2 fill:#2c3e50,stroke:#fff,color:#fff
    style C3 fill:#2c3e50,stroke:#fff,color:#fff

Security: Lessons from SDCH and the RFC 9842 Solution

In 2008, Google implemented SDCH (Shared Dictionary Compression over HTTP) in Chrome — a similar concept. However, SDCH was removed in 2017 due to critical security vulnerabilities:

• CRIME/BREACH attacks — attackers could infer encrypted response contents by observing compression size variations
• Same-Origin Policy violations — dictionaries could be shared cross-origin, creating data leakage channels

RFC 9842 (standardized September 2025) addresses these comprehensively:

Practical Adoption Strategy for 2026 Projects

graph TD
    START[Assess Project] --> Q1{Deploy > once/week?}
    Q1 -->|Yes| Q2{JS/CSS bundles > 100KB?}
    Q1 -->|No| SKIP[Brotli/Zstd is sufficient]
    Q2 -->|Yes| Q3{Targeting Chrome 130+?}
    Q2 -->|No| SKIP
    Q3 -->|Yes| IMPL[Implement Shared Dictionaries]
    Q3 -->|No| WAIT[Wait for Safari/Firefox support]
    IMPL --> S1[Step 1: Add Use-As-Dictionary header]
    S1 --> S2[Step 2: Handle Available-Dictionary requests]
    S2 --> S3[Step 3: Serve dcb/dcz responses]
    S3 --> S4[Step 4: Monitor compression ratio]

    style START fill:#f8f9fa,stroke:#e94560,color:#2c3e50
    style IMPL fill:#e94560,stroke:#fff,color:#fff
    style SKIP fill:#f8f9fa,stroke:#e0e0e0,color:#888
    style WAIT fill:#f8f9fa,stroke:#ff9800,color:#2c3e50
    style S1 fill:#f8f9fa,stroke:#4CAF50,color:#2c3e50
    style S2 fill:#f8f9fa,stroke:#4CAF50,color:#2c3e50
    style S3 fill:#f8f9fa,stroke:#4CAF50,color:#2c3e50
    style S4 fill:#f8f9fa,stroke:#4CAF50,color:#2c3e50

Implementation Checklist

1. Measure baseline — Check current bundle sizes with Gzip/Brotli. If the delta between 2 consecutive versions is small (under 5% changes), Shared Dictionaries will be extremely effective.
2. Choose algorithm — dcz (Zstandard) is recommended: significantly faster than Brotli when using dictionaries, supports larger dictionaries.
3. Configure server — Add Use-As-Dictionary header for static assets (JS, CSS, WASM). Handle Available-Dictionary header from clients.
4. CDN compatibility — Ensure your CDN (Cloudflare, CloudFront, Fastly) doesn't strip the new headers. Extend the Vary header accordingly.
5. Fallback — Always serve Brotli/Gzip for unsupported clients. This is progressive enhancement, not a breaking change.

The Future: From Web to the Agentic Internet

Cloudflare highlights a key trend: in the Agentic Web era, not just users but AI agents continuously fetch web pages — and they request far more frequently than humans. Shared Dictionaries reduce load not only for end-users but for the entire infrastructure as agent traffic grows exponentially.

With RFC 9842 standardized, Chrome already supporting it, Firefox and Safari implementing, and Cloudflare preparing beta — 2026 is the year Shared Dictionaries move from experiment to production. This isn't a micro-optimization — it's an architectural shift that makes the web fundamentally faster for every subsequent deploy.

References

• Shared Dictionaries: compression that keeps up with the agentic web — Cloudflare Blog
• Supercharge compression efficiency with shared dictionaries — Chrome for Developers
• Dictionary Compression is finally here, and it's ridiculously good — HTTP Toolkit
• Compression Dictionary Transport — MDN Web Docs
• The Ultimate Guide to Shared Compression Dictionaries — DebugBear
• RFC 9842 — Compression Dictionary Transport
• Zstandard — Real-time data compression algorithm