Binary Caching

The distinction between Compiler Caching (Module 2.4) and Binary Package Caching is Critical.

• Compiler Caching (ccache/sccache): Accelerates the compilation of your source code by hashing translation units. It operates at the object file (.o) level.
• Binary Package Caching (vcpkg/Conan): Accelerates the acquisition of dependencies by storing pre-compiled libraries. It operates at the package level (.lib``.dllHeaders).

Without binary caching, a Clean Build of a project involving heavy dependencies (e.g., Qt, LLVM, Boost, Folly) entails compiling millions of lines of C++ code, often taking hours. With binary Caching, this process becomes an I/O-bound download operation, taking seconds.

An Artifact Registry serves as the storage backend for pre-compiled binaries. This ensures that All developers and CI agents share the exact same binary artifacts, enforcing ABI consistency and Eliminating redundant compilation.

1. JFrog Artifactory: The dominant solution in C++. It supports generic storage, Conan repositories, and NuGet feeds (used by vcpkg). It offers distinct repository types:

• Local: Artifacts built internally.
• Remote: Proxies to public registries (ConanCenter) with caching.
• Virtual: An aggregation of Local and Remote for a single access point.

2. Sonatype Nexus: A strong alternative supporting similar repository formats (Conan, Raw, NuGet).

3. Cloud Storage (S3/GCS/Azure): For source-based managers like vcpkg, a raw blob storage bucket is often sufficient and more cost-effective than a dedicated artifact server.

Vcpkg originally built everything from source. Modern vcpkg implements a binary caching layer that Hashes the Package ABI (a combination of the source version, compiler flags, and build options).

If a matching hash exists in the cache, the build is skipped, and the artifacts are extracted Directly into the project.

Vcpkg does not use a config file for caching; it relies on the VCPKG_BINARY_SOURCES environment Variable. This follows the Provider Model.

Useful for sharing builds between different projects on the same disk.

# Windows
$env:VCPKG_BINARY_SOURCES="clear;files,C:\vcpkg-cache,readwrite"

# Linux/macOS
export VCPKG_BINARY_SOURCES="clear;files,/var/cache/vcpkg,readwrite"

Vcpkg treats binary artifacts as NuGet packages. This is the standard protocol for Windows-centric Or Enterprise pipelines.

export VCPKG_BINARY_SOURCES="clear;nuget,https://artifactory.example.com/nuget/vcpkg-cache,readwrite"

Note: This requires mono on Linux/macOS to run the NuGet executable.

Vcpkg recently added native support for HTTP PUT/GET, removing the need for NuGet wrapping. This is Ideal for GitHub Actions Cache or S3 presigned URLs.

export VCPKG_BINARY_SOURCES="clear;http,http://cache.internal/vcpkg/,readwrite"

1. Restoration: CI Agent checks if the hash of vcpkg.json + vcpkg-configuration.json matches a stored cache key.
2. Hit: vcpkg downloads binaries. Build time: 0s.
3. Miss: vcpkg builds from source.
4. Ingestion: Upon successful build, vcpkg zips the artifacts and uploads them to the defined readwrite source.

Conan was designed as a binary-first manager. It assumes the existence of a remote server (the “Remote”) to store packages.

Remotes are prioritized lists of servers.

# List remotes
conan remote list

# Add a private Artifactory
conan remote add internal-repo https://artifactory.company.com/artifactory/api/conan/conan-local

# Authentication
conan user -p <API_KEY> -r internal-repo <USERNAME>

Unlike vcpkg (which auto-caches on build), Conan requires explicit upload steps. This prevents Broken or temporary builds from polluting the shared registry.

CI Pipeline Step:

# 1. Build the package locally
conan create . -pr:b=default -pr:h=linux-release

# 2. Upload to the remote (Only if build succeeds)
# --all uploads recipes and binaries
conan upload my-pkg/1.0.0 -r internal-repo --all

To maximize efficiency, C++ CI pipelines should be split into Producer jobs and Consumer Jobs.

• Goal: Pre-warm the cache.
• Frequency: Runs nightly or when vcpkg.json / conanfile.py changes.
• Operation:

1. Clean environment.
2. Build all dependencies from source with readwrite access to the Artifact Registry.
3. Uploads artifacts.

• Goal: Rapid feedback for developers.
• Frequency: Runs on every commit.
• Operation:

1. Configured with read-only access to the Artifact Registry.
2. Downloads pre-compiled dependencies.
3. Compiles only the project source code.

This topology ensures that a developer changing main.cpp never waits for Qt6 to compile.

Artifacts in the registry should be Immutable. Once version 1.0.0 of a package is uploaded With a specific hash, it should never be overwritten. If the build configuration changes (e.g., Enabling SSL support), the Package ID changes, creating a new binary artifact rather than Overwriting the old one.

The “Producer” CI job should sign the artifacts. Consumers should verify signatures to ensure that The binary was built by a trusted CI agent and not injected by a compromised developer machine.

• Conan: Supports lockfiles (conan.lock) to strictly enforce dependency graph reproducibility.
• vcpkg: Relies on Git SHAs in the registry baseline for provenance.

While binary package caching accelerates dependency acquisition, compiler caching accelerates The compilation of your own source code by memoizing translation unit compilations.

A compiler cache intercepts invocations of the compiler (gcc``clang``cl.exe) and computes a hash key from all inputs that affect the output object file. If a matching key exists in the Cache, the cached .o file is returned instead of invoking the compiler.

The hash is computed from:

1. The compiler executable itself (hash of the binary, ensuring compiler upgrades invalidate the cache).
2. Source file content (the .cpp file).
3. Compiler flags (e.g., -std=c++23 -O2 -DNDEBUG).
4. Include file contents (all #includeD headers, recursively).
5. Preprocessor defines (both from flags and #define directives in included headers).
6. Command-line arguments (output file name, etc.).

If any of these inputs change, the cache misses and the compiler runs normally.

Ccache is the most widely used C/C++ compiler cache. It stores cached objects in a flat directory Structure on the local filesystem.

# Install
sudo apt install ccache        # Debian/Ubuntu
brew install ccache            # macOS

# Configure cache size (default is 5GB)
ccache -M 20G

# Prepend ccache to compiler path (CMake)
cmake -S . -B build -DCMAKE_C_COMPILER_LAUNCHER=ccache -DCMAKE_CXX_COMPILER_LAUNCHER=ccache

ccache -s

Sample output:

cache directory                     /home/user/.cache/ccache
primary config                      /home/user/.ccache/ccache.conf
cache hit (direct)                  12450
cache hit (preprocessed)            890
cache miss                          3420
cache hit rate                      79.54 %
called for link                     234
compile failed                      12
preprocessor error                  5

Ccache uses a four-level hash directory structure:

~/.cache/ccache/
├── a/
│   ├── b1/
│   │   └── c2d3e4f5/
│   │       └── manifest  # metadata (compiler, flags)
│   │       └── result    # compressed .o file

This structure avoids directory bloat (thousands of files in a single directory) while keeping Lookup O(1).

Sccache is a modern alternative to ccache written in Rust. Its key advantage is native support for Remote caching backends.

Sccache supports:

• S3-compatible storage (AWS S3, MinIO, Cloudflare R2)
• Redis (key-value store)
• GCS (Google Cloud Storage)
• Azure Blob Storage
• Memcached

# Set environment variables
export SCCACHE_BUCKET=my-build-cache
export SCCACHE_REGION=us-east-1
export SCCACHE_S3_KEY_PREFIX=cpp-project/
export AWS_ACCESS_KEY_ID=AKIA...
export AWS_SECRET_ACCESS_KEY=...

# Use with CMake
cmake -S . -B build \
  -DCMAKE_C_COMPILER_LAUNCHER=sccache \
  -DCMAKE_CXX_COMPILER_LAUNCHER=sccache

export SCCACHE_REDIS_ENDPOINT=redis://cache.internal:6379
export SCCACHE_REDIS_PASSWORD=secret

# Use with CMake
cmake -S . -B build \
  -DCMAKE_C_COMPILER_LAUNCHER=sccache \
  -DCMAKE_CXX_COMPILER_LAUNCHER=sccache

• ccache works transparently with GCC.
• GCC”s -fprofile-generate / -fprofile-use (PGO) is not cacheable by ccache because the output depends on runtime profile data, not just source inputs.

• ccache and sccache both work with Clang.
• Clang’s modules (-fmodules) can cause cache misses because the module cache is a side effect not captured in the hash key. Use -fno-implicit-module-maps if caching is critical.

• ccache supports MSVC but with caveats. MSVC uses #pragma once extensively, which ccache handles by normalizing include paths.
• sccache has native MSVC support and is generally preferred on Windows.

Cache entries are invalidated when any input to the hash changes. This is automatic and correct in The common case.

1. Timestamps in macros: #define BUILD_TIME __TIME__ causes every compilation to produce a unique hash. Use __DATE__ and __TIME__ only in non-cached builds.
2. Absolute include paths: If #include "/home/user/project/config.h" is used instead of a relative path, moving the project invalidates the cache. Prefer relative or project-relative includes.
3. Random seeds: Embedding rand() or UUID generation in headers at compile time breaks caching.

Cache corruption occurs when the stored artifact no longer matches the expected output. This can Happen due to:

• Disk filesystem errors: Check with fsck or use ccache -c to clear the cache.
• Concurrent writes: Multiple build agents writing to the same cache backend. Use SCCACHE_START_SERVER=1 to serialize access.
• Storage backend inconsistencies: S3 eventual consistency can rarely return stale data. Sccache adds content verification hashes to mitigate this.

# Clear ccache entirely
ccache -C

# Clear only corrupted entries (ccache 4.x+)
ccache -X

- name: ccache
  uses: hendrikmuhs/ccache-action@v1
  with:
    key: ${{ github.job }}-${{ runner.os }}-${{ hashFiles('**/CMakeLists.txt') }}
    max-size: 500M

This action automatically caches ~/.cache/ccache between workflow runs using GitHub Actions cache.

- name: Run sccache-cache
  uses: mozilla-actions/sccache-action@v1

- name: Build
  env:
    SCCACHE_GHA_ENABLED: "true''
  run: |
    cmake -S . -B build \
      -DCMAKE_C_COMPILER_LAUNCHER=sccache \
      -DCMAKE_CXX_COMPILER_LAUNCHER=sccache
    cmake --build build

Typical cache hit rates in real-world C++ CI pipelines:

Key observation: Header-heavy codebases see lower hit rates because changing a widely-included Header invalidates all translation units that include it. This is a fundamental property of the C++ Compilation model, not a cache limitation.

The optimal CI setup combines both layers:

1. Binary package cache (vcpkg/Conan): Eliminates dependency compilation entirely. Dependencies are downloaded as pre-built artifacts.
2. Compiler cache (ccache/sccache): Caches the project”s own compilation. Only changed translation units are recompiled.

# Pseudocode CI pipeline
steps:
  - restore: vcpkg binary cache # hit: 0s, miss: download/build deps
  - restore: sccache compiler cache # hit: 0s, miss: recompile
  - build: project sources
  - save: sccache compiler cache # store new .o files
  - save: vcpkg binary cache # store new packages (if deps changed)

This combined approach can reduce a 45-minute CI build to under 2 minutes for incremental changes.

1. Forgetting to set CMAKE_CXX_COMPILER_LAUNCHER: Without this, CMake invokes the compiler directly, bypassing the cache entirely. Verify with ccache -s that the “called” counter increases.
2. Cache poisoning with instrumented builds: Running sanitize``coverageOr debug builds populates the cache with instrumented .o files. Subsequent release builds may accidentally use these. Use separate cache directories or clear the cache between build types.
3. Ignoring the cache size limit: If the cache reaches its maximum size, ccache uses an LRU (Least Recently Used) eviction policy. Frequently-used cache entries may be evicted if the cache is too small. Monitor with ccache -s and adjust with ccache -M.

• Dependency Resolution — How package managers resolve version graphs
• vcpkg — Binary caching configuration for vcpkg
• Build Caching — CMake and build-system-level caching
• Code Coverage — Instrumented builds (should not be cached)

This topic covers the core concepts of binary caching, including underlying theory, practical implementation, and key applications.

Key concepts include:

• CPU architecture and the fetch-decode-execute cycle
• memory hierarchy (cache, RAM, virtual)
• input/output systems
• operating systems and scheduling
• interrupts and polling

Understanding these concepts thoroughly is essential for both examinations and practical programming, and requires both theoretical knowledge and hands-on practice.

Worked examples demonstrating the application of key concepts are covered in the detailed sub-pages linked above.