Preprocessing and AST Generation, and Object Code

Understanding the translation stages are critical for debugging compilation errors, optimizing build Times, and understanding. The pipeline processes one Translation Unit (TU) at a time. A TU is the output of the Preprocessor (a single source file plus all included headers).

The preprocessor is a legacy text-manipulation engine inherited from C. It operates on source text before the compiler understands C++ syntax. It does not know what a class or function is; it Only knows tokens.

1. File Inclusion (#include): The preprocessor locates the target file and recursively copy-pastes its entire contents into the current file. This explains why C++ compilation is slow: a single .cpp file often expands to tens of thousands of lines of code after headers are merged.
2. Macro Expansion (#define): Token replacement. This happens blindly, disregarding scope or namespaces.
3. Conditional Compilation (#ifdef): Text is stripped or kept based on conditions.

To debug macro expansion or header inclusion issues, you can stop the pipeline after preprocessing.

# -E: Run preprocessor only
# -P: Disable linemarker generation (cleaner output)
clang++ -E -P main.cpp -o main.i

:: /P: Preprocess to a file (.i)
cl /P main.cpp

The output (main.i) is the exact text the compiler frontend receives. If a syntax error occurs “inside a macro,” inspecting this file reveals the generated code causing the issue.

Once the preprocessor generates the TU, the compiler frontend (Clang/GCC) takes over. This stage Validates the code against the C++ Language Standard.

The source stream is converted into a sequence of tokens (kw_int``identifier_main``l_paren r_paren).

The tokens are assembled into an Abstract Syntax Tree (AST).

• Syntax: Does the code follow the grammar? (e.g., matching braces, semicolons).
• Semantics: Does the code make sense? (e.g., type checking, overload resolution, access control).

Template Note: At this stage, templates are parsed for basic syntax, but they are not fully Instantiated until they are used.

Clang allows direct visualization of the AST. This is invaluable for understanding how the compiler Interprets complex templates or precedence rules.

# -Xclang -ast-dump: Dump the AST to stdout
# -fsyntax-only: Do not generate code
clang++ -Xclang -ast-dump -fsyntax-only main.cpp

Example Output:

FunctionDecl 0x... <line:3:1, line:5:1> line:3:5 main 'int ()'
`-CompoundStmt 0x... <col:12, line:5:1>
  `-ReturnStmt 0x... <line:4:5, col:12>
    `-IntegerLiteral 0x... <col:12> 'int' 0

The AST is translated into an Intermediate Representation. This is an architecture-agnostic Assembly language. This is where the majority of optimizations (inlining, loop unrolling, Vectorization) occur.

• GCC: Uses GIMPLE and RTL.
• LLVM (Clang): Uses LLVM IR (.ll).

LLVM IR is a Static Single Assignment (SSA) based language. It models an infinite-register machine.

# -emit-llvm: Generate LLVM bitcode/text
# -S: Output as text (assembly/IR) rather than binary
# -O3: Apply optimizations to the IR
clang++ -S -emit-llvm -O3 main.cpp -o main.ll

# -fdump-tree-all: Dumps internal tree representations
g++ -fdump-tree-all main.cpp

Reading the optimized IR confirms whether high-level C++ abstractions (like std::vector or Lambdas) were successfully compiled away into efficient scalar operations (“Zero-Cost Abstractions”).

The Backend maps the generic IR to the specific Instruction Set Architecture (ISA) of the target Machine (x86_64, ARM64, WASM).

1. Instruction Selection: Mapping IR operations to machine opcodes.
2. Register Allocation: Mapping infinite IR registers to the limited physical CPU registers (RAX, RDI, etc.).
3. Instruction Scheduling: Reordering instructions to optimize pipeline throughput.

To see the final assembly before it is encoded into binary:

# -S: Compile only; do not assemble or link
# -masm=intel: Use Intel syntax (dest, source) instead of AT&T
g++ -S -masm=intel -O3 main.cpp -o main.s

The Assembler converts the textual assembly (.s) into a relocatable object file (.o or .obj). This file contains machine code, but it is not executable. It contains placeholders for memory Addresses (symbols) that are outside the current TU.

• ELF (Executable and Linkable Format): Linux, BSD, Android.
• PE/COFF (Portable Executable): Windows.
• Mach-O: macOS, iOS.

An object file is divided into Sections. Understanding these is essential for systems Programming and embedded work.

Analyze the contents of object files to debug linking errors or binary size bloat.

# Headers and Sections
readelf -h -S main.o

# Symbol Table (Defined vs Undefined)
nm -C main.o
# 'T' = Text (Defined), 'U' = Undefined (Needs Linking)

# Disassembly
objdump -d -C -M intel main.o

# Sections
otool -l main.o

# Symbols
nm -m main.o

# Headers and Sections
dumpbin /HEADERS main.obj

# Symbols
dumpbin /SYMBOLS main.obj

The compiler produces a collection of Object Files. These files contain optimized machine code But rely on “Undefined Symbols” (like std::cout or function calls to other files). They cannot run Yet. Resolving these symbols is the job of the Linker.

The C++ standard [N4950 §5.1.1] defines the translation of a source file into an executable as a Sequence of translation phases (also called “phases of translation”). These are numbered 1 Through 9:

A Translation Unit (TU) [N4950 §5.1.1] is the output of phases 1-7: a single preprocessed source File. The compiler processes one TU at a time, which is why the C++ model is called “separate Compilation.” The boundary between TUs is at the .cpp file level. When you compile a.cpp and b.cpp Separately, each is an independent TU. The compiler knows nothing about b.cpp when compiling a.cpp. This is why declarations (in headers) must be consistent across TUs — the linker cannot Detect type mismatches.

Macros are textual substitution with no understanding of C++ scope, types, or namespaces. This Causes a category of bugs that no other C++ feature can produce. Problem 1: Argument evaluation side effects.

#include <cstdio>
#define MAX(a, b) ((a) > (b) ? (a) : (b))
int main() {
    int x = 1;
    int y = MAX(x++, x++);  // UB: x++ is evaluated twice
    std::printf("y = %d, x = %d\n", y, x);
    // Possible output: y = 3, x = 4 (x incremented twice in one expression)
}

Problem 2: Operator precedence.

#include <cstdio>
#define SQUARE(x) x * x
int main() {
    int result = SQUARE(1 + 2);  // Expands to: 1 + 2 * 1 + 2 = 5, not 9
    std::printf("result = %d\n", result);  // Prints 5
}

Fixing with extra parentheses (#define SQUARE(x) ((x) * (x))) helps but doesn’t solve the Side-effect problem. The modern C++ solution is to use constexpr functions or inline functions Instead:

constexpr int square(int x) { return x * x; }  // Safe: evaluates argument once

Problem 3: Name shadowing.

#include <vector>
#define VALUE 42
int main() {
    // If any header or code defines a variable or macro named VALUE,
    // the preprocessor silently replaces it everywhere
    int VALUE = 10;  // Expands to: int 42 = 10; — compile error
}

Compilers define a set of macros that identify the platform, compiler version, and C++ standard. These are essential for writing portable code.

#include <iostream>
int main() {
    std::cout << "Compiler: "
#ifdef __clang__
              << "Clang " << __clang_major__ << "." << __clang_minor__
#elif defined(__GNUC__)
              << "GCC " << __GNUC__ << "." << __GNUC_MINOR__
#elif defined(_MSC_VER)
              << "MSVC " << _MSC_VER
#endif
              << "\n";
    std::cout << "C++ Standard: " << __cplusplus << "\n";
    // C++11: 201103L, C++14: 201402L, C++17: 201703L, C++20: 202002L, C++23: 202302L
    std::cout << "Platform: "
#ifdef _WIN32
              << "Windows"
#elif defined(__linux__)
              << "Linux"
#elif defined(__APPLE__)
              << "macOS"
#endif
              << "\n";
}

Important: Never use __cplusplus alone to detect compiler features. Use feature-test macros Defined by the standard (e.g., __cpp_constexpr``__cpp_concepts):

#if __cpp_concepts >= 201907L
    // Concepts are available
    template<typename T>
    concept Addable = requires(T a, T b) { a + b; };
#endif

Traditional include guards use preprocessor conditionals to prevent multiple inclusion:

#ifndef MY_HEADER_H
#define MY_HEADER_H
// ... header contents ...
#endif // MY_HEADER_H

#pragma once is a non-standard but universally supported directive that achieves the same result More concisely:

#pragma once
// ... header contents ...

The AST is a tree where each node represents a language construct. Understanding the AST structure Helps debug surprising compiler behavior.

const int* p = nullptr;

clang++ -Xclang -ast-dump -fsyntax-only input.cpp

The AST for this declaration includes:

VarDecl 0x... <line:1:1, col:20> col:15 p 'const int *' cinit
|-ValueStmt 0x...
| `-ImplicitCastExpr 0x... 'const int *' <NullToPointer>
|   `-CXXNullPtrLiteralExpr 0x... 'std::nullptr_t'

Note: the type is const int * (pointer to const int), not const int* (const pointer to int). The AST makes the actual type binding unambiguous, even though the human-readable syntax can be Confusing.

The compiler encodes function signatures, namespaces, and template arguments into a single string Called the mangled name. This is what appears in the object file’s symbol table. Name mangling Is necessary because the linker only sees flat strings, not C++ scope.

namespace math {
    int add(int a, int b) { return a + b; }
    double add(double a, double b) { return a + b; }
}

nm -C code.o
# _ZN4math3addEdd  →  math::add(double, double)
# _ZN4math3addEii  →  math::add(int, int)

The mangling scheme is compiler-specific (Itanium ABI for GCC/Clang, MSVC ABI for MSVC). Use nm -C (demangle) or c++filt to decode mangled names. The Itanium ABI mangling encodes:

• _Z prefix
• N for nested names
• Length-prefixed identifiers (4math = “math”, 3add = “add”)
• Type encodings (i = int, d = double, St = std::)

c++filt _ZN4math3addEii
# Output: math::add(int, int)

LLVM IR is a typed, SSA-based assembly language. Every value is computed exactly once and assigned To a register (%0``%1Etc.). This makes dataflow analysis trivial for the optimizer.

int sum(int n) {
    int total = 0;
    for (int i = 0; i < n; ++i)
        total += i;
    return total;
}

clang++ -S -emit-llvm -O2 sum.cpp -o sum.ll

Key IR concepts visible in the output:

define i32 @sum(i32 %n) {
entry:
  %cmp = icmp sgt i32 %n, 0
  %nminus1 = select i1 %cmp, i32 %n, i32 0
  %mul = mul i32 %nminus1, %n
  %div = lshr i32 %mul, 1
  ret i32 %div
}

The loop has been replaced with n*(n-1)/2 — a textbook example of loop optimization. This is why Reading the optimized IR is critical: it confirms that your high-level abstractions compiled away to Efficient code.

GCC uses a different IR called GIMPLE. It is a simplified form of C where every statement is an Assignment, a function call, or a branch — no complex expressions.

g++ -fdump-tree-optimized sum.cpp

When the compiler generates an object file, it cannot know the final addresses of symbols defined in Other translation units. It emits relocation entries — instructions to the linker saying “patch This offset with the final address of symbol X.”

# View relocations in an object file
readelf -r main.o

Each relocation entry specifies:

• Offset: Where in the section to patch.
• Type: How to apply the patch (absolute, PC-relative, GOT-relative, etc.).
• Symbol: Which symbol’s address to use.
• Addend: An optional constant offset. The linker resolves these relocations by computing the final virtual address of each symbol and Writing it into the appropriate location in the machine code.

• Debug builds without -g produce useless backtraces. The compiler emits DWARF debug info (.debug_info``.debug_line) only when -g is specified. Without it, gdb and addr2line cannot map addresses to source lines.
• Not using -fPIC for shared library code. Position-Independent Code generates PC-relative addressing instead of absolute addressing. Without it, the loader must perform text relocations (patching code sections at load time), which is slow and prevents code segment sharing across processes.
• Stripping before creating a debug link. Always run objcopy --only-keep-debug before strip and use objcopy --add-gnu-debuglink to associate the debug file with the stripped binary. Otherwise, you lose the ability to debug crashes in production.

1. Memorising content without understanding the underlying principles. This leads to poor application in unfamiliar contexts.
2. Focusing only on content knowledge without developing exam technique and question-answering skills.
3. Ignoring feedback from marked work and failing to address recurring weaknesses.
4. Not making connections between different topics within the subject to build a coherent understanding.

The key principles covered in this topic are linked in the sub-pages above. Focus on understanding the definitions, applying the formulas or frameworks, and evaluating strengths and limitations of each approach.

Worked examples demonstrating the application of key concepts are covered in the detailed sub-pages linked above.