{"id":"cpp-undefined-behavior","title":"Undefined behavior","sections":[{"id":"overview","title":"Overview","html":"<p>Here is a bug that every ML systems engineer meets sooner or later. A custom kernel computes the mean of each row of an activation matrix. It compiles without a warning, it runs without a crash, and it returns numbers that look entirely reasonable. The model's accuracy drops by half a point, and nobody can say why. Two weeks later someone notices that the inner loop says <code>c &#x3C;= cols</code> where it should say <code>c &#x3C; cols</code>, so every row's sum included one element too many, which for every row but the last was the first element of the next row. Nothing crashed, because almost all of those extra reads stayed inside the same buffer. The answer was simply wrong.</p>\n<p>In Python the same mistake is impossible to miss. Index past the end of a list and you get <code>IndexError</code> with a line number. Add two integers that are too big and Python makes room for the result. Use a variable before assigning it and you get <code>NameError</code>. C++ makes none of these checks. When a program indexes past the end of an array, reads freed memory or overflows a signed integer, the C++ standard says the behavior is <strong>undefined</strong>: the language makes no promise about what happens next. The program may crash, may print a wrong answer, may print the right answer today and a wrong one after a compiler upgrade, or may do something stranger still.</p>\n<p>This chapter explains what undefined behavior is and why C++ has it, walks through the cases you will actually meet, shows why turning on the optimizer makes hidden bugs appear, and then introduces the tools that find them: compiler warnings, AddressSanitizer, UndefinedBehaviorSanitizer, and the standard library's own checks. The lab commits eight bugs on purpose and lets the tools explain each one.</p>","headings":[]},{"id":"what-undefined-means","title":"What \"undefined\" means","html":"<h3 id=\"what-undefined-means\">What \"undefined\" means</h3>\n<p>The C++ standard sorts the behavior of programs into a few kinds, and the differences matter:</p>\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n<div class=\"table-wrap\"><table><thead><tr><th>Kind</th><th>What the standard says</th><th>Example</th></tr></thead><tbody><tr><td>Well defined</td><td>Exactly one result</td><td><code>7 / 2</code> is 3</td></tr><tr><td>Implementation-defined</td><td>Each compiler picks a result and documents it</td><td><code>sizeof(long)</code> is 8 on Linux and 4 on 64-bit Windows</td></tr><tr><td>Unspecified</td><td>One of several results, and the compiler need not tell you which</td><td>the order in which <code>f(a(), b())</code> calls <code>a</code> and <code>b</code></td></tr><tr><td>Undefined</td><td>No requirements at all</td><td>reading <code>v[10]</code> from a 10-element array</td></tr></tbody></table></div>\n<p>The last row is stronger than it looks. Undefined behavior does not mean \"you get some garbage value\". The standard's section on program execution says that once a run of the program reaches undefined behavior, it places no requirement on that run, \"not even with regard to operations preceding the first undefined operation\". A compiler is therefore allowed to assume that undefined behavior never happens, and to transform your code on that assumption. Most of the strange effects later in this chapter come from that one rule.</p>\n<p>Why would a language be designed this way? Because every check costs time, and C++ was designed so that you pay only for what you ask for. If every <code>v[i]</code> had to check <code>i</code> against the size, every inner loop of every matmul would carry a compare and a branch. If signed overflow had to wrap, the compiler could not assume that <code>i + 1 > i</code> in a loop counter, and it would lose optimizations that matter, such as turning a 32-bit loop index into a 64-bit pointer increment. Leaving these cases undefined lets the compiler generate the fastest code for correct programs. The price is that incorrect programs can behave in any way at all, and the job of finding the incorrect parts falls on you and your tools.</p>\n<p>That price is not theoretical. The Chromium project reports that around 70% of its serious security bugs are memory safety bugs, which is to say undefined behavior of the kinds below. In ML systems the more common symptom is quieter: numbers that look plausible and are wrong.</p>","headings":[{"id":"what-undefined-means","title":"What \"undefined\" means","depth":3}]},{"id":"the-cases-you-will-meet","title":"The cases you will meet","html":"<h3 id=\"the-cases-you-will-meet\">The cases you will meet</h3>\n<p>Most undefined behavior in real code falls into a handful of patterns. Each one below has a mode in the lab.</p>\n<h4 id=\"reading-or-writing-out-of-bounds\">Reading or writing out of bounds</h4>\n<p>Accessing an array, a <code>std::vector</code> through <code>[]</code>, or any buffer through a pointer outside its bounds is undefined. What actually happens depends on what lies next to the buffer. Here is the row-mean bug from the start of the chapter, in the form the lab uses:</p>\n<pre class=\"shiki shiki-themes github-light github-dark\" style=\"background-color:#fff;--shiki-dark-bg:#24292e;color:#24292e;--shiki-dark:#e1e4e8\" tabindex=\"0\"><code><span class=\"line\"><span style=\"color:#6F42C1;--shiki-dark:#B392F0\">std</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">::</span><span style=\"color:#6F42C1;--shiki-dark:#B392F0\">vector</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">&#x3C;</span><span style=\"color:#D73A49;--shiki-dark:#F97583\">float</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">> </span><span style=\"color:#6F42C1;--shiki-dark:#B392F0\">m</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">(</span><span style=\"color:#6F42C1;--shiki-dark:#B392F0\">rows</span><span style=\"color:#D73A49;--shiki-dark:#F97583\"> *</span><span style=\"color:#E36209;--shiki-dark:#FFAB70\"> cols</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">);</span><span style=\"color:#6A737D;--shiki-dark:#6A737D\">       // a rows x cols matrix, stored row by row</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F97583\">const</span><span style=\"color:#D73A49;--shiki-dark:#F97583\"> float*</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\"> data </span><span style=\"color:#D73A49;--shiki-dark:#F97583\">=</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\"> m.</span><span style=\"color:#6F42C1;--shiki-dark:#B392F0\">data</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">();</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F97583\">for</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\"> (</span><span style=\"color:#D73A49;--shiki-dark:#F97583\">int</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\"> r </span><span style=\"color:#D73A49;--shiki-dark:#F97583\">=</span><span style=\"color:#005CC5;--shiki-dark:#79B8FF\"> 0</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">; r </span><span style=\"color:#D73A49;--shiki-dark:#F97583\">&#x3C;</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\"> rows; </span><span style=\"color:#D73A49;--shiki-dark:#F97583\">++</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">r) {</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F97583\">  float</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\"> sum </span><span style=\"color:#D73A49;--shiki-dark:#F97583\">=</span><span style=\"color:#005CC5;--shiki-dark:#79B8FF\"> 0.0</span><span style=\"color:#D73A49;--shiki-dark:#F97583\">f</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">;</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F97583\">  for</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\"> (</span><span style=\"color:#D73A49;--shiki-dark:#F97583\">int</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\"> c </span><span style=\"color:#D73A49;--shiki-dark:#F97583\">=</span><span style=\"color:#005CC5;--shiki-dark:#79B8FF\"> 0</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">; c </span><span style=\"color:#D73A49;--shiki-dark:#F97583\">&#x3C;=</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\"> cols; </span><span style=\"color:#D73A49;--shiki-dark:#F97583\">++</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">c)</span><span style=\"color:#6A737D;--shiki-dark:#6A737D\">        // the bug: &#x3C;= instead of &#x3C;</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">    sum </span><span style=\"color:#D73A49;--shiki-dark:#F97583\">+=</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\"> data[r </span><span style=\"color:#D73A49;--shiki-dark:#F97583\">*</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\"> cols </span><span style=\"color:#D73A49;--shiki-dark:#F97583\">+</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\"> c];</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">  means[r] </span><span style=\"color:#D73A49;--shiki-dark:#F97583\">=</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\"> sum </span><span style=\"color:#D73A49;--shiki-dark:#F97583\">/</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\"> cols;</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">}</span></span></code></pre>\n<p>For every row except the last, the extra element is the first element of the next row. That read is a bug in your program, but it is not undefined behavior as far as the language is concerned, because it stays inside the vector's buffer, so no tool that checks memory can see anything wrong. Only the last row reads past the end of the buffer, and that single read is undefined behavior. Whether it crashes depends on whether the next bytes belong to the program. Usually they do, so the program reads whatever happens to be there and carries on. This is the shape of most out-of-bounds bugs in tensor code: an off-by-one in a loop bound, an index computed with the wrong stride, or a shape that does not match the buffer.</p>\n<p>Writes are worse than reads, because they corrupt whatever lives next door. A stack array written one past its end can overwrite another local variable or the function's saved return address. A heap buffer written past its end can overwrite the allocator's bookkeeping, and the program then crashes much later, inside an unrelated <code>new</code> or <code>delete</code>, far from the line that caused it.</p>\n<h4 id=\"use-after-free-and-dangling-references\">Use after free and dangling references</h4>\n<p><a href=\"/learn/cpp-memory#dangling-a-pointer-that-outlives-its-object\">Chapter 3</a> showed the classic case: a pointer into a <code>std::vector</code> that has since grown and moved its elements. Any pointer, reference or view that outlives the object it refers to dangles, and using it is undefined. One version deserves special mention because modern C++ makes it easy to write:</p>\n<pre class=\"shiki shiki-themes github-light github-dark\" style=\"background-color:#fff;--shiki-dark-bg:#24292e;color:#24292e;--shiki-dark:#e1e4e8\" tabindex=\"0\"><code><span class=\"line\"><span style=\"color:#6F42C1;--shiki-dark:#B392F0\">std</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">::</span><span style=\"color:#6F42C1;--shiki-dark:#B392F0\">string</span><span style=\"color:#6F42C1;--shiki-dark:#B392F0\"> make_prompt</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">();</span></span>\n<span class=\"line\"></span>\n<span class=\"line\"><span style=\"color:#6F42C1;--shiki-dark:#B392F0\">std</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">::string_view view </span><span style=\"color:#D73A49;--shiki-dark:#F97583\">=</span><span style=\"color:#6F42C1;--shiki-dark:#B392F0\"> make_prompt</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">();</span><span style=\"color:#6A737D;--shiki-dark:#6A737D\">   // the std::string returned here is a temporary</span></span>\n<span class=\"line\"><span style=\"color:#6F42C1;--shiki-dark:#B392F0\">std</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">::</span><span style=\"color:#6F42C1;--shiki-dark:#B392F0\">printf</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">(</span><span style=\"color:#032F62;--shiki-dark:#9ECBFF\">\"</span><span style=\"color:#005CC5;--shiki-dark:#79B8FF\">%c\\n</span><span style=\"color:#032F62;--shiki-dark:#9ECBFF\">\"</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">, view[</span><span style=\"color:#005CC5;--shiki-dark:#79B8FF\">0</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">]);</span><span style=\"color:#6A737D;--shiki-dark:#6A737D\">            // the temporary was destroyed at the end of the line above</span></span></code></pre>\n<p>A <code>std::string_view</code> is a pointer and a length that refers to characters owned by someone else, much like the <code>std::span</code> of chapter 4. Here the owner is a temporary <code>std::string</code>, which is destroyed at the end of the statement that created it, so the view refers to freed memory from the very next line. Tokenizers and request parsers in inference servers pass <code>string_view</code>s around precisely because they avoid copies, so this bug appears in exactly the code that cares most about speed. Clang notices this simple case and warns with <code>-Wdangling-gsl</code>, which is on by default. Most dangling views are not this simple, and no compiler warns about them.</p>\n<h4 id=\"freeing-twice\">Freeing twice</h4>\n<p>Calling <code>delete</code> twice on the same pointer, or <code>delete</code> on a pointer that did not come from <code>new</code>, is undefined. It usually happens when two parts of a program each believe they own a buffer. Modern allocators catch some cases themselves: glibc's allocator aborts with a message that begins <code>free(): double free detected</code> for small blocks, and macOS's allocator aborts with a message saying that the pointer being freed was not allocated. Neither is guaranteed, and in the bad case the second <code>delete</code> succeeds and corrupts the allocator's lists, so that two later allocations receive the same memory. Chapter 8's smart pointers exist largely to make ownership clear enough that this cannot happen.</p>\n<h4 id=\"reading-uninitialized-memory\">Reading uninitialized memory</h4>\n<p><a href=\"/learn/cpp-syntax#a-variable-you-forgot-to-initialize\">Chapter 2</a> introduced this one: a local variable of a built-in type without an initializer holds whatever bits were already in its memory, and reading it is undefined.</p>\n<pre class=\"shiki shiki-themes github-light github-dark\" style=\"background-color:#fff;--shiki-dark-bg:#24292e;color:#24292e;--shiki-dark:#e1e4e8\" tabindex=\"0\"><code><span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F97583\">float</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\"> sum;</span><span style=\"color:#6A737D;--shiki-dark:#6A737D\">                      // no initializer</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F97583\">for</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\"> (</span><span style=\"color:#D73A49;--shiki-dark:#F97583\">int</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\"> i </span><span style=\"color:#D73A49;--shiki-dark:#F97583\">=</span><span style=\"color:#005CC5;--shiki-dark:#79B8FF\"> 0</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">; i </span><span style=\"color:#D73A49;--shiki-dark:#F97583\">&#x3C;</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\"> n; </span><span style=\"color:#D73A49;--shiki-dark:#F97583\">++</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">i) sum </span><span style=\"color:#D73A49;--shiki-dark:#F97583\">+=</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\"> x[i];</span></span></code></pre>\n<p>The result depends on what the stack held before, so it can differ between runs, between Debug and Release builds, and between machines, which makes it one of the hardest bugs to reproduce. The same applies to a <code>struct</code> member you forgot to set and to memory from <code>new float[n]</code>, which is not zeroed. Compilers often warn about the simple cases with <code>-Wall</code>. The C++26 standard changes the rules for local variables: reading one before it is set becomes <strong>erroneous behavior</strong>, which is still a bug that compilers are encouraged to diagnose, but no longer undefined, and the variable holds some fixed value instead of anything at all. Until your compiler implements it, both GCC and Clang offer <code>-ftrivial-auto-var-init=zero</code>, which zeroes uninitialized locals and is used in hardened builds of some large projects. The habit that prevents the bug costs nothing: write <code>float sum = 0.0f;</code>.</p>\n<h4 id=\"signed-integer-overflow\">Signed integer overflow</h4>\n<p>As chapter 2 explained, unsigned arithmetic wraps, but overflowing a signed integer is undefined. In AI infrastructure the classic trigger is an index or an offset computed in <code>int</code>. Take the logits of a batch: a tensor of shape <code>[batch, seq, vocab]</code>. With a sequence length of 2,048 and Llama 3's vocabulary of 128,256 tokens, the number of elements is:</p>\n<ul>\n<li>batch 8: 8 × 2,048 × 128,256 = 2,101,346,304, just under the largest <code>int</code>, 2,147,483,647;</li>\n<li>batch 16: 4,202,692,608, nearly twice the largest <code>int</code>.</li>\n</ul>\n<p>So <code>int offset = batch * seq * vocab - 1;</code> works in every test with batch 8 and is undefined in production with batch 16. On real hardware the multiplication usually wraps, giving a negative offset, and the kernel then reads or writes far outside the tensor. <a href=\"/learn/cpp-pointers#a-naive-matmul-with-pointers\">Chapter 4</a> met the same problem in matmul indexes, which is why PyTorch checks whether a tensor is small enough for 32-bit index math before choosing a kernel. The fix is to compute sizes and offsets in a 64-bit type, <code>std::int64_t</code> or <code>std::size_t</code>, and to cast before multiplying, not after: <code>static_cast&#x3C;std::int64_t>(batch) * seq * vocab</code>.</p>\n<h4 id=\"misaligned-and-type-punned-loads\">Misaligned and type-punned loads</h4>\n<p>Binary formats tempt you to read numbers straight out of a byte buffer:</p>\n<pre class=\"shiki shiki-themes github-light github-dark\" style=\"background-color:#fff;--shiki-dark-bg:#24292e;color:#24292e;--shiki-dark:#e1e4e8\" tabindex=\"0\"><code><span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F97583\">const</span><span style=\"color:#D73A49;--shiki-dark:#F97583\"> unsigned</span><span style=\"color:#D73A49;--shiki-dark:#F97583\"> char*</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\"> bytes </span><span style=\"color:#D73A49;--shiki-dark:#F97583\">=</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\"> file_data </span><span style=\"color:#D73A49;--shiki-dark:#F97583\">+</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\"> header_size;</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F97583\">float</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\"> scale </span><span style=\"color:#D73A49;--shiki-dark:#F97583\">=</span><span style=\"color:#D73A49;--shiki-dark:#F97583\"> *reinterpret_cast&#x3C;const</span><span style=\"color:#D73A49;--shiki-dark:#F97583\"> float*></span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">(bytes);</span><span style=\"color:#6A737D;--shiki-dark:#6A737D\">   // two problems</span></span></code></pre>\n<p>This line has two problems. First, a <code>float</code> must sit at an address that is a multiple of 4, as chapter 3 explained, and if <code>header_size</code> is odd, it does not. x86-64 and ARM64 processors both load a misaligned float correctly in ordinary code, so nothing visible goes wrong on a laptop, but the behavior is undefined, the compiler may generate vector instructions that do require alignment, and some processors fault. Second, the bytes are not a <code>float</code> object as far as the language is concerned, and reading memory through a pointer to an unrelated type breaks the <strong>strict aliasing</strong> rule, which lets the compiler assume that a <code>float*</code> and an <code>int*</code> never point at the same memory. The portable fix for both is to copy the bytes into a real variable:</p>\n<pre class=\"shiki shiki-themes github-light github-dark\" style=\"background-color:#fff;--shiki-dark-bg:#24292e;color:#24292e;--shiki-dark:#e1e4e8\" tabindex=\"0\"><code><span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F97583\">float</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\"> scale;</span></span>\n<span class=\"line\"><span style=\"color:#6F42C1;--shiki-dark:#B392F0\">std</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">::</span><span style=\"color:#6F42C1;--shiki-dark:#B392F0\">memcpy</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">(</span><span style=\"color:#D73A49;--shiki-dark:#F97583\">&#x26;</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">scale, bytes, sizeof scale);</span><span style=\"color:#6A737D;--shiki-dark:#6A737D\">         // correct for any alignment, and free in practice</span></span>\n<span class=\"line\"><span style=\"color:#6F42C1;--shiki-dark:#B392F0\">std</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">::</span><span style=\"color:#D73A49;--shiki-dark:#F97583\">uint32_t</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\"> bits </span><span style=\"color:#D73A49;--shiki-dark:#F97583\">=</span><span style=\"color:#6F42C1;--shiki-dark:#B392F0\"> std</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">::</span><span style=\"color:#6F42C1;--shiki-dark:#B392F0\">bit_cast</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">&#x3C;</span><span style=\"color:#6F42C1;--shiki-dark:#B392F0\">std</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">::</span><span style=\"color:#D73A49;--shiki-dark:#F97583\">uint32_t</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">>(scale);</span><span style=\"color:#6A737D;--shiki-dark:#6A737D\">   // C++20: the bits of a float, as chapter 2 showed</span></span></code></pre>\n<p>Compilers recognise a small <code>memcpy</code> like this and turn it into a single load, so the safe version costs nothing. Chapter 21 returns to this when it reads weight files with <code>mmap</code>.</p>\n<h4 id=\"data-races\">Data races</h4>\n<p>Two threads accessing the same memory at the same time, with at least one of them writing and nothing ordering the accesses, is a data race, and a data race is undefined behavior too. <code>++counter</code> from two threads can lose updates, and worse things can happen when the optimizer keeps a value in a register that another thread is changing. Part IV of the course is about doing this correctly, and chapter 16 introduces ThreadSanitizer, the tool that finds races.</p>","headings":[{"id":"the-cases-you-will-meet","title":"The cases you will meet","depth":3},{"id":"reading-or-writing-out-of-bounds","title":"Reading or writing out of bounds","depth":4},{"id":"use-after-free-and-dangling-references","title":"Use after free and dangling references","depth":4},{"id":"freeing-twice","title":"Freeing twice","depth":4},{"id":"reading-uninitialized-memory","title":"Reading uninitialized memory","depth":4},{"id":"signed-integer-overflow","title":"Signed integer overflow","depth":4},{"id":"misaligned-and-type-punned-loads","title":"Misaligned and type-punned loads","depth":4},{"id":"data-races","title":"Data races","depth":4}]},{"id":"why-the-optimizer-makes-bugs-appear","title":"Why the optimizer makes bugs appear","html":"<h3 id=\"why-the-optimizer-makes-bugs-appear\">Why the optimizer makes bugs appear</h3>\n<p>A common experience in C++ goes like this. The program works in a Debug build. You switch to Release, and the answers change or the program hangs. It is tempting to blame the compiler, but in almost every case the program had undefined behavior all along, and the optimizer made it visible.</p>\n<p>The reason is the rule from the first section. The optimizer assumes your program never has undefined behavior, and it uses that assumption to prove things about your code. Chapter 2 promised an example, and here it is:</p>\n<pre class=\"shiki shiki-themes github-light github-dark\" style=\"background-color:#fff;--shiki-dark-bg:#24292e;color:#24292e;--shiki-dark:#e1e4e8\" tabindex=\"0\"><code><span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F97583\">bool</span><span style=\"color:#6F42C1;--shiki-dark:#B392F0\"> next_is_bigger</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">(</span><span style=\"color:#D73A49;--shiki-dark:#F97583\">int</span><span style=\"color:#E36209;--shiki-dark:#FFAB70\"> x</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">) { </span><span style=\"color:#D73A49;--shiki-dark:#F97583\">return</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\"> x </span><span style=\"color:#D73A49;--shiki-dark:#F97583\">+</span><span style=\"color:#005CC5;--shiki-dark:#79B8FF\"> 1</span><span style=\"color:#D73A49;--shiki-dark:#F97583\"> ></span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\"> x; }</span></span></code></pre>\n<p>For every <code>x</code> except the largest <code>int</code>, this is true. For the largest <code>int</code>, <code>x + 1</code> overflows, which is undefined, so the compiler may assume it never happens. Under that assumption, <code>x + 1 > x</code> is always true, and GCC and Clang with optimization compile the whole function to \"return true\". In an unoptimized build, the addition usually happens on the hardware, wraps to the most negative <code>int</code>, and the function returns false for the largest <code>int</code>. Neither compiler is wrong, because the program asked a question that has no answer. The lab's <code>optimizer.cpp</code> shows both results.</p>\n<p>The same reasoning has removed safety checks in real software. In 2009, the Linux kernel's <code>tun</code> network driver contained code of this shape:</p>\n<pre class=\"shiki shiki-themes github-light github-dark\" style=\"background-color:#fff;--shiki-dark-bg:#24292e;color:#24292e;--shiki-dark:#e1e4e8\" tabindex=\"0\"><code><span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F97583\">struct</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\"> sock </span><span style=\"color:#D73A49;--shiki-dark:#F97583\">*</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">sk </span><span style=\"color:#D73A49;--shiki-dark:#F97583\">=</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\"> tun</span><span style=\"color:#D73A49;--shiki-dark:#F97583\">-></span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">sk;</span><span style=\"color:#6A737D;--shiki-dark:#6A737D\">    // dereferences tun</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F97583\">if</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\"> (</span><span style=\"color:#D73A49;--shiki-dark:#F97583\">!</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">tun) </span><span style=\"color:#D73A49;--shiki-dark:#F97583\">return</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\"> POLLERR;</span><span style=\"color:#6A737D;--shiki-dark:#6A737D\">     // checks tun for null, too late</span></span></code></pre>\n<p>Because <code>tun</code> had already been dereferenced on the first line, the compiler reasoned that it could not be null, since dereferencing a null pointer is undefined, and it deleted the check. An attacker who could arrange for <code>tun</code> to be null and map memory at address 0 could then take over the machine. The bug became CVE-2009-1897, and the kernel has been built with GCC's <code>-fno-delete-null-pointer-checks</code> ever since.</p>\n<p>Loops are affected too. In 2012, a GCC developer found that a new GCC version turned a loop in the SPEC CPU 2006 benchmark <code>464.h264ref</code> into an infinite loop. The loop's increment read <code>d[16]</code>, one past the end of a 16-element array, just before the test that would have ended the loop. Since that read is undefined, the compiler concluded that the loop must exit before the index reached 16, and it removed the exit test. The bug report was closed as invalid, because the benchmark, not the compiler, was at fault. The lesson for you is that \"it works in Debug\" is no evidence that a program is correct, and \"it broke after a compiler upgrade\" usually means a bug was already there.</p>","headings":[{"id":"why-the-optimizer-makes-bugs-appear","title":"Why the optimizer makes bugs appear","depth":3}]},{"id":"tools-that-find-it","title":"Tools that find it","html":"<h3 id=\"tools-that-find-it\">Tools that find it</h3>\n<p>You cannot find undefined behavior by reading output, because a program with undefined behavior often produces correct-looking output. You find it with tools, and they come in three layers: compiler warnings, which cost nothing; sanitizers, which instrument the program to check it as it runs; and checks inside the standard library itself.</p>\n<h4 id=\"compiler-warnings-come-first\">Compiler warnings come first</h4>\n<p>Every lab in this course compiles with <code>-Wall -Wextra -Wpedantic</code>, and that is the cheapest bug detector you have. Compilers warn about many of the simple cases: uninitialized variables, a <code>string_view</code> bound to a temporary, returning a reference to a local, comparing signed and unsigned values, and indexes that are out of bounds when both the index and the array size are known at compile time. Projects that care add <code>-Werror</code> in continuous integration, so that a warning fails the build instead of scrolling past. Warnings can only see what is visible at compile time, though, and most real bugs depend on values that arrive at run time.</p>\n<h4 id=\"addresssanitizer\">AddressSanitizer</h4>\n<p><strong>AddressSanitizer</strong>, or ASan, finds memory errors at run time: out-of-bounds reads and writes on the heap, the stack and global variables, use after free, use of a stack variable after its scope ends, and double free. You turn it on with one flag, <code>-fsanitize=address</code>, at both compile and link time, and it works with both GCC and Clang, including Apple's Clang.</p>\n<p>It works by keeping a second, smaller map of memory, called <strong>shadow memory</strong>, in which one byte describes eight bytes of the program's memory: whether they are valid to access. The compiler inserts a check of the shadow before every load and store. Each allocation is surrounded by small poisoned areas called <strong>redzones</strong>, so that a read just past the end of a buffer lands on poisoned memory, and freed memory is poisoned and held in a quarantine for a while before it is reused, so that a use after free also lands on poisoned memory. When a check fails, ASan stops the program and prints a report. The cost is real but modest: the Clang documentation gives a typical slowdown of 2x, which is fine for tests and debugging but not for production.</p>\n<p>An ASan report is long, and knowing where to look saves time. It has four parts:</p>\n<ol>\n<li>A headline that names the kind of error, such as <code>ERROR: AddressSanitizer: heap-buffer-overflow</code> or <code>heap-use-after-free</code>, followed by whether it was a read or a write and how many bytes.</li>\n<li>The stack trace of the bad access. The first frame in your own code is almost always the line to fix.</li>\n<li>A description of the memory involved: for an overflow, how many bytes past the end of which allocation the access landed; for a use after free, a second stack trace showing where the memory was freed and a third showing where it was allocated.</li>\n<li>A map of the shadow memory around the address, which you can usually skip.</li>\n</ol>\n<p>Build with <code>-g</code>, which CMake's Debug build type does for you, so that the traces show file names and line numbers, and keep <code>-fno-omit-frame-pointer</code>, so that the traces are complete.</p>\n<h4 id=\"undefinedbehaviorsanitizer\">UndefinedBehaviorSanitizer</h4>\n<p><strong>UndefinedBehaviorSanitizer</strong>, or UBSan, checks for undefined operations that are not about memory: signed integer overflow, division by zero, shifts by too many bits, misaligned loads and stores, dereferencing a null pointer, converting a float that is too large into an integer, and indexing an array out of bounds when the array's size is known to the compiler. You turn it on with <code>-fsanitize=undefined</code>. Its slowdown is small, so many projects run it alongside ASan, and <code>-fsanitize=address,undefined</code> builds both into one program. Its reports are one line each. Here is the one Clang's UBSan should print for the lab's <code>overflow 16</code> mode, after the full path of the file:</p>\n<pre class=\"shiki shiki-themes github-light github-dark\" style=\"background-color:#fff;--shiki-dark-bg:#24292e;color:#24292e;--shiki-dark:#e1e4e8\" tabindex=\"0\"><code><span class=\"line\"><span>ub_zoo.cpp:101:26: runtime error: signed integer overflow: 32768 * 128256 cannot be represented in type 'int'</span></span></code></pre>\n<p>By default UBSan prints the error and lets the program continue, so one run can report several problems. In tests you usually want the first error to fail the run instead, which <code>-fno-sanitize-recover=all</code> does.</p>\n<h4 id=\"the-others-memorysanitizer-threadsanitizer-and-valgrind\">The others: MemorySanitizer, ThreadSanitizer and Valgrind</h4>\n<p>Three more tools cover the gaps:</p>\n<ul>\n<li><strong>MemorySanitizer</strong> (<code>-fsanitize=memory</code>) finds reads of uninitialized memory, which neither ASan nor UBSan checks. It works only with Clang and only on Linux and a few BSDs, not on macOS, and it needs every library in the program, including the C++ standard library, to be built with it, or it reports false errors.</li>\n<li><strong>ThreadSanitizer</strong> (<code>-fsanitize=thread</code>) finds data races. Chapter 16 uses it. It slows a program much more than ASan does, by 5x to 15x according to the Clang documentation.</li>\n<li><strong>Valgrind</strong>'s Memcheck tool runs an unmodified program on a simulated processor and catches out-of-bounds heap accesses, use after free, leaks and reads of uninitialized values, all without recompiling. It is far slower than the sanitizers, often tens of times, and it runs on Linux but not on Apple silicon Macs.</li>\n</ul>\n<p>ASan, MemorySanitizer and ThreadSanitizer each need their own shadow memory, so you cannot combine any two of them in one build. The usual arrangement in continuous integration is one build with ASan and UBSan together, one with ThreadSanitizer, and, on Linux with Clang, one with MemorySanitizer. Large C++ projects in AI infrastructure work this way: PyTorch's continuous integration includes AddressSanitizer builds, and llama.cpp's CMake files have options such as <code>LLAMA_SANITIZE_ADDRESS</code>, <code>LLAMA_SANITIZE_UNDEFINED</code> and <code>LLAMA_SANITIZE_THREAD</code>.</p>\n<h4 id=\"what-sanitizers-cannot-see\">What sanitizers cannot see</h4>\n<p>Sanitizers are powerful, and it is worth knowing exactly where they stop.</p>\n<p><strong>They check only the code that runs.</strong> A sanitizer instruments the program and watches it execute, so a bug on a path your test never takes stays hidden. The lab's <code>overflow</code> mode is clean for batch 8 and reports an error only for batch 16. Sanitizers are as good as the inputs you run them with.</p>\n<p><strong>They check allocations, not your intentions.</strong> ASan knows where each allocation starts and ends. It does not know that your buffer holds a matrix, or where one row stops and the next begins. In the row-mean bug, the extra reads for the first rows stay inside the buffer, and ASan says nothing about them. It reports only the last row's read, because that one leaves the allocation. Fixing the line ASan points at fixes all the rows, but only because the same loop produced every read.</p>\n<p><strong>Memory pools hide bugs from them.</strong> The same blindness grows with the size of the allocation. PyTorch's caching allocator and vLLM's KV cache, which chapter 3 described, take one large block of GPU memory and hand out pieces of it. A kernel that writes past the end of its tensor usually lands inside a neighboring tensor in the same block, so a memory checker that watches allocations sees a perfectly legal access. NVIDIA's <code>compute-sanitizer</code>, the GPU counterpart of ASan, has the same limit, which is why PyTorch lets you turn its caching allocator off with the environment variable <code>PYTORCH_NO_CUDA_MEMORY_CACHING=1</code> when you hunt for this kind of bug. Part VI of the course uses <code>compute-sanitizer</code>.</p>\n<p>Now test your sense of which tool catches what. Each snippet below has exactly one bug. Decide which sanitizer reports it when the code runs, assuming the buggy line executes with the values shown.</p>\n<div class=\"widget-slot\" data-widget=\"ub-detective\"></div>","headings":[{"id":"tools-that-find-it","title":"Tools that find it","depth":3},{"id":"compiler-warnings-come-first","title":"Compiler warnings come first","depth":4},{"id":"addresssanitizer","title":"AddressSanitizer","depth":4},{"id":"undefinedbehaviorsanitizer","title":"UndefinedBehaviorSanitizer","depth":4},{"id":"the-others-memorysanitizer-threadsanitizer-and-valgrind","title":"The others: MemorySanitizer, ThreadSanitizer and Valgrind","depth":4},{"id":"what-sanitizers-cannot-see","title":"What sanitizers cannot see","depth":4}]},{"id":"checked-and-unchecked-indexing","title":"Checked and unchecked indexing","html":"<h3 id=\"checked-and-unchecked-indexing\">Checked and unchecked indexing</h3>\n<p><code>std::vector</code> offers two ways to read an element. <code>v[i]</code> does no checking at all: if <code>i</code> is out of range, the behavior is undefined, exactly as with a raw pointer. <code>v.at(i)</code> checks <code>i</code> against <code>v.size()</code> and throws a <code>std::out_of_range</code> exception if it is out of range, which stops the program with a message if nothing catches it. Chapter 12 explains exceptions. The check costs a compare and a branch, which is negligible in most code and noticeable in the innermost loop of a kernel.</p>\n<p>So which should you use? The answer most codebases settle on is <code>[]</code> in hot loops whose bounds you control, and <code>.at()</code>, or an explicit check, wherever an index comes from outside: from a file, a network request, a user, or a model's output. A token id that arrives in a request and is used to index the embedding table should be checked, because a client can send any number. Inside the matmul loop, where the bounds come from the matrix's own shape, <code>[]</code> is fine.</p>\n<p><code>.at()</code> checks only the vector's own bounds, so the row-mean bug written with <code>m.at(r * cols + c)</code> would still compute wrong means for the first rows and throw only on the last row, exactly where ASan complained. Neither tool can know your matrix's shape. That is a reason to write tensor code with an index function that knows the shape, as chapter 4's <code>View</code> did, so that a check there can compare <code>c</code> against the number of columns.</p>\n<p>There is also a middle way. Both major standard libraries can check <code>[]</code> and many other preconditions themselves, with a compile-time switch:</p>\n<ul>\n<li>GCC's libstdc++ checks them when you define <code>_GLIBCXX_ASSERTIONS</code>, and aborts with a message when one fails.</li>\n<li>LLVM's libc++, which Apple's Clang uses, has <strong>hardening modes</strong> selected with <code>_LIBCPP_HARDENING_MODE</code>. The <code>fast</code> mode checks element access such as <code>v[i]</code>, and the libc++ documentation recommends it for production use because it is cheap. Older libc++ versions do not know this macro and silently ignore it, so check that it works, as the lab does.</li>\n</ul>\n<p>The lab's <code>LAB_HARDEN</code> option turns both on. These checks are cheap enough that some companies ship them in production, and C++26 adds standard library hardening to the language standard itself.</p>","headings":[{"id":"checked-and-unchecked-indexing","title":"Checked and unchecked indexing","depth":3}]},{"id":"lab-eight-bugs-and-the-tools-that-explain-them","title":"Lab: eight bugs and the tools that explain them","html":"<h3 id=\"lab-eight-bugs-and-the-tools-that-explain-them\">Lab: eight bugs and the tools that explain them</h3>\n<p>The lab is two programs: <a href=\"/labs/cpp/cpp-undefined-behavior/ub_zoo.cpp\"><code>ub_zoo.cpp</code></a>, which commits one bug per mode, and <a href=\"/labs/cpp/cpp-undefined-behavior/optimizer.cpp\"><code>optimizer.cpp</code></a>, whose answer changes with the optimization level. <a href=\"/labs/cpp/cpp-undefined-behavior/CMakeLists.txt\"><code>CMakeLists.txt</code></a> builds both and has two options, <code>LAB_SANITIZE</code> for ASan and UBSan and <code>LAB_HARDEN</code> for the standard library's checks. You need only a C++20 compiler and CMake. Build three versions side by side:</p>\n<pre class=\"shiki shiki-themes github-light github-dark\" style=\"background-color:#fff;--shiki-dark-bg:#24292e;color:#24292e;--shiki-dark:#e1e4e8\" tabindex=\"0\"><code><span class=\"line\"><span style=\"color:#6F42C1;--shiki-dark:#B392F0\">cmake</span><span style=\"color:#005CC5;--shiki-dark:#79B8FF\"> -S</span><span style=\"color:#032F62;--shiki-dark:#9ECBFF\"> .</span><span style=\"color:#005CC5;--shiki-dark:#79B8FF\"> -B</span><span style=\"color:#032F62;--shiki-dark:#9ECBFF\"> build</span><span style=\"color:#005CC5;--shiki-dark:#79B8FF\">     -DCMAKE_BUILD_TYPE=Debug</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">                    &#x26;&#x26; </span><span style=\"color:#6F42C1;--shiki-dark:#B392F0\">cmake</span><span style=\"color:#005CC5;--shiki-dark:#79B8FF\"> --build</span><span style=\"color:#032F62;--shiki-dark:#9ECBFF\"> build</span></span>\n<span class=\"line\"><span style=\"color:#6F42C1;--shiki-dark:#B392F0\">cmake</span><span style=\"color:#005CC5;--shiki-dark:#79B8FF\"> -S</span><span style=\"color:#032F62;--shiki-dark:#9ECBFF\"> .</span><span style=\"color:#005CC5;--shiki-dark:#79B8FF\"> -B</span><span style=\"color:#032F62;--shiki-dark:#9ECBFF\"> build-san</span><span style=\"color:#005CC5;--shiki-dark:#79B8FF\"> -DCMAKE_BUILD_TYPE=Debug</span><span style=\"color:#005CC5;--shiki-dark:#79B8FF\"> -DLAB_SANITIZE=ON</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">  &#x26;&#x26; </span><span style=\"color:#6F42C1;--shiki-dark:#B392F0\">cmake</span><span style=\"color:#005CC5;--shiki-dark:#79B8FF\"> --build</span><span style=\"color:#032F62;--shiki-dark:#9ECBFF\"> build-san</span></span>\n<span class=\"line\"><span style=\"color:#6F42C1;--shiki-dark:#B392F0\">cmake</span><span style=\"color:#005CC5;--shiki-dark:#79B8FF\"> -S</span><span style=\"color:#032F62;--shiki-dark:#9ECBFF\"> .</span><span style=\"color:#005CC5;--shiki-dark:#79B8FF\"> -B</span><span style=\"color:#032F62;--shiki-dark:#9ECBFF\"> build-rel</span><span style=\"color:#005CC5;--shiki-dark:#79B8FF\"> -DCMAKE_BUILD_TYPE=Release</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">                  &#x26;&#x26; </span><span style=\"color:#6F42C1;--shiki-dark:#B392F0\">cmake</span><span style=\"color:#005CC5;--shiki-dark:#79B8FF\"> --build</span><span style=\"color:#032F62;--shiki-dark:#9ECBFF\"> build-rel</span></span></code></pre>\n<h4 id=\"part-1-read-the-warnings\">Part 1: read the warnings</h4>\n<p>Before running anything, read the warnings from the first build. Clang should warn that the <code>string_view</code> in <code>dangling_view</code> refers to an object destroyed at the end of the statement, and GCC or Clang will probably warn that <code>sum</code> in <code>uninit</code> is used uninitialized. Note which bugs produced no warning at all. Those are the ones that depend on values the compiler cannot see.</p>\n<h4 id=\"part-2-run-the-zoo-without-help\">Part 2: run the zoo without help</h4>\n<p>Run every mode with the plain Debug build and write down what happens:</p>\n<pre class=\"shiki shiki-themes github-light github-dark\" style=\"background-color:#fff;--shiki-dark-bg:#24292e;color:#24292e;--shiki-dark:#e1e4e8\" tabindex=\"0\"><code><span class=\"line\"><span style=\"color:#6F42C1;--shiki-dark:#B392F0\">./build/ub_zoo</span><span style=\"color:#032F62;--shiki-dark:#9ECBFF\"> row-means</span></span>\n<span class=\"line\"><span style=\"color:#6F42C1;--shiki-dark:#B392F0\">./build/ub_zoo</span><span style=\"color:#032F62;--shiki-dark:#9ECBFF\"> stack-oob</span></span>\n<span class=\"line\"><span style=\"color:#6F42C1;--shiki-dark:#B392F0\">./build/ub_zoo</span><span style=\"color:#032F62;--shiki-dark:#9ECBFF\"> dangling-view</span></span>\n<span class=\"line\"><span style=\"color:#6F42C1;--shiki-dark:#B392F0\">./build/ub_zoo</span><span style=\"color:#032F62;--shiki-dark:#9ECBFF\"> double-free</span></span>\n<span class=\"line\"><span style=\"color:#6F42C1;--shiki-dark:#B392F0\">./build/ub_zoo</span><span style=\"color:#032F62;--shiki-dark:#9ECBFF\"> uninit</span></span>\n<span class=\"line\"><span style=\"color:#6F42C1;--shiki-dark:#B392F0\">./build/ub_zoo</span><span style=\"color:#032F62;--shiki-dark:#9ECBFF\"> overflow</span><span style=\"color:#005CC5;--shiki-dark:#79B8FF\"> 8</span></span>\n<span class=\"line\"><span style=\"color:#6F42C1;--shiki-dark:#B392F0\">./build/ub_zoo</span><span style=\"color:#032F62;--shiki-dark:#9ECBFF\"> overflow</span><span style=\"color:#005CC5;--shiki-dark:#79B8FF\"> 16</span></span>\n<span class=\"line\"><span style=\"color:#6F42C1;--shiki-dark:#B392F0\">./build/ub_zoo</span><span style=\"color:#032F62;--shiki-dark:#9ECBFF\"> misaligned</span></span>\n<span class=\"line\"><span style=\"color:#6F42C1;--shiki-dark:#B392F0\">./build/ub_zoo</span><span style=\"color:#032F62;--shiki-dark:#9ECBFF\"> at</span></span></code></pre>\n<p>Expect most of them to finish as if nothing were wrong. <code>row-means</code> prints its means next to the correct ones, and the first two rows are too large, because each sum includes the first element of the next row: row 0 should show 3.75 where 2.50 is correct, and row 1 should show 8.75 where 6.50 is correct. The last row's mean depends on whatever lies past the buffer. <code>overflow 16</code> prints a negative <code>int</code> offset next to the correct 64-bit one, while <code>overflow 8</code> is correct. <code>double-free</code> is the mode most likely to be stopped by the allocator itself. <code>at</code> throws, but only when the index leaves the vector, on the last row, after printing two wrong means.</p>\n<h4 id=\"part-3-run-the-zoo-with-the-sanitizers\">Part 3: run the zoo with the sanitizers</h4>\n<p>Now run the same modes from <code>build-san</code>. ASan stops the program at the first memory error, so run each mode separately. For each one, find the headline, the first frame in <code>ub_zoo.cpp</code>, and, for the use after free, the frames that show where the memory was freed and allocated. Then fill in a table with three columns, one for each kind of report: ASan, UBSan, or nothing. You should find that ASan reports <code>row-means</code> (on the last row only), <code>stack-oob</code>, <code>dangling-view</code> and <code>double-free</code>, that UBSan reports <code>overflow 16</code> and <code>misaligned</code>, and that <code>stack-oob</code> may be reported by UBSan as well, because the array's size is known. <code>uninit</code> should pass both sanitizers silently, since neither checks for it. On Linux with Clang, you can rebuild with <code>-fsanitize=memory</code> instead and watch MemorySanitizer report it.</p>\n<h4 id=\"part-4-debug-versus-release\">Part 4: Debug versus Release</h4>\n<p>Run <code>optimizer</code> from both the Debug and the Release build:</p>\n<pre class=\"shiki shiki-themes github-light github-dark\" style=\"background-color:#fff;--shiki-dark-bg:#24292e;color:#24292e;--shiki-dark:#e1e4e8\" tabindex=\"0\"><code><span class=\"line\"><span style=\"color:#6F42C1;--shiki-dark:#B392F0\">./build/optimizer</span><span style=\"color:#005CC5;--shiki-dark:#79B8FF\"> 2147483647</span></span>\n<span class=\"line\"><span style=\"color:#6F42C1;--shiki-dark:#B392F0\">./build-rel/optimizer</span><span style=\"color:#005CC5;--shiki-dark:#79B8FF\"> 2147483647</span></span>\n<span class=\"line\"><span style=\"color:#6F42C1;--shiki-dark:#B392F0\">./build/optimizer</span><span style=\"color:#005CC5;--shiki-dark:#79B8FF\"> 5</span></span>\n<span class=\"line\"><span style=\"color:#6F42C1;--shiki-dark:#B392F0\">./build-rel/optimizer</span><span style=\"color:#005CC5;--shiki-dark:#79B8FF\"> 5</span></span></code></pre>\n<p>With 5, both builds should agree. With the largest <code>int</code>, <code>next_is_bigger</code> should print false in the Debug build, where the addition wraps on the hardware, and true in the Release build, where the optimizer has replaced the function with \"return true\". GCC sometimes simplifies the comparison even without optimization, so with GCC both builds may print true. <code>next_is_bigger_safe</code> asks the same question without overflowing, and it is false in every build. <code>doublings</code> counts how many times a number can be doubled before it overflows. The loop's only real exit is the overflow, and the optimizer is allowed to assume that a positive number doubled stays positive, so a Release build may report a different count, possibly the safety limit of 100, than the Debug build does. Whatever you see, the source code has not changed, and only the optimizer's assumptions differ. Then run <code>./build-san/optimizer 2147483647</code> and read UBSan's reports, which name the overflowing line in each function.</p>\n<h4 id=\"part-5-the-standard-librarys-own-checks\">Part 5: the standard library's own checks</h4>\n<p>Build a fourth version with the standard library's checks on and no sanitizers, and run the row-mean bug through it, after changing the loop in <code>row_means</code> to index the vector with <code>m[r * kCols + c]</code> instead of the raw pointer:</p>\n<pre class=\"shiki shiki-themes github-light github-dark\" style=\"background-color:#fff;--shiki-dark-bg:#24292e;color:#24292e;--shiki-dark:#e1e4e8\" tabindex=\"0\"><code><span class=\"line\"><span style=\"color:#6F42C1;--shiki-dark:#B392F0\">cmake</span><span style=\"color:#005CC5;--shiki-dark:#79B8FF\"> -S</span><span style=\"color:#032F62;--shiki-dark:#9ECBFF\"> .</span><span style=\"color:#005CC5;--shiki-dark:#79B8FF\"> -B</span><span style=\"color:#032F62;--shiki-dark:#9ECBFF\"> build-harden</span><span style=\"color:#005CC5;--shiki-dark:#79B8FF\"> -DCMAKE_BUILD_TYPE=Debug</span><span style=\"color:#005CC5;--shiki-dark:#79B8FF\"> -DLAB_HARDEN=ON</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\"> &#x26;&#x26; </span><span style=\"color:#6F42C1;--shiki-dark:#B392F0\">cmake</span><span style=\"color:#005CC5;--shiki-dark:#79B8FF\"> --build</span><span style=\"color:#032F62;--shiki-dark:#9ECBFF\"> build-harden</span></span>\n<span class=\"line\"><span style=\"color:#6F42C1;--shiki-dark:#B392F0\">./build-harden/ub_zoo</span><span style=\"color:#032F62;--shiki-dark:#9ECBFF\"> row-means</span></span></code></pre>\n<p>The program should now stop on the last row with a message from the library, or with a trap in libc++'s fast mode, which prints little or nothing. If it still runs to the end, your standard library does not support the checks this way, which is worth noting.</p>\n<h4 id=\"stretch-goals\">Stretch goals</h4>\n<ul>\n<li>Fix every bug in <code>ub_zoo.cpp</code>, then run all modes under the sanitizer build until none reports anything. Remember that a clean run proves only that these inputs are clean.</li>\n<li>Add <code>-fno-sanitize-recover=all</code> to the sanitizer flags in <code>CMakeLists.txt</code> and see how UBSan's behavior changes for <code>overflow 16</code>.</li>\n<li>Run <code>./build-san/ub_zoo misaligned 0</code> and <code>misaligned 4</code>. UBSan stays quiet because those addresses are aligned. Explain why the cast is still undefined, according to the strict aliasing rule, and why the <code>memcpy</code> version is correct for every offset.</li>\n<li>Add <code>-fwrapv</code> to the Release build of <code>optimizer</code> and run it again. The flag defines signed overflow to wrap, so the optimizer can no longer assume it away. Some projects build with it, or with the related <code>-fno-strict-overflow</code>, which the Linux kernel uses.</li>\n</ul>","headings":[{"id":"lab-eight-bugs-and-the-tools-that-explain-them","title":"Lab: eight bugs and the tools that explain them","depth":3},{"id":"part-1-read-the-warnings","title":"Part 1: read the warnings","depth":4},{"id":"part-2-run-the-zoo-without-help","title":"Part 2: run the zoo without help","depth":4},{"id":"part-3-run-the-zoo-with-the-sanitizers","title":"Part 3: run the zoo with the sanitizers","depth":4},{"id":"part-4-debug-versus-release","title":"Part 4: Debug versus Release","depth":4},{"id":"part-5-the-standard-librarys-own-checks","title":"Part 5: the standard library's own checks","depth":4},{"id":"stretch-goals","title":"Stretch goals","depth":4}]},{"id":"check-yourself","title":"Check yourself","html":"<h3 id=\"check-yourself\">Check yourself</h3>\n<details>\n<summary>What is the difference between implementation-defined, unspecified and undefined behavior?</summary>\n<p>Implementation-defined behavior has one result chosen and documented by each compiler, such as <code>sizeof(long)</code>. Unspecified behavior has one of several allowed results, and the compiler need not say which, such as the order in which a function's arguments are evaluated. Undefined behavior has no requirements at all: the standard says nothing about what the whole run of the program does, and the compiler may assume it never happens.</p>\n</details>\n<details>\n<summary>A program gives correct results in Debug and wrong results in Release. Why is \"the optimizer has a bug\" usually the wrong first guess?</summary>\n<p>The optimizer assumes the program has no undefined behavior and transforms the code on that assumption. If the program overflows a signed integer, reads out of bounds or reads uninitialized memory, the optimized code is allowed to behave differently from the unoptimized code, and both are correct translations of a program that has no defined meaning. Compiler bugs exist, but undefined behavior in the program is far more common, and running the Debug build under ASan and UBSan is the quickest way to find it.</p>\n</details>\n<details>\n<summary>A kernel computes row means with c &#x3C;= cols. ASan reports one heap-buffer-overflow, on the last row. Are the other rows' means correct?</summary>\n<p>No. Every row reads one element too many, but for every row except the last, the extra element is the first element of the next row, which is still inside the buffer, so ASan cannot see anything wrong. ASan checks the bounds of allocations, not the shape of your data. Fixing the loop bound fixes every row.</p>\n</details>\n<details>\n<summary>Why does int offset = batch * seq * vocab; pass every test with batch 8 and fail in production with batch 16, for seq 2,048 and vocab 128,256?</summary>\n<p>With batch 8 the product is 2,101,346,304, just under the largest 32-bit <code>int</code>, 2,147,483,647. With batch 16 it is 4,202,692,608, which overflows, and signed overflow is undefined. In practice the value usually wraps to a negative number and the kernel indexes far outside the tensor. Computing the offset in <code>std::int64_t</code>, casting before the multiplication, fixes it. UBSan reports it, but only for a run that actually uses batch 16.</p>\n</details>\n<details>\n<summary>Which tool would you use to find a read of an uninitialized variable, and why not AddressSanitizer?</summary>\n<p>MemorySanitizer, on Linux with Clang, or Valgrind's Memcheck. AddressSanitizer tracks which memory is valid to access, not whether it has been written, so reading an uninitialized but validly allocated variable looks fine to it. A compiler warning such as <code>-Wuninitialized</code> catches the simplest cases, and <code>-ftrivial-auto-var-init=zero</code> removes the randomness for local variables.</p>\n</details>\n<details>\n<summary>When should you write v.at(i) instead of v[i]?</summary>\n<p>When the index comes from outside the code's control, such as a file, a request, a user or a model's output: for example a token id from a client used to index an embedding table. There, a check that throws is far better than undefined behavior. In inner loops whose bounds come from the data's own shape, <code>v[i]</code> avoids the cost of the check, and a hardened standard library build or the sanitizers can check those accesses during testing.</p>\n</details>\n<h3 id=\"further-reading\">Further reading</h3>\n<ul>\n<li><a href=\"https://en.cppreference.com/w/cpp/language/ub\" target=\"_blank\" rel=\"noopener noreferrer\">cppreference: undefined behavior</a> defines the four kinds of behavior and shows examples of optimizations that rely on undefined behavior.</li>\n<li><a href=\"https://blog.llvm.org/2011/05/what-every-c-programmer-should-know.html\" target=\"_blank\" rel=\"noopener noreferrer\">What Every C Programmer Should Know About Undefined Behavior</a>, a three-part series by Chris Lattner on the LLVM blog, explains why compilers exploit undefined behavior and what that buys in speed.</li>\n<li><a href=\"https://blog.regehr.org/archives/213\" target=\"_blank\" rel=\"noopener noreferrer\">A Guide to Undefined Behavior in C and C++</a>, by John Regehr, is a careful explanation from a researcher who has spent years finding these bugs in real compilers and programs.</li>\n<li><a href=\"https://lwn.net/Articles/342330/\" target=\"_blank\" rel=\"noopener noreferrer\">Fun with NULL pointers</a>, on LWN, tells the story of the Linux <code>tun</code> driver bug in detail, and <a href=\"https://gcc.gnu.org/PR53073\" target=\"_blank\" rel=\"noopener noreferrer\">GCC bug 53073</a> is the discussion of the <code>h264ref</code> infinite loop.</li>\n<li>The Clang documentation for <a href=\"https://clang.llvm.org/docs/AddressSanitizer.html\" target=\"_blank\" rel=\"noopener noreferrer\">AddressSanitizer</a>, <a href=\"https://clang.llvm.org/docs/UndefinedBehaviorSanitizer.html\" target=\"_blank\" rel=\"noopener noreferrer\">UndefinedBehaviorSanitizer</a>, <a href=\"https://clang.llvm.org/docs/MemorySanitizer.html\" target=\"_blank\" rel=\"noopener noreferrer\">MemorySanitizer</a> and <a href=\"https://clang.llvm.org/docs/ThreadSanitizer.html\" target=\"_blank\" rel=\"noopener noreferrer\">ThreadSanitizer</a> lists what each tool checks, its cost and the platforms it supports.</li>\n<li><a href=\"https://libcxx.llvm.org/Hardening.html\" target=\"_blank\" rel=\"noopener noreferrer\">libc++ hardening modes</a> documents the checks in LLVM's standard library and how to enable them.</li>\n<li><a href=\"https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2024/p2795r5.html\" target=\"_blank\" rel=\"noopener noreferrer\">P2795R5, Erroneous behaviour for uninitialized reads</a> is the C++26 proposal that changes the rules for uninitialized local variables.</li>\n<li><a href=\"https://www.chromium.org/Home/chromium-security/memory-safety/\" target=\"_blank\" rel=\"noopener noreferrer\">Chromium's memory safety page</a> gives the figure that about 70% of the project's serious security bugs are memory safety problems.</li>\n<li><a href=\"https://docs.nvidia.com/compute-sanitizer/ComputeSanitizer/index.html\" target=\"_blank\" rel=\"noopener noreferrer\">NVIDIA Compute Sanitizer</a> is the GPU counterpart of these tools, used in Part VI.</li>\n</ul>","headings":[{"id":"check-yourself","title":"Check yourself","depth":3},{"id":"further-reading","title":"Further reading","depth":3}]}]}