{"id":"ood-lru-cache","title":"Design an LRU cache","sections":[{"id":"overview","title":"Overview","html":"<p>A <strong>cache</strong> keeps copies of recent answers in fast memory so that repeated requests don't have to redo slow work. A search engine is a good example. Computing the results for \"weather paris\" means querying an index spread over many machines, but thousands of people type that same query every hour. If the search service keeps the last few thousand queries and their results in memory, most of those people get an answer straight from memory.</p>\n<p>Memory is limited, so the cache has a fixed <strong>capacity</strong>. When it's full and a new query arrives, something has to go. An <strong>LRU (least recently used) cache</strong> throws out the entry that has gone longest without being read or written. The bet is that a query nobody has asked for in an hour is less likely to be asked next than one asked a minute ago. Python's <code>functools.lru_cache</code> decorator works this way, and Redis offers an <code>allkeys-lru</code> eviction policy, which its documentation calls \"a good default option if you have no reason to prefer any others.\"</p>\n<p>This chapter is an <strong>object-oriented design</strong> question. In this kind of interview question you're asked to model something in code: you clarify the scope with the interviewer, name the classes and say what each one is responsible for, write working code, and then discuss how the design would change under new requirements. For an LRU cache, the interviewer is mostly checking one thing: can you make every operation run in constant time?</p>","headings":[]},{"id":"constraints-and-assumptions","title":"Constraints and assumptions","html":"<h3 id=\"constraints-and-assumptions\">Constraints and assumptions</h3>\n<p>Ask these questions first. Each answer rules out work you'd otherwise have to do.</p>\n<ul>\n<li><strong>What are we caching?</strong> The results of web queries. The key is the query string and the value is its results. <em>Why it matters:</em> it tells you what the keys look like (strings, which Python can hash) and suggests the access pattern: a few popular queries asked over and over, which is exactly where LRU does well.</li>\n<li><strong>Can we assume inputs are valid, or do we have to validate them?</strong> Assume they're valid. <em>Why it matters:</em> no checks for empty or malformed queries, so the code can focus on the data structure.</li>\n<li><strong>Can we assume this fits in memory?</strong> Yes. <em>Why it matters:</em> the whole cache lives in one process. Splitting it across machines is an extension.</li>\n</ul>\n<p>A few more answers we assume, because the code depends on them. State them out loud in an interview:</p>\n<ul>\n<li><strong>How is capacity measured?</strong> In entries: the cache holds at most <code>capacity</code> queries, whatever the size of their results. Measuring in bytes is a follow-up.</li>\n<li><strong>What does <code>get</code> return on a miss?</strong> <code>None</code>. The caller then computes the results and calls <code>set</code>.</li>\n<li><strong>Does a <code>get</code> count as a use?</strong> Yes. A hit moves the entry to the most recently used position. That's what makes it LRU, as opposed to evicting the oldest insertion, which is called first in, first out (FIFO).</li>\n<li><strong>What does <code>set</code> do for a query that's already cached?</strong> It replaces the results and marks the entry as most recently used. It never evicts anything, since the number of entries doesn't change.</li>\n<li><strong>What if <code>set</code> adds a new query when the cache is full?</strong> It first removes the least recently used entry, then adds the new one.</li>\n<li><strong>Do we need thread safety or expiry times?</strong> Not for now. Both come up as follow-ups.</li>\n</ul>","headings":[{"id":"constraints-and-assumptions","title":"Constraints and assumptions","depth":3}]},{"id":"class-design","title":"Class design","html":"<h3 id=\"class-design\">Class design</h3>\n<p>The cache needs two things at once: find any query's entry in O(1), and know which entry is least recently used in O(1). No single built-in structure does both, so we combine a hash map (a Python <code>dict</code>) with a doubly linked list. A <strong>doubly linked list</strong> is a chain of nodes where each node points to both the node before it and the node after it.</p>\n<figure class=\"diagram\" data-text=\"LRUCache get, set -> dict query → Node (looks up); LRUCache get, set -> RecencyList most → least recent (reorders, evicts); RecencyList most → least recent -> Node query, results, prev, next (links); dict query → Node -> Node query, results, prev, next (points to)\"><div class=\"diagram-scroll\"><svg class=\"dg\" viewBox=\"0 0 480 268\" width=\"480\" role=\"img\" aria-label=\"LRUCache finds nodes through its dict and orders them with its RecencyList\"><defs><marker id=\"dg-arrow\" viewBox=\"0 0 10 10\" refX=\"9\" refY=\"5\" markerWidth=\"7\" markerHeight=\"7\" orient=\"auto-start-reverse\"><path class=\"dg-arrowhead\" d=\"M0,1 L9,5 L0,9 z\"></path></marker></defs><line class=\"dg-edge\" x1=\"151\" y1=\"42\" x2=\"327\" y2=\"42\" marker-end=\"url(#dg-arrow)\"></line><line class=\"dg-edge\" x1=\"80\" y1=\"71\" x2=\"80\" y2=\"183\" marker-end=\"url(#dg-arrow)\"></line><line class=\"dg-edge\" x1=\"151\" y1=\"222\" x2=\"327\" y2=\"222\" marker-end=\"url(#dg-arrow)\"></line><line class=\"dg-edge\" x1=\"400\" y1=\"76\" x2=\"400\" y2=\"183\" marker-end=\"url(#dg-arrow)\"></line><rect class=\"dg-node dg-box\" x=\"12\" y=\"16\" width=\"136\" height=\"52\" rx=\"8\"></rect><text class=\"dg-label\" text-anchor=\"middle\"><tspan x=\"80\" y=\"38.5\">LRUCache</tspan><tspan x=\"80\" y=\"54.5\">get, set</tspan></text><path class=\"dg-node dg-store\" d=\"M332,18 a68,7 0 0,1 136,0 v48 a68,7 0 0,1 -136,0 z\"></path><path class=\"dg-line\" d=\"M332,18 a68,7 0 0,0 136,0\"></path><text class=\"dg-label\" text-anchor=\"middle\"><tspan x=\"400\" y=\"41.5\">dict</tspan><tspan x=\"400\" y=\"57.5\">query → Node</tspan></text><rect class=\"dg-node dg-box\" x=\"12\" y=\"188\" width=\"136\" height=\"68\" rx=\"8\"></rect><text class=\"dg-label\" text-anchor=\"middle\"><tspan x=\"80\" y=\"210.5\">RecencyList</tspan><tspan x=\"80\" y=\"226.5\">most → least</tspan><tspan x=\"80\" y=\"242.5\">recent</tspan></text><rect class=\"dg-node dg-box\" x=\"332\" y=\"188\" width=\"136\" height=\"68\" rx=\"8\"></rect><text class=\"dg-label\" text-anchor=\"middle\"><tspan x=\"400\" y=\"210.5\">Node</tspan><tspan x=\"400\" y=\"226.5\">query, results,</tspan><tspan x=\"400\" y=\"242.5\">prev, next</tspan></text><rect class=\"dg-edge-bg\" x=\"209.2\" y=\"33\" width=\"59.6\" height=\"18\" rx=\"4\"></rect><text class=\"dg-edge-label\" x=\"239\" y=\"46\" text-anchor=\"middle\">looks up</text><rect class=\"dg-edge-bg\" x=\"25.4\" y=\"118\" width=\"109.2\" height=\"18\" rx=\"4\"></rect><text class=\"dg-edge-label\" x=\"80\" y=\"131\" text-anchor=\"middle\">reorders, evicts</text><rect class=\"dg-edge-bg\" x=\"218.5\" y=\"213\" width=\"41\" height=\"18\" rx=\"4\"></rect><text class=\"dg-edge-label\" x=\"239\" y=\"226\" text-anchor=\"middle\">links</text><rect class=\"dg-edge-bg\" x=\"367.1\" y=\"120.5\" width=\"65.8\" height=\"18\" rx=\"4\"></rect><text class=\"dg-edge-label\" x=\"400\" y=\"133.5\" text-anchor=\"middle\">points to</text></svg></div><figcaption>LRUCache finds nodes through its dict and orders them with its RecencyList</figcaption></figure>\n<ul>\n<li><strong><code>Node</code></strong> holds one cache entry: the query, its results, and links to its neighbors in recency order.</li>\n<li><strong><code>RecencyList</code></strong> keeps nodes ordered from most to least recently used. It can add a node at the front, move a node to the front, and remove the last node, each in O(1). It knows nothing about queries or capacity.</li>\n<li><strong><code>LRUCache</code></strong> is the public interface. It enforces the capacity, decides when an entry counts as used, and keeps the dict and the list in step.</li>\n</ul>\n<p>Keeping the list separate from the cache gives each class one job. You could test <code>RecencyList</code> on its own, or reuse it for a different eviction policy.</p>","headings":[{"id":"class-design","title":"Class design","depth":3}]},{"id":"solution","title":"Solution","html":"<h3 id=\"solution\">Solution</h3>\n<pre class=\"shiki shiki-themes github-light github-dark\" style=\"background-color:#fff;--shiki-dark-bg:#24292e;color:#24292e;--shiki-dark:#e1e4e8\" tabindex=\"0\"><code><span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F97583\">from</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\"> typing </span><span style=\"color:#D73A49;--shiki-dark:#F97583\">import</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\"> Any, Optional</span></span>\n<span class=\"line\"></span>\n<span class=\"line\"></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F97583\">class</span><span style=\"color:#6F42C1;--shiki-dark:#B392F0\"> Node</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">:</span></span>\n<span class=\"line\"><span style=\"color:#032F62;--shiki-dark:#9ECBFF\">    \"\"\"One cached query and its results, linked into the recency list.\"\"\"</span></span>\n<span class=\"line\"></span>\n<span class=\"line\"><span style=\"color:#005CC5;--shiki-dark:#79B8FF\">    __slots__</span><span style=\"color:#D73A49;--shiki-dark:#F97583\"> =</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\"> (</span><span style=\"color:#032F62;--shiki-dark:#9ECBFF\">\"query\"</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">, </span><span style=\"color:#032F62;--shiki-dark:#9ECBFF\">\"results\"</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">, </span><span style=\"color:#032F62;--shiki-dark:#9ECBFF\">\"prev\"</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">, </span><span style=\"color:#032F62;--shiki-dark:#9ECBFF\">\"next\"</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">)</span></span>\n<span class=\"line\"></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F97583\">    def</span><span style=\"color:#005CC5;--shiki-dark:#79B8FF\"> __init__</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">(self, query: Any </span><span style=\"color:#D73A49;--shiki-dark:#F97583\">=</span><span style=\"color:#005CC5;--shiki-dark:#79B8FF\"> None</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">, results: Any </span><span style=\"color:#D73A49;--shiki-dark:#F97583\">=</span><span style=\"color:#005CC5;--shiki-dark:#79B8FF\"> None</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">) -> </span><span style=\"color:#005CC5;--shiki-dark:#79B8FF\">None</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">:</span></span>\n<span class=\"line\"><span style=\"color:#005CC5;--shiki-dark:#79B8FF\">        self</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">.query </span><span style=\"color:#D73A49;--shiki-dark:#F97583\">=</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\"> query        </span><span style=\"color:#6A737D;--shiki-dark:#6A737D\"># kept so eviction knows which key to delete</span></span>\n<span class=\"line\"><span style=\"color:#005CC5;--shiki-dark:#79B8FF\">        self</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">.results </span><span style=\"color:#D73A49;--shiki-dark:#F97583\">=</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\"> results</span></span>\n<span class=\"line\"><span style=\"color:#005CC5;--shiki-dark:#79B8FF\">        self</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">.prev: Optional[</span><span style=\"color:#032F62;--shiki-dark:#9ECBFF\">\"Node\"</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">] </span><span style=\"color:#D73A49;--shiki-dark:#F97583\">=</span><span style=\"color:#005CC5;--shiki-dark:#79B8FF\"> None</span></span>\n<span class=\"line\"><span style=\"color:#005CC5;--shiki-dark:#79B8FF\">        self</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">.next: Optional[</span><span style=\"color:#032F62;--shiki-dark:#9ECBFF\">\"Node\"</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">] </span><span style=\"color:#D73A49;--shiki-dark:#F97583\">=</span><span style=\"color:#005CC5;--shiki-dark:#79B8FF\"> None</span></span>\n<span class=\"line\"></span>\n<span class=\"line\"></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F97583\">class</span><span style=\"color:#6F42C1;--shiki-dark:#B392F0\"> RecencyList</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">:</span></span>\n<span class=\"line\"><span style=\"color:#032F62;--shiki-dark:#9ECBFF\">    \"\"\"A doubly linked list ordered from most to least recently used.\"\"\"</span></span>\n<span class=\"line\"></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F97583\">    def</span><span style=\"color:#005CC5;--shiki-dark:#79B8FF\"> __init__</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">(self) -> </span><span style=\"color:#005CC5;--shiki-dark:#79B8FF\">None</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">:</span></span>\n<span class=\"line\"><span style=\"color:#005CC5;--shiki-dark:#79B8FF\">        self</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">._head </span><span style=\"color:#D73A49;--shiki-dark:#F97583\">=</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\"> Node()       </span><span style=\"color:#6A737D;--shiki-dark:#6A737D\"># sentinel before the most recent node</span></span>\n<span class=\"line\"><span style=\"color:#005CC5;--shiki-dark:#79B8FF\">        self</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">._tail </span><span style=\"color:#D73A49;--shiki-dark:#F97583\">=</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\"> Node()       </span><span style=\"color:#6A737D;--shiki-dark:#6A737D\"># sentinel after the least recent node</span></span>\n<span class=\"line\"><span style=\"color:#005CC5;--shiki-dark:#79B8FF\">        self</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">._head.next </span><span style=\"color:#D73A49;--shiki-dark:#F97583\">=</span><span style=\"color:#005CC5;--shiki-dark:#79B8FF\"> self</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">._tail</span></span>\n<span class=\"line\"><span style=\"color:#005CC5;--shiki-dark:#79B8FF\">        self</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">._tail.prev </span><span style=\"color:#D73A49;--shiki-dark:#F97583\">=</span><span style=\"color:#005CC5;--shiki-dark:#79B8FF\"> self</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">._head</span></span>\n<span class=\"line\"></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F97583\">    def</span><span style=\"color:#6F42C1;--shiki-dark:#B392F0\"> add_to_front</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">(self, node: Node) -> </span><span style=\"color:#005CC5;--shiki-dark:#79B8FF\">None</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">:</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">        node.prev </span><span style=\"color:#D73A49;--shiki-dark:#F97583\">=</span><span style=\"color:#005CC5;--shiki-dark:#79B8FF\"> self</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">._head</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">        node.next </span><span style=\"color:#D73A49;--shiki-dark:#F97583\">=</span><span style=\"color:#005CC5;--shiki-dark:#79B8FF\"> self</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">._head.next</span></span>\n<span class=\"line\"><span style=\"color:#005CC5;--shiki-dark:#79B8FF\">        self</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">._head.next.prev </span><span style=\"color:#D73A49;--shiki-dark:#F97583\">=</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\"> node</span></span>\n<span class=\"line\"><span style=\"color:#005CC5;--shiki-dark:#79B8FF\">        self</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">._head.next </span><span style=\"color:#D73A49;--shiki-dark:#F97583\">=</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\"> node</span></span>\n<span class=\"line\"></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F97583\">    def</span><span style=\"color:#6F42C1;--shiki-dark:#B392F0\"> unlink</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">(self, node: Node) -> </span><span style=\"color:#005CC5;--shiki-dark:#79B8FF\">None</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">:</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">        node.prev.next </span><span style=\"color:#D73A49;--shiki-dark:#F97583\">=</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\"> node.next</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">        node.next.prev </span><span style=\"color:#D73A49;--shiki-dark:#F97583\">=</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\"> node.prev</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">        node.prev </span><span style=\"color:#D73A49;--shiki-dark:#F97583\">=</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\"> node.next </span><span style=\"color:#D73A49;--shiki-dark:#F97583\">=</span><span style=\"color:#005CC5;--shiki-dark:#79B8FF\"> None</span></span>\n<span class=\"line\"></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F97583\">    def</span><span style=\"color:#6F42C1;--shiki-dark:#B392F0\"> move_to_front</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">(self, node: Node) -> </span><span style=\"color:#005CC5;--shiki-dark:#79B8FF\">None</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">:</span></span>\n<span class=\"line\"><span style=\"color:#005CC5;--shiki-dark:#79B8FF\">        self</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">.unlink(node)</span></span>\n<span class=\"line\"><span style=\"color:#005CC5;--shiki-dark:#79B8FF\">        self</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">.add_to_front(node)</span></span>\n<span class=\"line\"></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F97583\">    def</span><span style=\"color:#6F42C1;--shiki-dark:#B392F0\"> remove_last</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">(self) -> Node:</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">        node </span><span style=\"color:#D73A49;--shiki-dark:#F97583\">=</span><span style=\"color:#005CC5;--shiki-dark:#79B8FF\"> self</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">._tail.prev</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F97583\">        if</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\"> node </span><span style=\"color:#D73A49;--shiki-dark:#F97583\">is</span><span style=\"color:#005CC5;--shiki-dark:#79B8FF\"> self</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">._head:</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F97583\">            raise</span><span style=\"color:#005CC5;--shiki-dark:#79B8FF\"> IndexError</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">(</span><span style=\"color:#032F62;--shiki-dark:#9ECBFF\">\"remove_last from an empty list\"</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">)</span></span>\n<span class=\"line\"><span style=\"color:#005CC5;--shiki-dark:#79B8FF\">        self</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">.unlink(node)</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F97583\">        return</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\"> node</span></span>\n<span class=\"line\"></span>\n<span class=\"line\"></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F97583\">class</span><span style=\"color:#6F42C1;--shiki-dark:#B392F0\"> LRUCache</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">:</span></span>\n<span class=\"line\"><span style=\"color:#032F62;--shiki-dark:#9ECBFF\">    \"\"\"Caches query results and evicts the least recently used query.\"\"\"</span></span>\n<span class=\"line\"></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F97583\">    def</span><span style=\"color:#005CC5;--shiki-dark:#79B8FF\"> __init__</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">(self, capacity: </span><span style=\"color:#005CC5;--shiki-dark:#79B8FF\">int</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">) -> </span><span style=\"color:#005CC5;--shiki-dark:#79B8FF\">None</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">:</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F97583\">        if</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\"> capacity </span><span style=\"color:#D73A49;--shiki-dark:#F97583\">&#x3C;</span><span style=\"color:#005CC5;--shiki-dark:#79B8FF\"> 1</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">:</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F97583\">            raise</span><span style=\"color:#005CC5;--shiki-dark:#79B8FF\"> ValueError</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">(</span><span style=\"color:#032F62;--shiki-dark:#9ECBFF\">\"capacity must be at least 1\"</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">)</span></span>\n<span class=\"line\"><span style=\"color:#005CC5;--shiki-dark:#79B8FF\">        self</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">.capacity </span><span style=\"color:#D73A49;--shiki-dark:#F97583\">=</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\"> capacity</span></span>\n<span class=\"line\"><span style=\"color:#005CC5;--shiki-dark:#79B8FF\">        self</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">._nodes: dict[Any, Node] </span><span style=\"color:#D73A49;--shiki-dark:#F97583\">=</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\"> {}   </span><span style=\"color:#6A737D;--shiki-dark:#6A737D\"># query -> its node in the list</span></span>\n<span class=\"line\"><span style=\"color:#005CC5;--shiki-dark:#79B8FF\">        self</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">._recency </span><span style=\"color:#D73A49;--shiki-dark:#F97583\">=</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\"> RecencyList()</span></span>\n<span class=\"line\"></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F97583\">    def</span><span style=\"color:#6F42C1;--shiki-dark:#B392F0\"> get</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">(self, query: Any) -> Any:</span></span>\n<span class=\"line\"><span style=\"color:#032F62;--shiki-dark:#9ECBFF\">        \"\"\"Return the cached results, or None on a miss. A hit counts as a use.\"\"\"</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">        node </span><span style=\"color:#D73A49;--shiki-dark:#F97583\">=</span><span style=\"color:#005CC5;--shiki-dark:#79B8FF\"> self</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">._nodes.get(query)</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F97583\">        if</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\"> node </span><span style=\"color:#D73A49;--shiki-dark:#F97583\">is</span><span style=\"color:#005CC5;--shiki-dark:#79B8FF\"> None</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">:</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F97583\">            return</span><span style=\"color:#005CC5;--shiki-dark:#79B8FF\"> None</span></span>\n<span class=\"line\"><span style=\"color:#005CC5;--shiki-dark:#79B8FF\">        self</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">._recency.move_to_front(node)</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F97583\">        return</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\"> node.results</span></span>\n<span class=\"line\"></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F97583\">    def</span><span style=\"color:#005CC5;--shiki-dark:#79B8FF\"> set</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">(self, query: Any, results: Any) -> </span><span style=\"color:#005CC5;--shiki-dark:#79B8FF\">None</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">:</span></span>\n<span class=\"line\"><span style=\"color:#032F62;--shiki-dark:#9ECBFF\">        \"\"\"Store results for the query, evicting the oldest entry if full.\"\"\"</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">        node </span><span style=\"color:#D73A49;--shiki-dark:#F97583\">=</span><span style=\"color:#005CC5;--shiki-dark:#79B8FF\"> self</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">._nodes.get(query)</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F97583\">        if</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\"> node </span><span style=\"color:#D73A49;--shiki-dark:#F97583\">is</span><span style=\"color:#D73A49;--shiki-dark:#F97583\"> not</span><span style=\"color:#005CC5;--shiki-dark:#79B8FF\"> None</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">:</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">            node.results </span><span style=\"color:#D73A49;--shiki-dark:#F97583\">=</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\"> results</span></span>\n<span class=\"line\"><span style=\"color:#005CC5;--shiki-dark:#79B8FF\">            self</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">._recency.move_to_front(node)</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F97583\">            return</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F97583\">        if</span><span style=\"color:#005CC5;--shiki-dark:#79B8FF\"> len</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">(</span><span style=\"color:#005CC5;--shiki-dark:#79B8FF\">self</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">._nodes) </span><span style=\"color:#D73A49;--shiki-dark:#F97583\">==</span><span style=\"color:#005CC5;--shiki-dark:#79B8FF\"> self</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">.capacity:</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">            oldest </span><span style=\"color:#D73A49;--shiki-dark:#F97583\">=</span><span style=\"color:#005CC5;--shiki-dark:#79B8FF\"> self</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">._recency.remove_last()</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F97583\">            del</span><span style=\"color:#005CC5;--shiki-dark:#79B8FF\"> self</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">._nodes[oldest.query]</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">        node </span><span style=\"color:#D73A49;--shiki-dark:#F97583\">=</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\"> Node(query, results)</span></span>\n<span class=\"line\"><span style=\"color:#005CC5;--shiki-dark:#79B8FF\">        self</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">._recency.add_to_front(node)</span></span>\n<span class=\"line\"><span style=\"color:#005CC5;--shiki-dark:#79B8FF\">        self</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">._nodes[query] </span><span style=\"color:#D73A49;--shiki-dark:#F97583\">=</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\"> node</span></span>\n<span class=\"line\"></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F97583\">    def</span><span style=\"color:#005CC5;--shiki-dark:#79B8FF\"> __len__</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">(self) -> </span><span style=\"color:#005CC5;--shiki-dark:#79B8FF\">int</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">:</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F97583\">        return</span><span style=\"color:#005CC5;--shiki-dark:#79B8FF\"> len</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">(</span><span style=\"color:#005CC5;--shiki-dark:#79B8FF\">self</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">._nodes)</span></span></code></pre>\n<p>The key parts:</p>\n<ul>\n<li><strong>The dict maps a query to its node</strong>, not to its results. A <code>get</code> finds the node in O(1), and because the node is also a link in the list, the cache can move it to the front without searching the list.</li>\n<li><strong>Each node stores its own query.</strong> When <code>set</code> evicts the last node, it needs to delete that node's entry from the dict too, and the node is the only place that knows which query it belongs to. Forgetting this field is the most common bug in this exercise.</li>\n<li><strong>Sentinel nodes.</strong> <code>_head</code> and <code>_tail</code> are empty placeholder nodes that never hold data and never move. Every real node always has a node on each side, so <code>add_to_front</code> and <code>unlink</code> are four and two pointer assignments with no <code>if</code> statements. Without sentinels, you'd need special cases for an empty list, a one-node list, and removing the first or last node, and those special cases are where linked-list bugs live.</li>\n<li><strong><code>get</code> reorders the list.</strong> A hit calls <code>move_to_front</code>, so reading an entry protects it from eviction. A miss returns <code>None</code> and changes nothing.</li>\n<li><strong><code>set</code> handles two cases.</strong> For a query that's already cached, it replaces the results and moves the node to the front. For a new query, it evicts first if the cache is full, then adds the node at the front and records it in the dict. Evicting before inserting keeps the cache from ever holding <code>capacity + 1</code> entries.</li>\n<li><strong><code>len(self._nodes)</code> is the size.</strong> The dict already knows how many entries there are, so a separate counter would be one more thing to keep in step.</li>\n<li><strong><code>unlink</code> clears the removed node's pointers.</strong> Correctness doesn't depend on it. It means a stale node can't accidentally walk back into the list, and it drops references to neighbors that may themselves have been evicted.</li>\n<li><strong><code>capacity &#x3C; 1</code> is rejected.</strong> A cache with no room would try to evict from an empty list on its first <code>set</code>. <code>RecencyList.remove_last</code> also raises <code>IndexError</code> on an empty list, so a misuse fails loudly instead of corrupting the links.</li>\n</ul>","headings":[{"id":"solution","title":"Solution","depth":3}]},{"id":"walking-through-an-example","title":"Walking through an example","html":"<h3 id=\"walking-through-an-example\">Walking through an example</h3>\n<p>Take <code>LRUCache(capacity=3)</code>. The list column shows the entries from most to least recently used.</p>\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n<div class=\"table-wrap\"><table><thead><tr><th>Step</th><th>Call</th><th>What happens</th><th>List afterwards</th></tr></thead><tbody><tr><td>1</td><td><code>set(\"weather\", W1)</code></td><td>New query, cache has room. Add at front.</td><td>weather</td></tr><tr><td>2</td><td><code>set(\"python\", P1)</code></td><td>New query, add at front.</td><td>python, weather</td></tr><tr><td>3</td><td><code>set(\"maps\", M1)</code></td><td>New query, add at front. The cache is now full.</td><td>maps, python, weather</td></tr><tr><td>4</td><td><code>get(\"weather\")</code></td><td>Hit. Return W1 and move \"weather\" to the front.</td><td>weather, maps, python</td></tr><tr><td>5</td><td><code>set(\"news\", N1)</code></td><td>New query and the cache is full. Evict the last node, \"python\", and delete it from the dict. Add \"news\" at front.</td><td>news, weather, maps</td></tr><tr><td>6</td><td><code>get(\"python\")</code></td><td>Miss. Return <code>None</code>. Nothing moves.</td><td>news, weather, maps</td></tr><tr><td>7</td><td><code>set(\"maps\", M2)</code></td><td>Existing query. Replace M1 with M2 and move to front. No eviction.</td><td>maps, news, weather</td></tr><tr><td>8</td><td><code>set(\"sports\", S1)</code></td><td>New query, cache full. Evict \"weather\".</td><td>sports, maps, news</td></tr></tbody></table></div>\n<p>Step 4 is the one that separates LRU from a FIFO queue. \"weather\" was the oldest insertion, but reading it at step 4 made it recent, so at step 5 the cache evicted \"python\" instead.</p>\n<p>Here's the structure after step 8. The dict points into the middle of the list, which is what makes every operation O(1).</p>\n<figure class=\"diagram\" data-text=\"head sentinel <-> sports; sports <-> maps; maps <-> news; news <-> tail sentinel; dict: sports, maps, news -> maps (one pointer per key)\"><div class=\"diagram-scroll\"><svg class=\"dg\" viewBox=\"0 0 800 265\" width=\"800\" role=\"img\" aria-label=\"After step 8. The dict finds any node directly; the list order decides who is evicted next.\"><defs><marker id=\"dg-arrow\" viewBox=\"0 0 10 10\" refX=\"9\" refY=\"5\" markerWidth=\"7\" markerHeight=\"7\" orient=\"auto-start-reverse\"><path class=\"dg-arrowhead\" d=\"M0,1 L9,5 L0,9 z\"></path></marker></defs><line class=\"dg-edge\" x1=\"153\" y1=\"42\" x2=\"167\" y2=\"42\" marker-end=\"url(#dg-arrow)\" marker-start=\"url(#dg-arrow)\"></line><line class=\"dg-edge\" x1=\"313\" y1=\"42\" x2=\"327\" y2=\"42\" marker-end=\"url(#dg-arrow)\" marker-start=\"url(#dg-arrow)\"></line><line class=\"dg-edge\" x1=\"473\" y1=\"42\" x2=\"487\" y2=\"42\" marker-end=\"url(#dg-arrow)\" marker-start=\"url(#dg-arrow)\"></line><line class=\"dg-edge\" x1=\"633\" y1=\"42\" x2=\"647\" y2=\"42\" marker-end=\"url(#dg-arrow)\" marker-start=\"url(#dg-arrow)\"></line><line class=\"dg-edge\" x1=\"400\" y1=\"188\" x2=\"400\" y2=\"71\" marker-end=\"url(#dg-arrow)\"></line><rect class=\"dg-node dg-note\" x=\"12\" y=\"16\" width=\"136\" height=\"52\" rx=\"8\"></rect><text class=\"dg-label dg-muted\" text-anchor=\"middle\"><tspan x=\"80\" y=\"38.5\">head</tspan><tspan x=\"80\" y=\"54.5\">sentinel</tspan></text><rect class=\"dg-node dg-box\" x=\"172\" y=\"18\" width=\"136\" height=\"48\" rx=\"8\"></rect><text class=\"dg-label\" text-anchor=\"middle\"><tspan x=\"240\" y=\"46.5\">sports</tspan></text><rect class=\"dg-node dg-box\" x=\"332\" y=\"18\" width=\"136\" height=\"48\" rx=\"8\"></rect><text class=\"dg-label\" text-anchor=\"middle\"><tspan x=\"400\" y=\"46.5\">maps</tspan></text><rect class=\"dg-node dg-box\" x=\"492\" y=\"18\" width=\"136\" height=\"48\" rx=\"8\"></rect><text class=\"dg-label\" text-anchor=\"middle\"><tspan x=\"560\" y=\"46.5\">news</tspan></text><rect class=\"dg-node dg-note\" x=\"652\" y=\"16\" width=\"136\" height=\"52\" rx=\"8\"></rect><text class=\"dg-label dg-muted\" text-anchor=\"middle\"><tspan x=\"720\" y=\"38.5\">tail</tspan><tspan x=\"720\" y=\"54.5\">sentinel</tspan></text><path class=\"dg-node dg-store\" d=\"M332,198 a68,7 0 0,1 136,0 v48 a68,7 0 0,1 -136,0 z\"></path><path class=\"dg-line\" d=\"M332,198 a68,7 0 0,0 136,0\"></path><text class=\"dg-label\" text-anchor=\"middle\"><tspan x=\"400\" y=\"221.5\">dict: sports,</tspan><tspan x=\"400\" y=\"237.5\">maps, news</tspan></text><rect class=\"dg-edge-bg\" x=\"336.1\" y=\"120.5\" width=\"127.8\" height=\"18\" rx=\"4\"></rect><text class=\"dg-edge-label\" x=\"400\" y=\"133.5\" text-anchor=\"middle\">one pointer per key</text></svg></div><figcaption>After step 8. The dict finds any node directly; the list order decides who is evicted next.</figcaption></figure>","headings":[{"id":"walking-through-an-example","title":"Walking through an example","depth":3}]},{"id":"complexity-and-trade-offs","title":"Complexity and trade-offs","html":"<h3 id=\"complexity-and-trade-offs\">Complexity and trade-offs</h3>\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n<div class=\"table-wrap\"><table><thead><tr><th>Operation</th><th>Time</th><th>Why</th></tr></thead><tbody><tr><td><code>get</code> hit</td><td>O(1)</td><td>One dict lookup, then a fixed number of pointer updates to move the node.</td></tr><tr><td><code>get</code> miss</td><td>O(1)</td><td>One dict lookup.</td></tr><tr><td><code>set</code> update</td><td>O(1)</td><td>Dict lookup, replace results, move to front.</td></tr><tr><td><code>set</code> insert with eviction</td><td>O(1)</td><td>Unlink the tail's neighbor, one dict delete, one link at front, one dict insert.</td></tr></tbody></table></div>\n<p>Space is O(capacity). Each entry costs one dict slot plus one node with two pointers, on top of the query and results themselves.</p>\n<h4 id=\"why-a-hash-map-plus-a-doubly-linked-list\">Why a hash map plus a doubly linked list</h4>\n<p>It helps to show the interviewer the alternatives you rejected:</p>\n<ul>\n<li><strong>A dict plus a \"last used\" timestamp in each entry.</strong> <code>get</code> and <code>set</code> are O(1), but eviction has to scan every entry to find the oldest timestamp: O(n).</li>\n<li><strong>A dict plus a min-heap ordered by timestamp.</strong> Eviction becomes O(log n), but every <code>get</code> changes a timestamp, and updating an element's position inside a heap is O(log n) too.</li>\n<li><strong>A dict plus a singly linked list.</strong> Moving a node to the front requires unlinking it, and unlinking needs the node before it. With only forward pointers, finding that node means walking from the start of the list: O(n). The back pointer is what makes removal from the middle O(1).</li>\n</ul>\n<p>With a doubly linked list, each node knows both neighbors, and the dict gives you the node directly. Every operation is a lookup plus a constant number of pointer changes.</p>\n<h4 id=\"pythons-shortcut-ordereddict\">Python's shortcut: <code>OrderedDict</code></h4>\n<p>Python's <code>collections.OrderedDict</code> already combines a hash map with an ordering it can change cheaply. Its documentation says it \"can handle frequent reordering operations better than <code>dict</code>\" and that \"this makes it suitable for implementing various kinds of LRU caches.\" <code>move_to_end(key)</code> marks a key as most recent, and <code>popitem(last=False)</code> removes the oldest. The same behavior in a dozen lines:</p>\n<pre class=\"shiki shiki-themes github-light github-dark\" style=\"background-color:#fff;--shiki-dark-bg:#24292e;color:#24292e;--shiki-dark:#e1e4e8\" tabindex=\"0\"><code><span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F97583\">from</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\"> collections </span><span style=\"color:#D73A49;--shiki-dark:#F97583\">import</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\"> OrderedDict</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F97583\">from</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\"> typing </span><span style=\"color:#D73A49;--shiki-dark:#F97583\">import</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\"> Any</span></span>\n<span class=\"line\"></span>\n<span class=\"line\"></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F97583\">class</span><span style=\"color:#6F42C1;--shiki-dark:#B392F0\"> OrderedDictLRUCache</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">:</span></span>\n<span class=\"line\"><span style=\"color:#032F62;--shiki-dark:#9ECBFF\">    \"\"\"The same behavior as LRUCache, built on OrderedDict.\"\"\"</span></span>\n<span class=\"line\"></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F97583\">    def</span><span style=\"color:#005CC5;--shiki-dark:#79B8FF\"> __init__</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">(self, capacity: </span><span style=\"color:#005CC5;--shiki-dark:#79B8FF\">int</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">) -> </span><span style=\"color:#005CC5;--shiki-dark:#79B8FF\">None</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">:</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F97583\">        if</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\"> capacity </span><span style=\"color:#D73A49;--shiki-dark:#F97583\">&#x3C;</span><span style=\"color:#005CC5;--shiki-dark:#79B8FF\"> 1</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">:</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F97583\">            raise</span><span style=\"color:#005CC5;--shiki-dark:#79B8FF\"> ValueError</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">(</span><span style=\"color:#032F62;--shiki-dark:#9ECBFF\">\"capacity must be at least 1\"</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">)</span></span>\n<span class=\"line\"><span style=\"color:#005CC5;--shiki-dark:#79B8FF\">        self</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">.capacity </span><span style=\"color:#D73A49;--shiki-dark:#F97583\">=</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\"> capacity</span></span>\n<span class=\"line\"><span style=\"color:#005CC5;--shiki-dark:#79B8FF\">        self</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">._entries: OrderedDict[Any, Any] </span><span style=\"color:#D73A49;--shiki-dark:#F97583\">=</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\"> OrderedDict()  </span><span style=\"color:#6A737D;--shiki-dark:#6A737D\"># oldest first</span></span>\n<span class=\"line\"></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F97583\">    def</span><span style=\"color:#6F42C1;--shiki-dark:#B392F0\"> get</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">(self, query: Any) -> Any:</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F97583\">        if</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\"> query </span><span style=\"color:#D73A49;--shiki-dark:#F97583\">not</span><span style=\"color:#D73A49;--shiki-dark:#F97583\"> in</span><span style=\"color:#005CC5;--shiki-dark:#79B8FF\"> self</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">._entries:</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F97583\">            return</span><span style=\"color:#005CC5;--shiki-dark:#79B8FF\"> None</span></span>\n<span class=\"line\"><span style=\"color:#005CC5;--shiki-dark:#79B8FF\">        self</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">._entries.move_to_end(query)</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F97583\">        return</span><span style=\"color:#005CC5;--shiki-dark:#79B8FF\"> self</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">._entries[query]</span></span>\n<span class=\"line\"></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F97583\">    def</span><span style=\"color:#005CC5;--shiki-dark:#79B8FF\"> set</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">(self, query: Any, results: Any) -> </span><span style=\"color:#005CC5;--shiki-dark:#79B8FF\">None</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">:</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F97583\">        if</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\"> query </span><span style=\"color:#D73A49;--shiki-dark:#F97583\">in</span><span style=\"color:#005CC5;--shiki-dark:#79B8FF\"> self</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">._entries:</span></span>\n<span class=\"line\"><span style=\"color:#005CC5;--shiki-dark:#79B8FF\">            self</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">._entries.move_to_end(query)</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F97583\">        elif</span><span style=\"color:#005CC5;--shiki-dark:#79B8FF\"> len</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">(</span><span style=\"color:#005CC5;--shiki-dark:#79B8FF\">self</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">._entries) </span><span style=\"color:#D73A49;--shiki-dark:#F97583\">==</span><span style=\"color:#005CC5;--shiki-dark:#79B8FF\"> self</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">.capacity:</span></span>\n<span class=\"line\"><span style=\"color:#005CC5;--shiki-dark:#79B8FF\">            self</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">._entries.popitem(</span><span style=\"color:#E36209;--shiki-dark:#FFAB70\">last</span><span style=\"color:#D73A49;--shiki-dark:#F97583\">=</span><span style=\"color:#005CC5;--shiki-dark:#79B8FF\">False</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">)</span></span>\n<span class=\"line\"><span style=\"color:#005CC5;--shiki-dark:#79B8FF\">        self</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">._entries[query] </span><span style=\"color:#D73A49;--shiki-dark:#F97583\">=</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\"> results</span></span></code></pre>\n<p>In production Python, use this or <code>functools.lru_cache</code>. In an interview, mention it, and then expect to be asked to write the linked list yourself, because that's the part being tested.</p>\n<p><code>functools.lru_cache</code> is the decorator form, for caching a function's return values by its arguments. Its documentation says the default <code>maxsize</code> is 128, and setting <code>maxsize=None</code> disables the LRU feature so the cache grows without bound (<code>functools.cache</code>, added in Python 3.9, is that unbounded version). Since a dictionary holds the results, the arguments must be hashable. The wrapped function gains <code>cache_info()</code>, which reports hits, misses, maxsize and current size, and <code>cache_clear()</code>. The documentation also gives the reasoning for LRU in one line: it \"works best when the most recent calls are the best predictors of upcoming calls.\"</p>\n<h4 id=\"thread-safety\">Thread safety</h4>\n<p>Our cache isn't safe to share between threads. Even <code>get</code> writes, since it moves a node. If two threads move nodes at the same moment, their pointer updates can interleave and leave the list corrupted, with nodes pointing to neighbors that no longer point back. The fix is in the extensions below. For comparison, <code>functools.lru_cache</code> is documented as threadsafe in the sense that its data structure \"will remain coherent during concurrent updates\", though the wrapped function may run more than once if a second thread calls it before the first call's result is cached.</p>\n<h4 id=\"exact-lru-vs-approximate-lru\">Exact LRU vs approximate LRU</h4>\n<p>Exact LRU costs two pointers per entry and a list update on every read. Large caches often trade a little accuracy for less memory. Redis is the standard example. Its eviction documentation explains that it \"samples a small number of keys at random and then evicts the ones with the longest time since last access\". The sample size is set by <code>maxmemory-samples</code>, which defaults to 5. Since Redis 3.0 it also keeps a pool of good eviction candidates between evictions. The docs say Redis avoids true LRU \"because it costs more memory\", and that the approximation \"is virtually equivalent for an application using Redis.\" <a href=\"/learn/cache#when-the-cache-is-full\">Caching</a> compares LRU with the other eviction policies.</p>","headings":[{"id":"complexity-and-trade-offs","title":"Complexity and trade-offs","depth":3},{"id":"why-a-hash-map-plus-a-doubly-linked-list","title":"Why a hash map plus a doubly linked list","depth":4},{"id":"pythons-shortcut-ordereddict","title":"Python's shortcut: OrderedDict","depth":4},{"id":"thread-safety","title":"Thread safety","depth":4},{"id":"exact-lru-vs-approximate-lru","title":"Exact LRU vs approximate LRU","depth":4}]},{"id":"extensions-and-follow-up-questions","title":"Extensions and follow-up questions","html":"<h3 id=\"extensions-and-follow-up-questions\">Extensions and follow-up questions</h3>\n<p><strong>\"Entries should expire after a while.\"</strong> Add a <strong>time to live (TTL)</strong>: store an expiry time in each node when it's set. In <code>get</code>, if the node's time has passed, unlink it, delete it from the dict, and report a miss. Pass the clock into the constructor so tests can control time. Expired entries that nobody reads stay in memory until LRU evicts them; if that matters, add a background sweep. <a href=\"/learn/design-query-cache#when-to-update-the-cache\">Design a query cache</a> builds this into a cache for search results and discusses how to choose TTLs.</p>\n<p><strong>\"Make it safe for several threads.\"</strong> Put one lock around both methods:</p>\n<pre class=\"shiki shiki-themes github-light github-dark\" style=\"background-color:#fff;--shiki-dark-bg:#24292e;color:#24292e;--shiki-dark:#e1e4e8\" tabindex=\"0\"><code><span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F97583\">import</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\"> threading</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F97583\">from</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\"> typing </span><span style=\"color:#D73A49;--shiki-dark:#F97583\">import</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\"> Any</span></span>\n<span class=\"line\"></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F97583\">from</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\"> lru_cache </span><span style=\"color:#D73A49;--shiki-dark:#F97583\">import</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\"> LRUCache   </span><span style=\"color:#6A737D;--shiki-dark:#6A737D\"># the LRUCache class from the solution</span></span>\n<span class=\"line\"></span>\n<span class=\"line\"></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F97583\">class</span><span style=\"color:#6F42C1;--shiki-dark:#B392F0\"> ThreadSafeLRUCache</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">:</span></span>\n<span class=\"line\"><span style=\"color:#032F62;--shiki-dark:#9ECBFF\">    \"\"\"Wraps LRUCache so only one thread changes it at a time.\"\"\"</span></span>\n<span class=\"line\"></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F97583\">    def</span><span style=\"color:#005CC5;--shiki-dark:#79B8FF\"> __init__</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">(self, capacity: </span><span style=\"color:#005CC5;--shiki-dark:#79B8FF\">int</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">) -> </span><span style=\"color:#005CC5;--shiki-dark:#79B8FF\">None</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">:</span></span>\n<span class=\"line\"><span style=\"color:#005CC5;--shiki-dark:#79B8FF\">        self</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">._cache </span><span style=\"color:#D73A49;--shiki-dark:#F97583\">=</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\"> LRUCache(capacity)</span></span>\n<span class=\"line\"><span style=\"color:#005CC5;--shiki-dark:#79B8FF\">        self</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">._lock </span><span style=\"color:#D73A49;--shiki-dark:#F97583\">=</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\"> threading.Lock()</span></span>\n<span class=\"line\"></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F97583\">    def</span><span style=\"color:#6F42C1;--shiki-dark:#B392F0\"> get</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">(self, query: Any) -> Any:</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F97583\">        with</span><span style=\"color:#005CC5;--shiki-dark:#79B8FF\"> self</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">._lock:      </span><span style=\"color:#6A737D;--shiki-dark:#6A737D\"># a get moves a node, so it needs the lock too</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F97583\">            return</span><span style=\"color:#005CC5;--shiki-dark:#79B8FF\"> self</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">._cache.get(query)</span></span>\n<span class=\"line\"></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F97583\">    def</span><span style=\"color:#005CC5;--shiki-dark:#79B8FF\"> set</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">(self, query: Any, results: Any) -> </span><span style=\"color:#005CC5;--shiki-dark:#79B8FF\">None</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">:</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F97583\">        with</span><span style=\"color:#005CC5;--shiki-dark:#79B8FF\"> self</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">._lock:</span></span>\n<span class=\"line\"><span style=\"color:#005CC5;--shiki-dark:#79B8FF\">            self</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">._cache.set(query, results)</span></span></code></pre>\n<p>A read-write lock doesn't help here, since every <code>get</code> writes to the list. If one lock becomes a bottleneck, split the cache into several independent shards by hashing the key, each with its own lock and its own LRU list. Eviction is then LRU within each shard, which is close enough in practice. Another option is to record reads in a buffer and apply the reordering in batches.</p>\n<p><strong>\"What about LFU?\"</strong> <strong>Least frequently used (LFU)</strong> evicts the entry read the fewest times. It keeps long-term favorites that LRU might drop after a short quiet spell. An O(1) LFU keeps a dict from key to node and a second dict from each access count to a doubly linked list of nodes with that count, plus the current minimum count. A hit moves the node from the count-k list to the count-(k + 1) list. Eviction removes the least recent node from the minimum-count list. LFU's weakness is the reverse of LRU's: an entry that was popular last week keeps a high count and lingers. Redis's LFU mode, available since Redis 4.0, handles this with a small probabilistic counter per key that decays over time, by default once a minute.</p>\n<p><strong>\"One crawler-style burst of one-off queries flushes the whole cache.\"</strong> That's a known weakness of LRU: a scan of many keys, each used once, pushes out the popular ones. One fix is to admit an entry only on its second request. Python's <code>collections</code> documentation includes a recipe that does this, a cache that, in its words, avoids \"flushing the LRU cache with one-time requests\" by not caching \"until a request has been made more than once.\"</p>\n<p><strong>\"Limit the cache by memory instead of entry count.\"</strong> Store each entry's size in its node and keep a running total. On <code>set</code>, evict from the tail in a loop until the new entry fits. Decide what happens to a single entry larger than the whole budget (usually: don't cache it).</p>\n<p><strong>\"The cache must serve many servers.\"</strong> Run the cache as its own service on several machines and decide which machine owns each key with <a href=\"/learn/consistent-hashing\">consistent hashing</a>, so adding a cache node moves only a small share of keys. Each node runs its own LRU over the keys it owns. You also have to decide how the cache stays fresh when the underlying data changes, and how to protect the database when many entries miss at once. <a href=\"/learn/design-query-cache\">Design a query cache</a> works through this whole system, with an LRU cache at its core.</p>","headings":[{"id":"extensions-and-follow-up-questions","title":"Extensions and follow-up questions","depth":3}]},{"id":"check-yourself","title":"Check yourself","html":"<h3 id=\"check-yourself\">Check yourself</h3>\n<details>\n<summary>Your LRU cache works until the first eviction, then a later <code>get</code> returns results for a query that was supposedly evicted. What's the likely bug?</summary>\n<p>Eviction removed the node from the linked list but didn't delete its key from the dict, so the dict still points at the evicted node. Usually the node doesn't store its own key, so the code can't tell which dict entry to delete. Store the key in each node and delete <code>nodes[oldest.key]</code> whenever the tail is evicted.</p>\n</details>\n<details>\n<summary>Why must the list be doubly linked? Wouldn't a singly linked list with a dict work?</summary>\n<p>Moving a node to the front means unlinking it from wherever it is, and unlinking means pointing its predecessor at its successor. In a singly linked list, a node doesn't know its predecessor, so you'd walk from the head to find it, which is O(n). The back pointer makes the predecessor available in O(1).</p>\n</details>\n<details>\n<summary>With capacity 2, run <code>set(a)</code>, <code>set(b)</code>, <code>get(a)</code>, <code>set(c)</code>, <code>set(b)</code>. Which keys are cached at the end, in order from most to least recent?</summary>\n<p>After <code>set(a)</code>, <code>set(b)</code>: b, a. The <code>get(a)</code> makes it: a, b. <code>set(c)</code> evicts b: c, a. <code>set(b)</code> is a new key again, so it evicts a: b, c. A FIFO cache would have evicted a at the <code>set(c)</code> step instead.</p>\n</details>\n<details>\n<summary>What do sentinel nodes buy you, and what do they cost?</summary>\n<p>They make every real node have a neighbor on each side, so inserting and unlinking need no checks for an empty list or for the first and last node. That removes the special cases where linked-list bugs usually hide. The cost is two extra nodes per cache, which is nothing.</p>\n</details>\n<details>\n<summary>A teammate wraps <code>set</code> in a lock but leaves <code>get</code> unlocked, reasoning that reads are safe. What can go wrong?</summary>\n<p>In an LRU cache, <code>get</code> is a write: a hit moves the node to the front, which changes several pointers. A <code>get</code> running at the same time as a <code>set</code> or another <code>get</code> can interleave those updates and leave nodes whose <code>prev</code> and <code>next</code> don't agree, which loses entries or creates loops. Every method that touches the list needs the lock.</p>\n</details>\n<details>\n<summary>Redis doesn't keep an exact LRU list. Why, and when would you choose LFU over LRU?</summary>\n<p>An exact list costs two pointers per key and list work on every read, which adds up across millions of keys. Redis samples a few keys (5 by default) and evicts the least recently used among them, which its documentation describes as virtually equivalent for applications. Choose LFU when some keys stay popular over long periods but have quiet spells, since LRU would evict them during the lull; LFU keeps them because their count stays high.</p>\n</details>\n<h3 id=\"further-reading\">Further reading</h3>\n<ul>\n<li><a href=\"https://docs.python.org/3/library/functools.html#functools.lru_cache\" target=\"_blank\" rel=\"noopener noreferrer\"><code>functools.lru_cache</code></a> (Python documentation): the standard-library LRU decorator, including <code>maxsize</code>, <code>typed</code>, <code>cache_info()</code> and its thread-safety guarantee.</li>\n<li><a href=\"https://docs.python.org/3/library/collections.html#collections.OrderedDict\" target=\"_blank\" rel=\"noopener noreferrer\"><code>collections.OrderedDict</code></a> (Python documentation): <code>move_to_end</code> and <code>popitem(last=False)</code>, plus recipes for a time-bounded LRU cache and one that only caches repeated requests.</li>\n<li><a href=\"https://redis.io/docs/latest/develop/reference/eviction/\" target=\"_blank\" rel=\"noopener noreferrer\">Key eviction</a> (Redis documentation): every eviction policy, how Redis approximates LRU by sampling, and how its LFU counters decay.</li>\n<li><a href=\"/learn/cache\">Caching</a>: where caches sit in a system, how they're kept up to date, and how eviction policies compare.</li>\n<li><a href=\"/learn/design-query-cache\">Design a query cache</a>: an LRU cache with TTLs as the core of a search results cache, scaled across many machines.</li>\n<li><a href=\"/learn/ood-hash-table\">Design a hash map</a>: how the dict inside this cache works.</li>\n</ul>","headings":[{"id":"check-yourself","title":"Check yourself","depth":3},{"id":"further-reading","title":"Further reading","depth":3}]}]}