Yi's Blog

Yi's blog

Solving Jane Street's 'Dropped a Neural Net' Puzzle

<p>Jane Street’s January 2026 puzzle<sup id="fnref:1"><a href="#fn:1" class="footnote" rel="footnote" role="doc-noteref">1</a></sup>, <a href="https://huggingface.co/spaces/jane-street/droppedaneuralnet">“Dropped a Neural Net”</a>, presents a deceptively simple premise: a neural network was “dropped” and its 97 pieces scattered. Your job is to put them back together. Behind this simple framing lies a deep combinatorial optimization problem that I solved two different ways — first with gradient-based permutation learning and combined swaps, then again with a simpler approach that revealed a key insight: <strong>pairing corrections unlock cascading improvements in ordering</strong>.</p>  <h2 id="the-problem">The Problem</h2> <p>You’re given 97 weight/bias files (<code class="language-plaintext highlighter-rouge">piece_0.pth</code> through <code class="language-plaintext highlighter-rouge">piece_96.pth</code>) and a dataset (<code class="language-plaintext highlighter-rouge">historical_data.csv</code> with 10,000 rows of 48 input features, plus <code class="language-plaintext highlighter-rouge">pred</code> and <code class="language-plaintext highlighter-rouge">true</code> columns). The neural network architecture is:</p> <ul> <li><strong>48 residual blocks</strong>, each consisting of: <ul> <li>An “inp” layer: <code class="language-plaintext highlighter-rouge">Linear(48 → 96)</code> followed by ReLU</li> <li>An “out” layer: <code class="language-plaintext highlighter-rouge">Linear(96 → 48)</code></li> <li>A residual connection: <code class="language-plaintext highlighter-rouge">x = x + out(relu(inp(x)))</code></li> </ul> </li> <li><strong>1 final layer</strong>: <code class="language-plaintext highlighter-rouge">Linear(48 → 1)</code> producing the prediction</li> </ul> <p>The 97 pieces split into three groups by weight shape:</p> <ul> <li>48 pieces with shape <code class="language-plaintext highlighter-rouge">(96, 48)</code> — the inp layers</li> <li>48 pieces with shape <code class="language-plaintext highlighter-rouge">(48, 96)</code> — the out layers</li> <li>1 piece with shape <code class="language-plaintext highlighter-rouge">(1, 48)</code> — the final layer</li> </ul> <p>The solution is a permutation of indices 0–96 specifying which piece goes where. Positions 0,2,4,…,94 hold inp layers, positions 1,3,5,…,95 hold out layers, and position 96 holds the final layer. The solution is verified by <strong>SHA-256 hash</strong> — there’s exactly one correct answer, no MSE threshold to meet.</p> <p>This means you need to solve two sub-problems simultaneously:</p> <ol> <li><strong>Pairing</strong>: Which inp layer goes with which out layer in each block?</li> <li><strong>Ordering</strong>: In what sequence do the 48 blocks execute?</li> </ol> <p>The search space is enormous: 48! × 48! ≈ 10<sup>121</sup> possible configurations.</p> <h2 id="phase-1-first-order-approximations-mse-07">Phase 1: First-Order Approximations (MSE ~0.7)</h2> <p>My first instinct was to exploit the linear structure. If all 48 blocks see roughly the same input <code class="language-plaintext highlighter-rouge">X</code> (a first-order approximation), then each block’s contribution is independent, and we can use the <strong>Hungarian algorithm</strong> to find the optimal pairing.</p> <p>For each candidate pair <code class="language-plaintext highlighter-rouge">(i, j)</code>, I computed the block’s effect on the prediction:</p> <div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">h</span> <span class="o">=</span> <span class="n">F</span><span class="p">.</span><span class="nf">relu</span><span class="p">(</span><span class="n">F</span><span class="p">.</span><span class="nf">linear</span><span class="p">(</span><span class="n">X</span><span class="p">,</span> <span class="n">L1_W</span><span class="p">[</span><span class="n">i</span><span class="p">],</span> <span class="n">L1_B</span><span class="p">[</span><span class="n">i</span><span class="p">]))</span> <span class="n">delta</span> <span class="o">=</span> <span class="n">F</span><span class="p">.</span><span class="nf">linear</span><span class="p">(</span><span class="n">h</span><span class="p">,</span> <span class="n">L2_W</span><span class="p">[</span><span class="n">j</span><span class="p">],</span> <span class="n">L2_B</span><span class="p">[</span><span class="n">j</span><span class="p">])</span> <span class="c1"># (N, 48) </span><span class="n">pred_delta</span> <span class="o">=</span> <span class="p">(</span><span class="n">delta</span> <span class="o">*</span> <span class="n">l3_dir</span><span class="p">).</span><span class="nf">sum</span><span class="p">(</span><span class="n">dim</span><span class="o">=</span><span class="mi">1</span><span class="p">)</span> <span class="o">*</span> <span class="n">l3_w</span><span class="p">.</span><span class="nf">norm</span><span class="p">()</span> </code></pre></div></div> <p>Then built a cost matrix and ran <code class="language-plaintext highlighter-rouge">linear_sum_assignment</code>. This got MSE down to ~0.7 — a starting point, but far from correct. The first-order approximation breaks down because blocks modify <code class="language-plaintext highlighter-rouge">x</code> sequentially, and the cumulative change is large (~6× the input norm).</p> <h2 id="phase-2-gumbel-sinkhorn--differentiable-permutation-learning-mse-003">Phase 2: Gumbel-Sinkhorn — Differentiable Permutation Learning (MSE ~0.03)</h2> <p>The breakthrough came from treating permutations as differentiable objects using the <strong>Gumbel-Sinkhorn</strong> framework.</p> <h3 id="the-key-idea">The Key Idea</h3> <p>Instead of searching over discrete permutations, parameterize a continuous relaxation. A 48×48 matrix of learnable logits <code class="language-plaintext highlighter-rouge">log_alpha</code> is transformed into a doubly-stochastic matrix (a “soft permutation”) via iterated row/column normalization (Sinkhorn’s algorithm):</p> <div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="nf">sinkhorn</span><span class="p">(</span><span class="n">log_alpha</span><span class="p">,</span> <span class="n">n_iters</span><span class="o">=</span><span class="mi">25</span><span class="p">,</span> <span class="n">tau</span><span class="o">=</span><span class="mf">1.0</span><span class="p">):</span> <span class="n">log_alpha</span> <span class="o">=</span> <span class="n">log_alpha</span> <span class="o">/</span> <span class="n">tau</span> <span class="k">for</span> <span class="n">_</span> <span class="ow">in</span> <span class="nf">range</span><span class="p">(</span><span class="n">n_iters</span><span class="p">):</span> <span class="n">log_alpha</span> <span class="o">=</span> <span class="n">log_alpha</span> <span class="o">-</span> <span class="n">torch</span><span class="p">.</span><span class="nf">logsumexp</span><span class="p">(</span><span class="n">log_alpha</span><span class="p">,</span> <span class="n">dim</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span> <span class="n">keepdim</span><span class="o">=</span><span class="bp">True</span><span class="p">)</span> <span class="n">log_alpha</span> <span class="o">=</span> <span class="n">log_alpha</span> <span class="o">-</span> <span class="n">torch</span><span class="p">.</span><span class="nf">logsumexp</span><span class="p">(</span><span class="n">log_alpha</span><span class="p">,</span> <span class="n">dim</span><span class="o">=</span><span class="mi">0</span><span class="p">,</span> <span class="n">keepdim</span><span class="o">=</span><span class="bp">True</span><span class="p">)</span> <span class="k">return</span> <span class="n">log_alpha</span><span class="p">.</span><span class="nf">exp</span><span class="p">()</span> </code></pre></div></div> <p>Adding Gumbel noise before normalization enables exploration, and annealing the temperature <code class="language-plaintext highlighter-rouge">tau</code> from high to low gradually sharpens the soft permutation toward a hard one. The MSE loss is fully differentiable through this soft permutation, so we can use Adam to optimize the logits.</p> <h3 id="alternating-optimization">Alternating Optimization</h3> <p>Jointly optimizing both the ordering permutation and the pairing permutation is expensive — the forward pass with two soft permutations involves <code class="language-plaintext highlighter-rouge">O(48³)</code> operations per position. The key insight was to <strong>alternate</strong>:</p> <ol> <li><strong>Fix pairing, optimize ordering</strong>: The soft forward pass weights different block orderings: <div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="nf">forward_soft_order</span><span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="n">pairing</span><span class="p">,</span> <span class="n">order_weights</span><span class="p">):</span> <span class="k">for</span> <span class="n">pos</span> <span class="ow">in</span> <span class="nf">range</span><span class="p">(</span><span class="mi">48</span><span class="p">):</span> <span class="c1"># Precompute all block deltas with fixed pairing </span> <span class="n">all_deltas</span> <span class="o">=</span> <span class="p">[</span><span class="nf">block_i_j</span><span class="p">(</span><span class="n">x</span><span class="p">)</span> <span class="k">for</span> <span class="n">i</span><span class="p">,</span><span class="n">j</span> <span class="ow">in</span> <span class="n">pairing</span><span class="p">]</span> <span class="c1"># Weighted combination based on soft ordering </span> <span class="n">delta</span> <span class="o">=</span> <span class="nf">einsum</span><span class="p">(</span><span class="sh">'</span><span class="s">i,bid->bd</span><span class="sh">'</span><span class="p">,</span> <span class="n">order_weights</span><span class="p">[</span><span class="n">pos</span><span class="p">],</span> <span class="n">all_deltas</span><span class="p">)</span> <span class="n">x</span> <span class="o">=</span> <span class="n">x</span> <span class="o">+</span> <span class="n">delta</span> </code></pre></div> </div> </li> <li><strong>Fix ordering, optimize pairing</strong>: Each block position softly selects among all possible out layers: <div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="nf">forward_soft_pair</span><span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="n">order</span><span class="p">,</span> <span class="n">pair_weights</span><span class="p">):</span> <span class="k">for</span> <span class="n">inp_idx</span> <span class="ow">in</span> <span class="n">order</span><span class="p">:</span> <span class="n">h</span> <span class="o">=</span> <span class="nf">relu</span><span class="p">(</span><span class="nf">linear</span><span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="n">L1_W</span><span class="p">[</span><span class="n">inp_idx</span><span class="p">],</span> <span class="n">L1_B</span><span class="p">[</span><span class="n">inp_idx</span><span class="p">]))</span> <span class="c1"># Soft-select out layer </span> <span class="n">weighted_w</span> <span class="o">=</span> <span class="nf">einsum</span><span class="p">(</span><span class="sh">'</span><span class="s">j,jdo->do</span><span class="sh">'</span><span class="p">,</span> <span class="n">pair_weights</span><span class="p">[</span><span class="n">inp_idx</span><span class="p">],</span> <span class="n">L2_W</span><span class="p">)</span> <span class="n">delta</span> <span class="o">=</span> <span class="nf">linear</span><span class="p">(</span><span class="n">h</span><span class="p">,</span> <span class="n">weighted_w</span><span class="p">,</span> <span class="n">weighted_b</span><span class="p">)</span> <span class="n">x</span> <span class="o">=</span> <span class="n">x</span> <span class="o">+</span> <span class="n">delta</span> </code></pre></div> </div> </li> </ol> <p>Each sub-problem only involves one 48×48 permutation matrix, making it much faster. After optimization, I extract hard permutations using the Hungarian algorithm on the negative logits.</p> <p>With 5-6 alternations of 500-800 gradient steps each, MSE dropped from 0.8 to <strong>~0.03</strong> — an order of magnitude better than first-order methods.</p> <h3 id="why-alternating-works">Why Alternating Works</h3> <p>Alternating optimization works here because the ordering and pairing sub-problems are partially decoupled. Fixing one makes the other a “standard” assignment problem with a smooth loss landscape. The Gumbel noise acts as a form of stochastic exploration, and the temperature annealing provides a natural curriculum from exploration to exploitation.</p> <h2 id="phase-3-local-search--getting-stuck-mse-003">Phase 3: Local Search — Getting Stuck (MSE ~0.03)</h2> <p>With a good Gumbel-Sinkhorn solution in hand, I tried various local search strategies:</p> <ul> <li><strong>2-opt</strong>: Swap pairs of positions in the ordering, or pairs of pairings</li> <li><strong>3-opt</strong>: Try all triples of positions with all 6 permutations</li> <li><strong>Insertion moves</strong>: Remove a block and reinsert at every other position</li> <li><strong>Coordinate descent</strong>: For each position, try all 48×48 possible replacements</li> </ul> <p>None of these could escape the MSE ~0.03 basin. The solution was at a strict local minimum for all single-element and pair-element moves. Multiple random restarts with the Gumbel approach also converged to similar MSE values.</p> <h2 id="phase-4-two-paths-to-the-solution-mse-0008--00">Phase 4: Two Paths to the Solution (MSE 0.008 → 0.0)</h2> <p>From MSE ~0.008, I found two different approaches that both reach MSE = 0. Each reveals something different about the problem structure.</p> <h3 id="approach-a-combined-2-opt">Approach A: Combined 2-opt</h3> <p>The first insight was that standard 2-opt treats order swaps and pairing swaps as <strong>independent moves</strong>. But the correct solution might require simultaneously changing both the order AND the pairing of two positions.</p> <p><strong>Combined 2-opt</strong> tests all three modifications for each pair of positions <code class="language-plaintext highlighter-rouge">(p1, p2)</code>:</p> <ol> <li>Swap their order positions only</li> <li>Swap their pairings only</li> <li>Swap both order AND pairing simultaneously</li> </ol> <div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">for</span> <span class="n">p1</span> <span class="ow">in</span> <span class="nf">range</span><span class="p">(</span><span class="mi">48</span><span class="p">):</span> <span class="k">for</span> <span class="n">p2</span> <span class="ow">in</span> <span class="nf">range</span><span class="p">(</span><span class="n">p1</span><span class="o">+</span><span class="mi">1</span><span class="p">,</span> <span class="mi">48</span><span class="p">):</span> <span class="n">i1</span><span class="p">,</span> <span class="n">i2</span> <span class="o">=</span> <span class="n">order</span><span class="p">[</span><span class="n">p1</span><span class="p">],</span> <span class="n">order</span><span class="p">[</span><span class="n">p2</span><span class="p">]</span> <span class="n">j1</span><span class="p">,</span> <span class="n">j2</span> <span class="o">=</span> <span class="n">pairing</span><span class="p">[</span><span class="n">i1</span><span class="p">],</span> <span class="n">pairing</span><span class="p">[</span><span class="n">i2</span><span class="p">]</span> <span class="k">for</span> <span class="n">swap_order</span><span class="p">,</span> <span class="n">swap_pair</span> <span class="ow">in</span> <span class="p">[(</span><span class="bp">True</span><span class="p">,</span><span class="bp">False</span><span class="p">),</span> <span class="p">(</span><span class="bp">False</span><span class="p">,</span><span class="bp">True</span><span class="p">),</span> <span class="p">(</span><span class="bp">True</span><span class="p">,</span><span class="bp">True</span><span class="p">)]:</span> <span class="k">if</span> <span class="n">swap_order</span><span class="p">:</span> <span class="n">order</span><span class="p">[</span><span class="n">p1</span><span class="p">],</span> <span class="n">order</span><span class="p">[</span><span class="n">p2</span><span class="p">]</span> <span class="o">=</span> <span class="n">i2</span><span class="p">,</span> <span class="n">i1</span> <span class="k">if</span> <span class="n">swap_pair</span><span class="p">:</span> <span class="n">pairing</span><span class="p">[</span><span class="n">i1</span><span class="p">],</span> <span class="n">pairing</span><span class="p">[</span><span class="n">i2</span><span class="p">]</span> <span class="o">=</span> <span class="n">j2</span><span class="p">,</span> <span class="n">j1</span> <span class="n">mse</span> <span class="o">=</span> <span class="nf">full_eval</span><span class="p">(</span><span class="n">order</span><span class="p">,</span> <span class="n">pairing</span><span class="p">)</span> <span class="k">if</span> <span class="n">mse</span> <span class="o"><</span> <span class="n">best_mse</span><span class="p">:</span> <span class="c1"># Accept improvement </span> <span class="bp">...</span> </code></pre></div></div> <p>This is <code class="language-plaintext highlighter-rouge">O(48² × 3)</code> = 6,912 evaluations per sweep. Starting from MSE 0.0085, it made 86 consecutive improving swaps in a single pass down to MSE = 0.</p> <p>The intuition: when two blocks have tangled errors, swapping just their order or just their pairing each makes things worse, but swapping <strong>both simultaneously</strong> moves between consistent configurations. In optimization terms, the individual moves each increase the loss, but their composition decreases it — a “valley” that requires moving diagonally.</p> <h3 id="approach-b-alternating-cycles-with-insertions-simpler-same-result">Approach B: Alternating Cycles with Insertions (Simpler, Same Result)</h3> <p>The second approach is simpler but equally effective: <strong>cycle through three move types</strong> and keep going long after apparent convergence.</p> <p>The three moves:</p> <ol> <li><strong>Pairing swaps</strong>: Try all <code class="language-plaintext highlighter-rouge">C(48,2)</code> = 1,128 L2 partner exchanges</li> <li><strong>Order swaps</strong>: Try all 1,128 position exchanges</li> <li><strong>Block insertions</strong>: For each of 48 blocks, remove it and try all 48 positions (2,304 evals)</li> </ol> <div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">for</span> <span class="nb">round</span> <span class="ow">in</span> <span class="nf">range</span><span class="p">(</span><span class="n">many</span><span class="p">):</span> <span class="c1"># Pairing swaps </span> <span class="k">for</span> <span class="n">i</span><span class="p">,</span> <span class="n">j</span> <span class="ow">in</span> <span class="nf">combinations</span><span class="p">(</span><span class="nf">range</span><span class="p">(</span><span class="mi">48</span><span class="p">),</span> <span class="mi">2</span><span class="p">):</span> <span class="n">swap</span> <span class="n">pairing</span><span class="p">[</span><span class="n">i</span><span class="p">],</span> <span class="n">pairing</span><span class="p">[</span><span class="n">j</span><span class="p">];</span> <span class="n">accept</span> <span class="k">if</span> <span class="n">improved</span> <span class="c1"># Order swaps </span> <span class="k">for</span> <span class="n">i</span><span class="p">,</span> <span class="n">j</span> <span class="ow">in</span> <span class="nf">combinations</span><span class="p">(</span><span class="nf">range</span><span class="p">(</span><span class="mi">48</span><span class="p">),</span> <span class="mi">2</span><span class="p">):</span> <span class="n">swap</span> <span class="n">order</span><span class="p">[</span><span class="n">i</span><span class="p">],</span> <span class="n">order</span><span class="p">[</span><span class="n">j</span><span class="p">];</span> <span class="n">accept</span> <span class="k">if</span> <span class="n">improved</span> <span class="c1"># Block insertions </span> <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nf">range</span><span class="p">(</span><span class="mi">48</span><span class="p">):</span> <span class="n">block</span> <span class="o">=</span> <span class="n">order</span><span class="p">.</span><span class="nf">pop</span><span class="p">(</span><span class="n">i</span><span class="p">)</span> <span class="k">try</span> <span class="nb">all</span> <span class="mi">48</span> <span class="n">insert</span> <span class="n">positions</span><span class="p">;</span> <span class="n">keep</span> <span class="n">best</span> </code></pre></div></div> <p>What makes this work is <strong>patience</strong> — continuing to cycle when each individual move type appears converged. The key discovery: <strong>pairing corrections trigger cascading order improvements</strong>.</p> <p>Starting from MSE 0.0098 (where standard 2-opt appeared stuck), the trajectory looked like this:</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Cycle 5: Pairing fix: 0.008274 ← corrected one L1/L2 pair ...18 order swaps... Order swap: 0.006588 ← cascade! ...7 insertions... Block insertion: 0.003861 Cycle 6: Pairing fix: 0.002379 ← biggest single improvement ...16 order swaps... Order swap: 0.000177 ← nearly there Block insertion: 0.000064 Block insertion: 0.000000 ← EXACT! </code></pre></div></div> <p>Each pairing correction fixed a block that had been paired with the wrong L2 layer. With the wrong partner, no ordering could make that block work correctly — so the optimizer was forced into a compromise. Once the pairing was fixed, a flood of previously-blocked order improvements became available.</p> <h3 id="why-insertions-matter">Why Insertions Matter</h3> <p>Insert moves find improvements that swaps cannot. A swap exchanges two elements; an insert slides one element to a new position, shifting everything in between. The final three moves to MSE = 0 were all insertions — they refined block positions with a precision that pairwise swaps couldn’t match.</p> <h3 id="error-analysis-the-tail-tells-the-story">Error Analysis: The Tail Tells the Story</h3> <p>At MSE ~0.01, analyzing the per-row error distribution was revealing:</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Percentiles of |error|: 50th: 0.026 (median row is nearly correct) 95th: 0.210 99th: 0.415 100th: 1.496 (worst row is way off) Top 100 rows: MSE 0.348 (35x more error per row) Bottom 9900: MSE 0.006 </code></pre></div></div> <p>The error was concentrated in ~45 extreme rows. This pattern — a mostly-correct solution with a few outliers — is the signature of a few specific misconfigurations rather than a globally wrong solution. It motivated continued cycling over restart.</p> <h2 id="the-full-pipeline">The Full Pipeline</h2> <p>Both paths share the same initialization and diverge at Phase 4:</p> <ol> <li><strong>First-order pairing</strong> (200 random restarts + swap optimization) → MSE ~0.7</li> <li><strong>Gumbel-Sinkhorn alternating optimization</strong> → MSE ~0.03</li> <li><strong>Standard 2-opt + insertion moves</strong> → MSE ~0.008</li> <li><strong>Either</strong>: <ul> <li><strong>(A) Combined 2-opt</strong> → MSE = 0.0 ✓ (single pass, ~7K evals)</li> <li><strong>(B) Alternating pair/order/insert cycles</strong> → MSE = 0.0 ✓ (~10 cycles, ~45 min)</li> </ul> </li> </ol> <p>Approach A is faster per pass but requires the insight to try simultaneous swaps. Approach B is slower but conceptually simpler — just keep cycling basic moves and let pairing corrections cascade into order improvements.</p> <p>Total computation: under an hour on a MacBook Pro (M-series, CPU only).</p> <h2 id="lessons-learned">Lessons Learned</h2> <p><strong>Differentiable relaxations are powerful initialization.</strong> Gumbel-Sinkhorn took us from a random permutation to within ~1% of the correct answer. Without it, local search would have no hope in a space of 10<sup>121</sup> configurations.</p> <p><strong>Pairing corrections unlock order improvements.</strong> A wrong L1/L2 pairing poisons the ordering — no arrangement of blocks can compensate for a block producing the wrong intermediate values. Each pairing fix unblocked 15-20 order improvements that had been invisible before.</p> <p><strong>Insert moves find what swaps miss.</strong> The final three moves to MSE = 0 were all block insertions. Insertions shift an entire segment of the ordering, exploring a richer neighborhood than pairwise swaps.</p> <p><strong>Cycle, don’t stop.</strong> After apparent convergence, continuing to cycle through move types found improvements for 5+ more rounds. Each round took ~90 seconds, so patience was cheap.</p> <p><strong>The right neighborhood matters more than the right algorithm.</strong> Standard 2-opt, 3-opt, simulated annealing, and coordinate descent all failed at MSE ~0.01. Both solutions came from expanding the move set — either by combining swap types (Approach A) or by adding insertions and being patient (Approach B).</p> <p><strong>Save incrementally.</strong> I learned this the hard way — a script that only saves at the end can lose hours of progress if killed. Every improving move should write to disk immediately.</p> <p><strong>Exact verification changes the game.</strong> The SHA-256 hash means only MSE = 0 is correct. This motivated exhaustive local search: even a tiny MSE improvement matters because there’s no “good enough.”</p> <h2 id="dead-ends-and-abandoned-approaches">Dead Ends and Abandoned Approaches</h2> <p>Before finding the two approaches that worked, I tried several others that didn’t pan out:</p> <p><strong>Simulated annealing.</strong> The natural response to getting stuck at a local minimum. I implemented SA with multiple move types (order swaps, pairing swaps, block insertions, segment reversals) and ran it for hundreds of thousands of steps. The problem: each evaluation requires a full sequential forward pass through 48 blocks on thousands of samples (~7ms per eval). At 500K steps, that’s nearly an hour per run — and SA needs many restarts to be effective. Worse, the high-dimensional discrete landscape (two interleaved 48-element permutations) makes it hard to set a temperature schedule that explores enough without wasting time in bad regions. The occasional improvements SA found were always things that deterministic local search could have found faster by just cycling more.</p> <p><strong>Greedy sequential construction.</strong> Rather than optimizing the ordering, build it greedily: at each step, try all remaining blocks and pick the one that minimizes the partial prediction error. This was fast (~1 second per full construction) but gave MSE ~1.8 — worse than the starting point. The problem is myopia: the block that looks best at step k might be terrible for what’s needed at steps k+1 through 47. The residual structure means early blocks fundamentally reshape the input for later blocks, so local greedy choices cascade into globally poor orderings.</p> <p><strong>3-opt (triple rotations).</strong> If 2-opt is stuck, try 3-opt — cyclic rotations of three elements. The cost is O(n³) = 17,296 triples, each tested in two rotation directions, times ~7ms per eval = ~4 minutes per sweep. I ran this on both ordering and pairing. It was too slow to iterate and never found improvements that the simpler approach (cycling 2-opt with insertions) couldn’t find faster. The 3-element moves that matter are better discovered by doing 2-opt after an insertion changes the landscape.</p> <p><strong>SiLU activation.</strong> The puzzle description says ReLU, but in first-order (non-residual) models, SiLU gives much lower MSE (~0.9 vs ~11.0). This was a red herring — SiLU only wins when you ignore the residual connections. In the full sequential model, ReLU gives MSE 0.12 while SiLU gives 4.37. The lesson: test with the full architecture, not a simplified proxy.</p> <p><strong>Group swaps.</strong> Instead of swapping individual blocks, try swapping contiguous groups of 2, 3, 4, or 8 blocks. This occasionally found tiny improvements (~0.001) but was never transformative. The blocks that need to move aren’t in contiguous groups — they’re scattered, and the real bottleneck is fixing pairings, not rearranging chunks.</p> <p><strong>Lasso/sparse selection.</strong> Precompute all 48×48 = 2,304 possible block outputs and use Lasso regression to select a sparse subset of 48. Elegant in theory, but Lasso doesn’t enforce the constraint that each L1 and L2 layer is used exactly once. Post-hoc matching from the Lasso solution didn’t produce better pairings than direct swap optimization.</p> <p><strong>Training a surrogate model, then matching layers.</strong> I trained a fresh neural network with the same architecture on the 10K dataset, hoping to match its learned layers against the puzzle pieces. The results were poor — I suspect 10K samples simply aren’t enough to recover a model similar enough to the target for layer-wise matching to work. The trained model converges to a different local minimum with different internal representations, making piece-to-layer correspondence unreliable.</p> <p><strong>Training a transformer to predict swaps.</strong> The most ambitious attempt: train a transformer model to learn which swaps improve the objective, then let it predict a sequence of moves to solve the puzzle. This ran into a bootstrapping problem — generating training data (pairs of configurations and their MSE changes) required the same expensive forward passes we were trying to avoid, and I couldn’t produce enough samples to train on. The model would need to generalize from a tiny fraction of the 10<sup>121</sup> search space, with no clear inductive bias for this specific combinatorial structure. In hindsight, domain-specific search (exploiting the residual network structure directly) was always going to beat a general-purpose learned search policy for a one-off puzzle like this.</p> <p>The common thread: <strong>the bottleneck was always pairing, not ordering.</strong> Approaches that focused on finding better orderings (SA, greedy construction, 3-opt, group swaps) couldn’t overcome wrong pairings. The approaches that worked were the ones that could fix pairings and then let order improvements cascade.</p> <hr /> <p>Good luck if you’re attempting this one — it’s a satisfying puzzle to crack.</p> <div class="footnotes" role="doc-endnotes"> <ol> <li id="fn:1"> <p>Jane Street publishes monthly puzzles at <a href="https://www.janestreet.com/puzzles/">janestreet.com/puzzles</a>. <a href="#fnref:1" class="reversefootnote" role="doc-backlink">↩</a></p> </li> </ol> </div>

2026/2/16

HRM Explained: A 27M Parameter Model That Reasons Without Chain-of-Thought

<p>What if you could build a model that solves complex Sudoku puzzles, navigates mazes, and tackles abstract reasoning — all with just 27 million parameters and 1,000 training examples? No pre-training on massive datasets, no Chain-of-Thought prompting, no language at all. That’s the claim behind the <strong>Hierarchical Reasoning Model (HRM)</strong> from Sapient Intelligence.</p> <p>In this post, I’ll walk through how HRM actually works by tracing the code and architecture step by step. I’ll also cover the important follow-up critiques that question some of these claims.</p>  <h2 id="the-big-idea">The Big Idea</h2> <p>Current LLMs reason by writing out their thinking step by step (Chain-of-Thought). This works, but it’s slow, requires huge models, and needs lots of training data. HRM takes a completely different approach: it reasons <strong>in latent space</strong> — inside the model’s hidden states — through iterative refinement.</p> <p>The core insight is borrowed from neuroscience: the human brain processes information hierarchically, with slow abstract planning and fast detailed computation happening at different timescales. HRM mimics this with two transformer modules that talk to each other.</p> <h2 id="the-two-level-architecture">The Two-Level Architecture</h2> <p>HRM has two recurrent transformer modules:</p> <p><strong>H-level (High-level planner)</strong> — 4 transformer layers, responsible for slow, abstract reasoning. Think of it as the part that asks: <em>“What strategy should I use?”</em></p> <p><strong>L-level (Low-level executor)</strong> — 4 transformer layers, responsible for fast, detailed computation. This handles: <em>“What goes in this specific cell?”</em></p> <p>They interact in a nested loop:</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>For each H-cycle (2x): For each L-cycle (2x): z_L = L_level(z_L, z_H + input_embeddings) z_H = H_level(z_H, z_L) </code></pre></div></div> <p>The L-level refines its understanding using the H-level’s guidance <strong>plus</strong> the raw input. Then the H-level updates its plan based on what L found. Both use <strong>non-causal attention</strong> — every position can see every other position simultaneously.</p> <p>One important detail: both modules are <code class="language-plaintext highlighter-rouge">ReasoningModule</code> wrappers that <strong>add</strong> the injection to the hidden state before running through their transformer layers:</p> <div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="nf">forward</span><span class="p">(</span><span class="n">self</span><span class="p">,</span> <span class="n">hidden_states</span><span class="p">,</span> <span class="n">input_injection</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">):</span> <span class="n">hidden_states</span> <span class="o">=</span> <span class="n">hidden_states</span> <span class="o">+</span> <span class="n">input_injection</span> <span class="c1"># inject </span> <span class="k">for</span> <span class="n">layer</span> <span class="ow">in</span> <span class="n">self</span><span class="p">.</span><span class="n">layers</span><span class="p">:</span> <span class="n">hidden_states</span> <span class="o">=</span> <span class="nf">layer</span><span class="p">(</span><span class="n">hidden_states</span><span class="o">=</span><span class="n">hidden_states</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">)</span> <span class="k">return</span> <span class="n">hidden_states</span> </code></pre></div></div> <p>So L doesn’t replace its state — it adds <code class="language-plaintext highlighter-rouge">z_H + input</code> to its existing state, then processes. Same for H adding <code class="language-plaintext highlighter-rouge">z_L</code>.</p> <h2 id="adaptive-computation-time-act-the-outer-loop">Adaptive Computation Time (ACT): The Outer Loop</h2> <p>The H/L cycles above describe what happens <strong>within a single step</strong>. But HRM can take <strong>multiple steps</strong>, deciding dynamically how long to think. This is the Adaptive Computation Time (ACT) wrapper.</p> <p>Each call to <code class="language-plaintext highlighter-rouge">model.forward(carry, batch)</code> is one ACT step. The training/evaluation loop calls it repeatedly:</p> <div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># Evaluation loop </span><span class="k">while</span> <span class="bp">True</span><span class="p">:</span> <span class="n">carry</span><span class="p">,</span> <span class="n">_</span><span class="p">,</span> <span class="n">metrics</span><span class="p">,</span> <span class="n">preds</span><span class="p">,</span> <span class="n">all_finish</span> <span class="o">=</span> <span class="nf">model</span><span class="p">(</span><span class="n">carry</span><span class="p">,</span> <span class="n">batch</span><span class="p">)</span> <span class="k">if</span> <span class="n">all_finish</span><span class="p">:</span> <span class="k">break</span> </code></pre></div></div> <p>The model can take up to 16 ACT steps (configurable). At each step, it decides: <strong>halt or continue?</strong></p> <p>Here’s how the two levels of looping connect:</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>ACT Step 1 ──→ H/L cycles (2x2) inside ──→ logits + Q-values │ Q says "continue" ↓ ACT Step 2 ──→ H/L cycles (2x2) inside ──→ logits + Q-values (carry from step 1 │ flows in) Q says "continue" ↓ ACT Step 3 ──→ H/L cycles (2x2) inside ──→ logits + Q-values │ Q says "HALT" ↓ Final answer used </code></pre></div></div> <p>With 16 ACT steps, each containing 2 H-cycles x 2 L-cycles, the model can perform up to <strong>64 L-passes + 32 H-passes</strong> — massive computational depth from a tiny model, because the same weights are reused every time.</p> <h2 id="z_h-and-z_l-the-models-working-memory">z_H and z_L: The Model’s Working Memory</h2> <p>So what exactly are <code class="language-plaintext highlighter-rouge">z_H</code> and <code class="language-plaintext highlighter-rouge">z_L</code>? They’re <strong>hidden state tensors</strong> — the model’s evolving “thoughts” at each level.</p> <p>Let’s make this concrete with a Sudoku example. A 9x9 puzzle gets flattened into 81 integers:</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>inputs = [5, 3, 0, 0, 7, 0, 0, 0, 0, 6, 0, 0, ...] cell1 cell2 cell3 ... cell81 </code></pre></div></div> <p>Each integer gets embedded into a 512-dimensional vector. Then a <strong>puzzle embedding</strong> (more on this later) is prepended as position 0. So the final sequence has 82 positions:</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>position 0: puzzle embedding ← 512-dim vector position 1: cell 1 embedding ← 512-dim vector position 2: cell 2 embedding ← 512-dim vector ... position 81: cell 81 embedding ← 512-dim vector </code></pre></div></div> <p>Both <code class="language-plaintext highlighter-rouge">z_H</code> and <code class="language-plaintext highlighter-rouge">z_L</code> have this same shape: <code class="language-plaintext highlighter-rouge">(batch_size, 82, 512)</code>. Each position holds a 512-dimensional vector representing the model’s current “thoughts” about that cell.</p> <p>When a sequence starts fresh, both are initialized to <strong>learned vectors</strong> — <code class="language-plaintext highlighter-rouge">H_init</code> and <code class="language-plaintext highlighter-rouge">L_init</code> — broadcast across all positions. The model starts with the same state everywhere and must differentiate through the input injection and attention.</p> <p>After each ACT step, both are <strong>detached</strong> (gradients cut) and stored in a <code class="language-plaintext highlighter-rouge">carry</code> dataclass. The next step picks up where the last left off — but no gradients flow backward between steps. This is what makes the whole thing memory-feasible.</p> <p>Position 0 is special. Since it holds the puzzle embedding (not a cell value), it acts as a <strong>global summary token</strong>. Through non-causal attention, it sees all 81 cells. The Q-head reads <code class="language-plaintext highlighter-rouge">z_H[:, 0]</code> specifically to make the halt/continue decision:</p> <div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">q_logits</span> <span class="o">=</span> <span class="n">self</span><span class="p">.</span><span class="nf">q_head</span><span class="p">(</span><span class="n">z_H</span><span class="p">[:,</span> <span class="mi">0</span><span class="p">])</span> <span class="c1"># position 0 → halt decision </span></code></pre></div></div> <p>And the final answer is read from the remaining positions:</p> <div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">output</span> <span class="o">=</span> <span class="n">self</span><span class="p">.</span><span class="nf">lm_head</span><span class="p">(</span><span class="n">z_H</span><span class="p">)[:,</span> <span class="n">puzzle_emb_len</span><span class="p">:]</span> <span class="c1"># positions 1-81 → predictions </span></code></pre></div></div> <h2 id="puzzle-embeddings-per-puzzle-identity">Puzzle Embeddings: Per-Puzzle Identity</h2> <p>Not all puzzle types need this, and the difference is revealing.</p> <p><strong>Sudoku</strong>: every puzzle follows the same rule (fill digits 1-9, no repeats in row/column/box). So <code class="language-plaintext highlighter-rouge">puzzle_identifiers = 0</code> for every example. One universal algorithm.</p> <p><strong>ARC</strong>: every puzzle has a <strong>different rule</strong>. Puzzle 42 might be “rotate the shape 90°”, puzzle 137 might be “fill enclosed regions with blue”. The model needs to know <em>which</em> puzzle it’s solving.</p> <p>For ARC, the dataset builder assigns each puzzle a unique integer ID (1 through ~960). The model has a learnable embedding table:</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>puzzle_emb: shape (961, 512) Row 0: [0, 0, ..., 0] ← blank (unused) Row 1: [0.12, -0.34, ..., 0.56] ← learned embedding for puzzle 1 Row 2: [-0.78, 0.91, ..., 0.23] ← learned embedding for puzzle 2 ... </code></pre></div></div> <p>Each embedding starts at zero and is trained via <strong>SignSGD</strong> — a simple optimizer that only uses the <strong>sign</strong> of the gradient:</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>w = w * (1 - lr * weight_decay) - lr * sign(gradient) </code></pre></div></div> <p>Every weight goes up by <code class="language-plaintext highlighter-rouge">lr</code> or down by <code class="language-plaintext highlighter-rouge">lr</code>, regardless of gradient magnitude. Why not Adam? Because puzzle embeddings are <strong>extremely sparse</strong> — with ~960 puzzles and a batch of 768, most rows get no gradient on any given step. Adam would approximate SignSGD anyway for such sparse updates, but SignSGD is simpler and needs zero optimizer state (no momentum, no second moment to track).</p> <p>The puzzle embedding is trained with a separate optimizer at 100x the learning rate of the main model (0.01 vs 0.0001) and 10x the weight decay (1.0 vs 0.1). It updates rarely, so it needs to move fast when it does.</p> <h2 id="the-q-learning-halting-mechanism">The Q-Learning Halting Mechanism</h2> <p>How does the model decide when to stop thinking? Through two Q-values produced by a tiny linear head:</p> <div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">self</span><span class="p">.</span><span class="n">q_head</span> <span class="o">=</span> <span class="nc">CastedLinear</span><span class="p">(</span><span class="n">hidden_size</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="n">bias</span><span class="o">=</span><span class="bp">True</span><span class="p">)</span> <span class="c1"># 512 → 2 numbers </span></code></pre></div></div> <p>It reads <code class="language-plaintext highlighter-rouge">z_H[:, 0]</code> (the summary token) and outputs:</p> <ul> <li><strong>q_halt</strong>: “how confident am I that my current answer is correct?”</li> <li><strong>q_continue</strong>: “how confident am I that continuing will lead to a correct answer?”</li> </ul> <p>If <code class="language-plaintext highlighter-rouge">q_halt > q_continue</code>, the model halts.</p> <h3 id="training-q_halt-supervised-from-ground-truth">Training q_halt: supervised from ground truth</h3> <div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">seq_is_correct</span> <span class="o">=</span> <span class="p">(</span><span class="n">number_of_correct_cells</span> <span class="o">==</span> <span class="n">total_cells</span><span class="p">)</span> <span class="c1"># True or False </span><span class="n">q_halt_loss</span> <span class="o">=</span> <span class="nf">binary_cross_entropy</span><span class="p">(</span><span class="n">q_halt_logits</span><span class="p">,</span> <span class="n">seq_is_correct</span><span class="p">)</span> </code></pre></div></div> <p>Simple. Did you get every cell right? Push <code class="language-plaintext highlighter-rouge">q_halt</code> toward 1. Wrong? Push toward 0.</p> <h3 id="training-q_continue-bootstrapping-from-the-future">Training q_continue: bootstrapping from the future</h3> <p>This is the trickier part. There’s no ground truth for “will continuing help?” So the model <strong>peeks ahead</strong> — it runs one more forward pass from the current carry state:</p> <div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">next_q_halt</span><span class="p">,</span> <span class="n">next_q_continue</span> <span class="o">=</span> <span class="n">self</span><span class="p">.</span><span class="nf">inner</span><span class="p">(</span><span class="n">new_inner_carry</span><span class="p">,</span> <span class="n">new_current_data</span><span class="p">)[</span><span class="o">-</span><span class="mi">1</span><span class="p">]</span> </code></pre></div></div> <p>The target for <code class="language-plaintext highlighter-rouge">q_continue</code> at step <code class="language-plaintext highlighter-rouge">t</code> is: <strong>the best outcome achievable from step <code class="language-plaintext highlighter-rouge">t+1</code> onward</strong>.</p> <div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">target</span> <span class="o">=</span> <span class="nf">sigmoid</span><span class="p">(</span> <span class="nf">where</span><span class="p">(</span><span class="n">is_last_step</span><span class="p">,</span> <span class="n">next_q_halt</span><span class="p">,</span> <span class="c1"># forced to halt next step </span> <span class="nf">max</span><span class="p">(</span><span class="n">next_q_halt</span><span class="p">,</span> <span class="n">next_q_continue</span><span class="p">)</span> <span class="c1"># best option at next step </span> <span class="p">)</span> <span class="p">)</span> </code></pre></div></div> <p>This is the Bellman equation from reinforcement learning. If at the next step, halting gives 82% confidence and continuing gives 69%, then the value of continuing now is 82% (you’d halt next step). The target follows whichever future path leads to the best outcome.</p> <h3 id="the-bootstrapping-cold-start">The bootstrapping cold start</h3> <p>At the beginning of training, both Q-values are meaningless. The Q-head is initialized with bias = -5, so <code class="language-plaintext highlighter-rouge">sigmoid(-5) ≈ 0.007</code> — the model believes there’s a 0.7% chance of being correct for everything. Since <code class="language-plaintext highlighter-rouge">q_halt ≈ q_continue</code>, nobody halts early; everything runs to the maximum 16 steps.</p> <p>The chain reaction goes:</p> <ol> <li><code class="language-plaintext highlighter-rouge">lm_loss</code> gradually teaches the model to produce correct answers</li> <li><code class="language-plaintext highlighter-rouge">q_halt</code> starts learning which answers are correct (grounded in truth)</li> <li>Once <code class="language-plaintext highlighter-rouge">q_halt</code> is meaningful at step 16, <code class="language-plaintext highlighter-rouge">q_continue</code> at step 15 gets a real target</li> <li>That propagates backward: step 14, 13, 12…</li> <li>Eventually the model learns to halt early for easy puzzles, run longer for hard ones</li> </ol> <h3 id="exploration">Exploration</h3> <p>Without exploration, the Q-head can get stuck — if it always halts at step 3, it never discovers that step 8 would give the right answer. So 10% of the time, each batch item gets a random minimum number of steps it must run before halting is allowed:</p> <div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">min_halt_steps</span> <span class="o">=</span> <span class="p">(</span><span class="nf">rand</span><span class="p">()</span> <span class="o"><</span> <span class="mf">0.1</span><span class="p">)</span> <span class="o">*</span> <span class="nf">randint</span><span class="p">(</span><span class="mi">2</span><span class="p">,</span> <span class="n">max_steps</span> <span class="o">+</span> <span class="mi">1</span><span class="p">)</span> <span class="n">halted</span> <span class="o">=</span> <span class="n">halted</span> <span class="o">&</span> <span class="p">(</span><span class="n">steps</span> <span class="o">>=</span> <span class="n">min_halt_steps</span><span class="p">)</span> </code></pre></div></div> <p>This ensures the model occasionally sees deeper computation and can update its estimates.</p> <h2 id="training-two-optimizers-one-loss">Training: Two Optimizers, One Loss</h2> <p>Each training step:</p> <ol> <li><strong>Forward pass</strong> — puzzle embeddings copied to local buffer, flow through L/H cycles, produce logits + Q-values</li> <li><strong>Single backward pass</strong> — gradients flow through everything</li> <li><strong>Two optimizers step</strong>: <ul> <li><strong>SignSGD</strong> for puzzle embeddings (lr=0.01, weight_decay=1.0)</li> <li><strong>Adam</strong> for all transformer weights (lr=0.0001, weight_decay=0.1)</li> </ul> </li> </ol> <p>The total loss combines three terms:</p> <div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">total_loss</span> <span class="o">=</span> <span class="n">lm_loss</span> <span class="o">+</span> <span class="mf">0.5</span> <span class="o">*</span> <span class="p">(</span><span class="n">q_halt_loss</span> <span class="o">+</span> <span class="n">q_continue_loss</span><span class="p">)</span> </code></pre></div></div> <p>All three losses backpropagate through the entire model. The Q-losses aren’t just training the Q-head — they shape the representations in <code class="language-plaintext highlighter-rouge">z_H</code> and <code class="language-plaintext highlighter-rouge">z_L</code> throughout, forcing the model to develop internal representations of “how solved is this puzzle.”</p> <h3 id="the-gradient-efficiency-trick">The gradient efficiency trick</h3> <p>Within each ACT step, only the <strong>final</strong> H/L cycle computes gradients. All earlier cycles run in <code class="language-plaintext highlighter-rouge">torch.no_grad()</code>:</p> <div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">with</span> <span class="n">torch</span><span class="p">.</span><span class="nf">no_grad</span><span class="p">():</span> <span class="c1"># Run H_cycles * L_cycles - 1 warmup iterations </span> <span class="k">for</span> <span class="n">H_step</span> <span class="ow">in</span> <span class="nf">range</span><span class="p">(</span><span class="n">H_cycles</span><span class="p">):</span> <span class="k">for</span> <span class="n">L_step</span> <span class="ow">in</span> <span class="nf">range</span><span class="p">(</span><span class="n">L_cycles</span><span class="p">):</span> <span class="k">if</span> <span class="ow">not</span> <span class="p">(</span><span class="n">last</span> <span class="n">H</span> <span class="ow">and</span> <span class="n">last</span> <span class="n">L</span><span class="p">):</span> <span class="n">z_L</span> <span class="o">=</span> <span class="nc">L_level</span><span class="p">(</span><span class="n">z_L</span><span class="p">,</span> <span class="n">z_H</span> <span class="o">+</span> <span class="nb">input</span><span class="p">)</span> <span class="k">if</span> <span class="ow">not</span> <span class="n">last</span> <span class="n">H</span><span class="p">:</span> <span class="n">z_H</span> <span class="o">=</span> <span class="nc">H_level</span><span class="p">(</span><span class="n">z_H</span><span class="p">,</span> <span class="n">z_L</span><span class="p">)</span> <span class="c1"># Only this final step has gradients: </span><span class="n">z_L</span> <span class="o">=</span> <span class="nc">L_level</span><span class="p">(</span><span class="n">z_L</span><span class="p">,</span> <span class="n">z_H</span> <span class="o">+</span> <span class="nb">input</span><span class="p">)</span> <span class="n">z_H</span> <span class="o">=</span> <span class="nc">H_level</span><span class="p">(</span><span class="n">z_H</span><span class="p">,</span> <span class="n">z_L</span><span class="p">)</span> </code></pre></div></div> <p>The hidden states carry forward information from the no-grad iterations, but only the final refinement contributes to the loss. This dramatically reduces memory usage.</p> <h2 id="limitations-no-branching-no-backtracking">Limitations: No Branching, No Backtracking</h2> <p>HRM’s computation is a <strong>single linear path</strong>:</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>carry → step 1 → step 2 → step 3 → ... → answer </code></pre></div></div> <p>As humans, when we solve puzzles, we do something different:</p> <ul> <li><em>“What if this cell is 5?”</em> → follow implications → contradiction → <strong>backtrack</strong></li> <li><em>“OK, what if it’s 7?”</em> → follow implications → works → keep going</li> </ul> <p>That’s tree search — branching, evaluating, backtracking. HRM can’t do this. If step 2 goes down a wrong path, step 3 builds on that wrong foundation.</p> <p>The non-causal attention can partially compensate by processing all positions simultaneously (like parallel constraint propagation rather than sequential hypothesis testing). But for tasks that fundamentally require exploring multiple hypotheses — like playing Go, where you need to simulate opponent responses many moves ahead — HRM’s single-path architecture won’t work.</p> <table> <thead> <tr> <th>Task type</th> <th>What’s needed</th> <th>HRM works?</th> </tr> </thead> <tbody> <tr> <td>Sudoku</td> <td>Constraint propagation</td> <td>Yes</td> </tr> <tr> <td>Maze</td> <td>Path finding</td> <td>Yes</td> </tr> <tr> <td>ARC</td> <td>Pattern recognition + rule inference</td> <td>Partially</td> </tr> <tr> <td>Go / Chess</td> <td>Multi-step adversarial tree search</td> <td>No</td> </tr> <tr> <td>Theorem proving</td> <td>Hypothesis testing + backtracking</td> <td>No</td> </tr> </tbody> </table> <h2 id="the-follow-up-critiques">The Follow-Up Critiques</h2> <p>Two important independent analyses appeared after HRM’s release, and they paint a different picture than the original paper.</p> <h3 id="arc-prize-team-analysis">ARC Prize Team Analysis</h3> <p>The <a href="https://arcprize.org/blog/hrm-analysis">ARC Prize team</a> verified HRM’s results and ran ablation studies. Their key findings:</p> <p><strong>The hierarchy barely matters.</strong> A regular transformer with the same parameter count came within ~5 percentage points of HRM without any hyperparameter tuning. The H/L architectural split isn’t the secret sauce.</p> <p><strong>The refinement loop is the real driver.</strong> Performance jumped +13 percentage points from zero to one refinement iteration. This is the ACT outer loop — but any recurrent architecture could benefit from iterative refinement.</p> <p><strong>Puzzle embeddings limit generalization.</strong> Since each puzzle gets a learned embedding by ID, the model can only work on puzzles it has seen during training. This makes HRM closer to “test-time training” (memorizing each puzzle’s pattern) than genuine reasoning that generalizes to novel puzzles.</p> <h3 id="ge-liao--poggio-analysis-arxiv-251000355">Ge, Liao & Poggio Analysis (arXiv 2510.00355)</h3> <p>Researchers from MIT published <a href="https://arxiv.org/abs/2510.00355">“Hierarchical Reasoning Models: Perspectives and Misconceptions”</a> with further findings:</p> <p><strong>A flat model works equally well.</strong> An 8-layer L-only model (no H module at all) achieved similar performance and trained faster (1h 48m vs 4h 21m).</p> <p><strong>The one-step gradient trick isn’t novel.</strong> The no-grad warmup + 1-step gradient pattern is mathematically equivalent to how diffusion models and Latent Consistency Models train. It’s a known technique.</p> <p><strong>ACT doesn’t help at inference.</strong> Running for the maximum number of steps always gives the best results. The learned halting policy is never actually useful — the code itself always runs to <code class="language-plaintext highlighter-rouge">halt_max_steps</code> during evaluation.</p> <p><strong>Is it even recurrent?</strong> Since only the last cycle has gradients and the carry is detached between ACT steps, the paper questions whether HRM is truly recurrent or just a very deep feedforward model.</p> <h2 id="whats-genuinely-interesting">What’s Genuinely Interesting</h2> <p>Despite the critiques, HRM points toward ideas worth taking seriously:</p> <p><strong>Latent-space reasoning works.</strong> Instead of generating tokens to “think” (Chain-of-Thought), you can reason inside hidden states. This is fundamentally faster — no autoregressive token generation — and the ARC results show it’s viable even at 27M parameters.</p> <p><strong>Iterative refinement is powerful.</strong> Running the same model multiple times with carried state is a simple idea with outsized impact. The +13pp jump from zero to one refinement iteration shows this clearly.</p> <p><strong>Small models can do complex reasoning.</strong> With the right architecture and training setup, you don’t need billions of parameters for tasks like Sudoku and maze solving. The computational depth comes from recurrence, not model size.</p> <p>The specific hierarchical architecture may not be essential, and the puzzle embeddings are a significant limitation. But the broader research direction — compact models that reason through iterative latent computation — is one worth watching.</p>

2026/2/12

BrushNet & BrushEdit Explained: From Inpainting Architecture to Intelligent Editing

<p>You’ve probably seen AI tools that can erase objects from photos and fill in the gap seamlessly. But how does the model know what to put there — and how does it figure out <em>where</em> to edit when you just say “remove the dog”? In this post, I’ll break down two papers: <strong>BrushNet</strong>, a clever architecture that adds inpainting ability to any diffusion model, and <strong>BrushEdit</strong>, an agent pipeline that wraps BrushNet with language understanding to turn natural instructions into image edits.</p>  <h2 id="part-1-brushnet--the-inpainting-engine">Part 1: BrushNet — The Inpainting Engine</h2> <h2 id="the-problem-teaching-a-model-to-fill-holes">The Problem: Teaching a Model to Fill Holes</h2> <p>Imagine you have a photo of a dog on a beach. You want to replace the dog with a sandcastle. You need a model that:</p> <ol> <li>Understands what’s around the hole (beach, sky, waves)</li> <li>Generates something new that matches (a sandcastle)</li> <li>Blends it seamlessly at the edges</li> </ol> <p>The simplest approach? Fine-tune the entire diffusion model for inpainting. But this has a big downside — you break the original model. It can’t do normal image generation anymore, and you can’t swap in a better base model later.</p> <p><strong>BrushNet’s solution:</strong> keep the original model frozen, and add a separate trainable branch alongside it.</p> <h2 id="the-two-branch-architecture">The Two-Branch Architecture</h2> <p>BrushNet runs <strong>two U-Nets in parallel</strong>:</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code> ┌─────────────────────────┐ Text prompt ──→│ Base U-Net (FROZEN) │──→ Predicted noise │ Has cross-attention │ │ to understand text │ └────────────▲────────────┘ │ + (add features) │ ┌────────────┴────────────┐ Masked image ─→│ BrushNet (TRAINABLE) │ + mask ────────→│ NO cross-attention │ + noisy latent →│ Processes spatial info │ └─────────────────────────┘ </code></pre></div></div> <p>The Base U-Net does what it always does — denoise an image guided by a text prompt. BrushNet runs alongside it, processing the mask and surrounding context, then <strong>injects hints</strong> into the Base U-Net at every layer.</p> <h2 id="what-goes-into-brushnet">What Goes Into BrushNet?</h2> <p>BrushNet takes 3 things, concatenated into a 9-channel input:</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>┌──────────────────┐ ┌──────────────────┐ ┌──────────────────┐ │ Noisy latent │ │ Masked image │ │ Binary mask │ │ (4 channels) │ │ (4 channels) │ │ (1 channel) │ │ │ │ │ │ │ │ Current state │ │ What's around │ │ Where the │ │ of denoising │ │ the hole │ │ hole is │ └──────────────────┘ └──────────────────┘ └──────────────────┘ │ │ │ └─────────────────────┴─────────────────────┘ │ Concatenate → 9 channels │ ┌─────▼─────┐ │ BrushNet │ └───────────┘ </code></pre></div></div> <h3 id="why-these-3-inputs-what-does-each-one-do">Why these 3 inputs? What does each one do?</h3> <p>Each input answers a different question:</p> <p><strong>1. Noisy latent <code class="language-plaintext highlighter-rouge">z_t</code> (4 channels) — “What step are we at?”</strong></p> <p>This is the current state of the image being denoised. At each timestep during the denoising loop, the image goes from pure noise to clean image. BrushNet needs to see this so it knows how much noise is left and can produce appropriate injection features for the current step.</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>t=T (start): z_t = pure noise → BrushNet: "everything is noisy, give strong guidance" t=T/2 (mid): z_t = half noise/half image → BrushNet: "refine the details" t=0 (end): z_t = nearly clean → BrushNet: "just fix edges" </code></pre></div></div> <p><strong>2. Masked image latent <code class="language-plaintext highlighter-rouge">z_masked</code> (4 channels) — “What’s around the hole?”</strong></p> <p>This is the original image with the masked region zeroed out, then VAE-encoded. It tells BrushNet what the surrounding context looks like — colors, textures, edges near the mask boundary.</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Original: [beach][dog][beach] Mask applied: [beach][ 0 ][beach] ← dog region zeroed out VAE encode: [4-channel latent] ← this goes to BrushNet </code></pre></div></div> <p>Why 4 channels instead of 3 (RGB)? Because the U-Net operates in VAE latent space, not pixel space. Raw pixels would be mismatched — like feeding English text into a Chinese language model. The VAE encoder translates the image into the same “language” the U-Net understands.</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Original image (512×512×3) │ Apply mask (zero out hole region) │ VAE Encoder │ Masked image latent (64×64×4) ← This goes to BrushNet </code></pre></div></div> <p><strong>3. Mask (1 channel) — “Where is the hole?”</strong></p> <p>A simple binary map: 1 = inpaint here, 0 = keep original. You might think BrushNet could figure this out from the masked image alone (just look for the zeros), but zeroed-out pixels are ambiguous:</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Without mask channel: z_masked has zeros at (2,3) → Is this black pixels or a hole? 🤷 With mask channel: z_masked has zeros at (2,3) + mask=1 at (2,3) → Definitely a hole! ✓ </code></pre></div></div> <h3 id="why-all-3-are-necessary">Why all 3 are necessary</h3> <table> <thead> <tr> <th>Without…</th> <th>Problem</th> </tr> </thead> <tbody> <tr> <td>Noisy latent</td> <td>BrushNet doesn’t know which denoising step → wrong features</td> </tr> <tr> <td>Masked image</td> <td>BrushNet can’t see surrounding context → can’t blend</td> </tr> <tr> <td>Mask</td> <td>BrushNet can’t distinguish “black pixel” from “hole”</td> </tr> </tbody> </table> <p>Each input answers a different question: <strong>when</strong> (timestep), <strong>what’s around</strong> (context), and <strong>where</strong> (hole location).</p> <h2 id="the-key-innovation-zero-convolutions">The Key Innovation: Zero Convolutions</h2> <p>Here’s the clever part. BrushNet’s features are injected into the Base U-Net through <strong>zero convolutions</strong> — 1×1 convolutions where all weights start at zero.</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>At training start: BrushNet feature ──→ ZeroConv ──→ 0.0 ──→ + Base U-Net feature (all zeros) (unchanged!) </code></pre></div></div> <p>Why? Because the Base U-Net is a carefully trained model. If you inject random noise into it on day one, you’d destroy its ability to generate images. Starting from zero means:</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Training step 0: BrushNet contributes nothing (U-Net works normally) Training step 100: BrushNet whispers tiny hints (weights: 0.001) Training step 10K: BrushNet provides real guidance (weights: 0.1) </code></pre></div></div> <h3 id="concrete-example">Concrete Example</h3> <p>Say BrushNet produces a feature value of <code class="language-plaintext highlighter-rouge">0.8</code> at some position. Here’s what the zero convolution does with it over training:</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Step 0: weight = 0.0 → 0.0 × 0.8 = 0.0 (silent) Step 1000: weight = 0.02 → 0.02 × 0.8 = 0.016 (whispering) Step 10000: weight = 0.25 → 0.25 × 0.8 = 0.2 (contributing) </code></pre></div></div> <p>It’s like slowly turning up the volume from mute. The Base U-Net is never shocked by sudden changes.</p> <h2 id="where-are-features-injected">Where Are Features Injected?</h2> <p>Unlike ControlNet (which only injects into the decoder), BrushNet injects at <strong>every single layer</strong> — all encoder blocks, the mid block, and all decoder blocks:</p> <p><img src="/images/blog/brushnet_architecture.png" alt="BrushNet Dual-Branch Architecture" /></p> <p>The left column (green) is the trainable BrushNet branch — no cross-attention to text. The right column (blue) is the frozen Base U-Net with text cross-attention. The red arrows are zero-conv injection points where BrushNet features are added element-wise to the Base U-Net.</p> <p>Each arrow is actually multiple injection points (one per sub-layer), totaling about <strong>25 injection points</strong> in total. This dense injection gives BrushNet pixel-level control, which is crucial for inpainting — you need precise boundaries where the generated content meets the original image.</p> <h2 id="why-no-cross-attention-in-brushnet">Why No Cross-Attention in BrushNet?</h2> <p>The Base U-Net has cross-attention layers that let it understand text prompts:</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Base U-Net block: ResBlock → CrossAttention("a sunflower") → output BrushNet block: ResBlock → output ↑ (removed!) </code></pre></div></div> <p>This is by design. BrushNet’s job is purely spatial — “here’s a hole, here’s what’s around it.” The text understanding stays in the Base U-Net. This separation means:</p> <ul> <li>BrushNet is <strong>smaller</strong> (~480M vs ~520M params) because it skips attention layers</li> <li>It focuses entirely on <strong>where</strong> to inpaint, not <strong>what</strong> to generate</li> <li><strong>What</strong> to generate is handled by the Base U-Net via the text prompt</li> </ul> <h2 id="how-training-works">How Training Works</h2> <p>The training loop is surprisingly simple — it uses the standard diffusion denoising loss:</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>For each training step: 1. Take a clean image "cat on a couch" 2. Generate a RANDOM mask (random shape, random position) 3. Apply mask to image (hole in it) 4. VAE-encode both z₀ (clean latent), z_masked (masked latent) 5. Add random noise to clean latent z_t = mix(z₀, noise, t) 6. Run through both branches: BrushNet(z_t, z_masked, mask) → injection features Base_UNet(z_t, text) + features → predicted noise 7. Loss = ‖ predicted_noise - actual_noise ‖² (MSE) </code></pre></div></div> <h3 id="wait--the-loss-compares-noise-not-images">Wait — the loss compares noise, not images?</h3> <p>Yes! The model predicts <strong>what noise was added</strong>, not what the clean image looks like. We know the actual noise because we added it ourselves in step 5. If the model can perfectly predict the noise, we can subtract it to recover the clean image.</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>We added noise ε to get z_t. Model predicts ε_θ. If ε_θ ≈ ε, then z₀ ≈ (z_t - ε_θ) / scale ← clean image recovered! </code></pre></div></div> <h3 id="no-special-mask-weighted-loss">No special mask-weighted loss?</h3> <p>Nope. The loss is computed over the <strong>entire</strong> image, not just the masked region. But the model naturally focuses on the mask because:</p> <ul> <li><strong>Outside the mask</strong>: the frozen Base U-Net already handles this well. BrushNet’s zero-convs learn to stay quiet here (contributing nothing reduces loss just fine).</li> <li><strong>Inside the mask</strong>: the Base U-Net struggles without context. BrushNet’s features are the only thing that helps here, so gradients push the zero-convs to output useful values.</li> </ul> <p>The mask guides learning <strong>implicitly through gradients</strong>, not explicitly through loss weighting.</p> <h3 id="training-data-just-clean-images">Training data: just clean images</h3> <p>BrushNet doesn’t need paired before/after examples. It’s self-supervised:</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Dataset: clean images + text descriptions (same data as Stable Diffusion) Masks: generated randomly during training </code></pre></div></div> <p>The model learns to reconstruct whatever was behind a random mask, using the surrounding context and text prompt. At inference, you provide a real mask of what you want to replace.</p> <h2 id="brushnet-vs-controlnet-vs-standard-inpainting">BrushNet vs. ControlNet vs. Standard Inpainting</h2> <table> <thead> <tr> <th>Feature</th> <th>SD Inpainting</th> <th>ControlNet</th> <th>BrushNet</th> </tr> </thead> <tbody> <tr> <td>Base model</td> <td>Modified (retrained)</td> <td>Frozen</td> <td>Frozen</td> </tr> <tr> <td>Branch coverage</td> <td>N/A (single model)</td> <td>Encoder only</td> <td>Full U-Net</td> </tr> <tr> <td>Injection points</td> <td>N/A</td> <td>~12 (decoder only)</td> <td>~25 (everywhere)</td> </tr> <tr> <td>Swap base models?</td> <td>No</td> <td>Yes</td> <td>Yes</td> </tr> <tr> <td>Extra params</td> <td>0</td> <td>~360M</td> <td>~480M</td> </tr> <tr> <td>Text handling</td> <td>Single model</td> <td>Branch has cross-attn</td> <td>Branch has NO cross-attn</td> </tr> <tr> <td>Best for</td> <td>General inpainting</td> <td>Structural control</td> <td>Precise inpainting</td> </tr> </tbody> </table> <h3 id="why-full-u-net-matters-for-inpainting">Why full U-Net matters for inpainting</h3> <p>ControlNet copies only the encoder half — it injects features into the decoder via the skip connections. This works well for structural guidance (edges, poses) but not for inpainting, where you need fine-grained control at every spatial resolution.</p> <p>The BrushNet paper showed this clearly:</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Full U-Net (BrushNet): PSNR 19.86 ← best quality Half U-Net: PSNR 19.01 ControlNet-style: PSNR 18.28 ← worst quality </code></pre></div></div> <p>Inpainting needs dense per-pixel control, especially at mask boundaries where generated content must blend seamlessly with the original image.</p> <h2 id="inference-putting-it-all-together">Inference: Putting It All Together</h2> <p>At inference time, the full pipeline looks like this:</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>1. User provides: image + mask + text prompt ("a sunflower") 2. Encode: masked_image = apply_mask(image, mask) z_masked = VAE_encode(masked_image) [4, 64, 64] mask_small = downsample(mask) [1, 64, 64] 3. Start from pure noise: z_T ~ N(0, I) [4, 64, 64] 4. Denoise loop (T steps, e.g. 25-50): for t in T → 0: brushnet_feats = BrushNet(z_t, z_masked, mask_small, t) noise_pred = BaseUNet(z_t, t, "a sunflower") + brushnet_feats z_{t-1} = scheduler_step(z_t, noise_pred) 5. Decode final latent: result = VAE_decode(z_0) [3, 512, 512] 6. Blend: output = blur_blend(result, original_image, mask) </code></pre></div></div> <p>The final blending step uses a Gaussian-blurred mask to smooth the transition between generated and original pixels, avoiding hard edges.</p> <h2 id="the-plug-and-play-promise">The Plug-and-Play Promise</h2> <p>Because the Base U-Net is never modified, you can:</p> <ul> <li>Train one BrushNet and use it with <strong>any</strong> compatible base model</li> <li>Swap in a photorealistic model, an anime model, or a custom fine-tune</li> <li>The base model keeps all its original capabilities (text-to-image still works)</li> <li>Adjust the <code class="language-plaintext highlighter-rouge">conditioning_scale</code> (0.0 to 1.0) to control how much BrushNet influences the output</li> </ul> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>scale = 0.0 → Base U-Net only (no inpainting guidance) scale = 0.5 → Gentle inpainting hints scale = 1.0 → Full BrushNet influence (default) </code></pre></div></div> <h2 id="model-size">Model Size</h2> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Base U-Net (frozen): ~520M params BrushNet (trainable): ~480M params └─ Zero-conv layers: 25 layers, ~20M params Total at inference: ~1,000M params (1B) </code></pre></div></div> <p>BrushNet is nearly the same size as the Base U-Net — the only difference is removing cross-attention layers (~40M params saved). The trade-off is clear: <strong>2x memory for plug-and-play flexibility</strong>.</p> <h2 id="brushnet-summary">BrushNet Summary</h2> <p>BrushNet gives us a powerful inpainting engine. But using it requires you to provide two things manually: a <strong>mask</strong> (where to edit) and a <strong>text prompt</strong> (what to generate). For simple cases that’s fine — draw a circle around the dog, type “a sunflower.”</p> <p>But what if you just want to say <strong>“remove the dog”</strong> and have the system figure out the rest?</p> <p>That’s exactly what BrushEdit does. It wraps BrushNet in an intelligent agent pipeline that automates the mask and prompt generation.</p> <hr /> <h2 id="part-2-brushedit--from-remove-the-dog-to-edited-image">Part 2: BrushEdit — From “Remove the Dog” to Edited Image</h2> <p>BrushEdit (arXiv 2412.10316) doesn’t change BrushNet’s architecture at all. Instead, it asks: <strong>how do you go from a natural language instruction to a BrushNet-ready mask and prompt?</strong></p> <p>The answer is an assembly line of 4 AI models:</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>User: "Remove the dog from the garden" │ ▼ ┌───────────────────────────┐ │ 1. MLLM (Qwen2-VL) │ "What kind of edit? What object?" │ Classify + Identify │ → edit_type = "remove" │ + Generate caption │ → target = "dog" └────────────┬──────────────┘ → caption = "garden with flowers" ▼ ┌───────────────────────────┐ │ 2. GroundingDINO │ "Where is the dog?" │ Text → bounding box │ → bbox around the dog └────────────┬──────────────┘ ▼ ┌───────────────────────────┐ │ 3. SAM │ "What's the exact shape?" │ Bbox → pixel mask │ → silhouette of the dog └────────────┬──────────────┘ ▼ ┌───────────────────────────┐ │ 4. BrushNet + SD 1.5 │ "Fill the hole" │ Mask + caption → image │ → dog replaced with garden └───────────────────────────┘ </code></pre></div></div> <p>Each model does one thing well. Let’s walk through each step.</p> <h2 id="step-1-the-mllm-understands-your-instruction">Step 1: The MLLM Understands Your Instruction</h2> <p>The MLLM (a vision-language model like Qwen2-VL or GPT-4o) is called <strong>three separate times</strong>, each with a different question. No fine-tuning — it’s used purely through prompt engineering.</p> <h3 id="call-1-what-kind-of-edit">Call 1: “What kind of edit?”</h3> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>System: "Classify this editing instruction into one of: addition, remove, local, global, background. Reply with a single word." User: "Remove the dog from the garden" → "remove" </code></pre></div></div> <p>This classification matters because each edit type needs a <strong>different mask strategy</strong>:</p> <table> <thead> <tr> <th>Edit Type</th> <th>What Happens to the Mask</th> </tr> </thead> <tbody> <tr> <td><strong>Remove</strong> “Remove the dog”</td> <td>Detect dog → segment it → dilate mask edges</td> </tr> <tr> <td><strong>Addition</strong> “Add a cat on the sofa”</td> <td>No detection needed — MLLM predicts a bounding box</td> </tr> <tr> <td><strong>Local</strong> “Make the car blue”</td> <td>Detect car → segment it → use mask as-is</td> </tr> <tr> <td><strong>Background</strong> “Change to a beach”</td> <td>Detect foreground → segment → <strong>invert</strong> the mask</td> </tr> <tr> <td><strong>Global</strong> “Make it nighttime”</td> <td>Mask the entire image</td> </tr> </tbody> </table> <h3 id="call-2-what-object">Call 2: “What object?”</h3> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>System: "Identify the main object being edited. Reply with no more than 5 words, a single noun phrase." User: "Remove the dog from the garden" → "dog" </code></pre></div></div> <p>This short phrase will be fed to GroundingDINO as a search query. It needs to be concise — just enough to find the right thing in the image.</p> <h3 id="call-3-what-should-the-result-look-like">Call 3: “What should the result look like?”</h3> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>System: "Describe what the image should look like AFTER the edit. Do NOT include elements that are removed or changed." User: [source image] + "Remove the dog from the garden" → "A peaceful garden path with green grass and flowers" </code></pre></div></div> <p>This becomes the text prompt for BrushNet’s inpainting. Notice: it describes the scene <strong>without</strong> the dog — because we’re removing it. The MLLM has to understand the instruction well enough to describe the <em>result</em>, not just parrot the input.</p> <h3 id="why-training-free-works-here">Why training-free works here</h3> <p>All three calls use the MLLM <strong>off-the-shelf</strong>. No fine-tuning. This means you can swap backends freely:</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>GPT-4o → Best quality, requires API key, costs money Qwen2-VL → Best open-source, runs locally, ~16 GB VRAM LLaVA → Lighter alternative, ~17 GB VRAM </code></pre></div></div> <p>The paper doesn’t fine-tune any of these models. It just writes good prompts. This is a deliberate design choice — it keeps the system modular and easy to upgrade as better VLMs come out.</p> <h2 id="step-2-groundingdino-finds-the-object">Step 2: GroundingDINO Finds the Object</h2> <p>Now we know we’re looking for “dog.” But where in the image is it?</p> <p>GroundingDINO is an open-vocabulary object detector. Unlike traditional detectors that only recognize a fixed set of classes (like COCO’s 80 categories), it takes <strong>any text query</strong> and finds matching objects:</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Input: image + "dog" Output: bounding box (128, 128, 384, 384), confidence 0.89 </code></pre></div></div> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>┌────────────────────────┐ │ │ │ ┌──────────┐ │ │ │ │ │ │ │ dog │ │ │ │ │ │ │ └──────────┘ │ │ ↑ │ │ bounding box │ │ from DINO │ └────────────────────────┘ </code></pre></div></div> <p>This works for any object you can describe in words. “Red car,” “wooden table,” “person in blue shirt” — GroundingDINO handles them all.</p> <p><strong>Exception: addition edits.</strong> If the instruction is “add a cat on the sofa,” there’s no cat to detect yet. In this case, GroundingDINO is skipped entirely. Instead, the MLLM predicts where the new object should go by outputting a bounding box: “given this 512×512 image, the cat should go at [256, 170, 128, 170].”</p> <h2 id="step-3-sam-cuts-the-exact-shape">Step 3: SAM Cuts the Exact Shape</h2> <p>A bounding box is too rough. The box around the dog also includes chunks of grass, maybe a bit of fence. We need the exact silhouette.</p> <p>SAM (Segment Anything Model) takes the bounding box and produces a pixel-precise mask:</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Before (bounding box): After (SAM mask): ┌────────────────────────┐ ┌────────────────────────┐ │ │ │ │ │ ┌──────────┐ │ │ ████████ │ │ │ grass │ │ │ ████████████ │ │ │ dog │ │ │ ██████████ │ │ │ grass │ │ │ ██████ │ │ └──────────┘ │ │ ██ │ │ │ │ │ └────────────────────────┘ └────────────────────────┘ Box includes background Mask follows the dog's around the dog exact silhouette </code></pre></div></div> <h3 id="edit-type-specific-mask-adjustments">Edit-type-specific mask adjustments</h3> <p>After SAM produces the mask, BrushEdit adjusts it based on the edit type:</p> <ul> <li><strong>Remove:</strong> <strong>Dilate</strong> the mask by a few pixels. Fur, hair, and shadows often extend slightly beyond the segmentation boundary. Expanding the mask catches these fuzzy edges.</li> <li><strong>Background:</strong> <strong>Invert</strong> the mask. Instead of masking the dog, mask everything <em>except</em> the dog. Now BrushNet will regenerate the entire background while keeping the dog untouched.</li> <li><strong>Local:</strong> Use the mask as-is. The object is being modified, so we need to cover exactly that region.</li> </ul> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Remove (dilated): Background (inverted): ┌────────────────────────┐ ┌────────────────────────┐ │ │ │████████████████████████│ │ ██████████ │ │████ ████████│ │ ██████████████ │ │██ ██████│ │ ████████████ │ │████ ████████│ │ ████████ │ │██████ ██████████│ │ ████ │ │████████████████████████│ │ │ │████████████████████████│ └────────────────────────┘ └────────────────────────┘ Expanded to catch fur/shadow Everything EXCEPT the dog </code></pre></div></div> <h2 id="step-4-brushnet-fills-the-hole">Step 4: BrushNet Fills the Hole</h2> <p>Now we have everything BrushNet needs:</p> <table> <thead> <tr> <th>Input</th> <th>Value</th> </tr> </thead> <tbody> <tr> <td><strong>Mask</strong></td> <td>Pixel-precise segmentation from SAM (dilated for removal)</td> </tr> <tr> <td><strong>Caption</strong></td> <td>“A peaceful garden path with green grass and flowers”</td> </tr> <tr> <td><strong>Original image</strong></td> <td>The source photo</td> </tr> </tbody> </table> <p>This is the exact same BrushNet pipeline we covered in Part 1:</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>1. masked_image = original × (1 - mask) ← zero out the dog region 2. z_masked = VAE.encode(masked_image) ← encode to latent space 3. conditioning = concat(z_masked, mask) ← 5-channel conditioning 4. Denoising loop (50 steps): BrushNet features = BrushNet(z_t, conditioning) noise_pred = Base_UNet(z_t, "garden with flowers") + BrushNet features z_{t-1} = scheduler.step(z_t, noise_pred) 5. result = VAE.decode(z_0) ← back to pixel space 6. output = blur(mask) × result + (1-blur(mask)) × original ← blend </code></pre></div></div> <p>The blurred mask blending at the end creates a smooth transition at the boundary. Without it, you’d see a hard edge where the generated content meets the original image. This single step accounts for a +10 PSNR improvement in ablation studies.</p> <h2 id="the-full-pipeline-end-to-end">The Full Pipeline, End to End</h2> <p>Let’s trace through one more example to make sure it’s clear. Instruction: <strong>“Change the background to a tropical beach.”</strong></p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Step 1: MLLM classifies → "background" MLLM identifies → "person" (the foreground object to keep) MLLM captions → "A person standing on a tropical beach with palm trees and turquoise water" Step 2: GroundingDINO("person") → bounding box around the person Step 3: SAM(bbox) → pixel mask of the person Mask is INVERTED → now covers everything EXCEPT the person Coverage: ~75% of the image Step 4: BrushNet inpaints the masked region (the background) using caption "tropical beach with palm trees" Person is preserved in the unmasked region Blended at edges for seamless transition </code></pre></div></div> <p>The key insight for background edits: GroundingDINO detects the <strong>foreground</strong> object (the person), SAM segments it, then the mask is <strong>inverted</strong>. BrushNet never touches the person — it only regenerates the background.</p> <h2 id="why-decompose-instead-of-end-to-end">Why Decompose Instead of End-to-End?</h2> <p>You might wonder: why not train one big model that takes “remove the dog” and directly outputs an edited image? That’s what InstructPix2Pix does. BrushEdit’s decomposed approach has three advantages:</p> <p><strong>1. Transparency.</strong> Every intermediate result is visible. You can see the edit classification (“remove”), the detected object (“dog”), the mask, and the caption. If something goes wrong, you know exactly where.</p> <p><strong>2. User control.</strong> You can override any step. Don’t like the auto-generated mask? Draw your own. Want a different caption? Type one. The pipeline doesn’t force you into a black box.</p> <p><strong>3. No paired training data.</strong> InstructPix2Pix needs millions of (instruction, before, after) triples — expensive to create. BrushEdit needs none. The MLLM is used off-the-shelf, GroundingDINO and SAM are pre-trained, and BrushNet trains on standard images with random masks.</p> <p>The trade-off is complexity. BrushEdit orchestrates 4 separate models totaling ~66 GB of weights. But each model is best-in-class at its job, and you can upgrade any component independently.</p> <h2 id="how-does-it-compare">How Does It Compare?</h2> <h3 id="vs-inversion-based-methods-ddimp2p-null-text">vs. Inversion-based methods (DDIM+P2P, Null-Text)</h3> <p>These methods invert the image to noise, then re-denoise with edits. BrushEdit skips inversion entirely — it generates directly in the masked region.</p> <table> <thead> <tr> <th>Method</th> <th>PSNR (quality)</th> <th>Time</th> </tr> </thead> <tbody> <tr> <td>DDIM + P2P</td> <td>22.67</td> <td>11s</td> </tr> <tr> <td>Null-Text + P2P</td> <td>26.52</td> <td>148s</td> </tr> <tr> <td><strong>BrushEdit</strong></td> <td><strong>32.16</strong></td> <td><strong>3.6s</strong></td> </tr> </tbody> </table> <p>5 PSNR better and 3-40x faster.</p> <h3 id="vs-original-brushnet">vs. Original BrushNet</h3> <p>BrushEdit uses BrushNet internally, but improves on it:</p> <table> <thead> <tr> <th> </th> <th>BrushNet</th> <th>BrushEdit</th> </tr> </thead> <tbody> <tr> <td>Mask generation</td> <td>Manual</td> <td>Automatic (MLLM + DINO + SAM)</td> </tr> <tr> <td>Caption</td> <td>Manual</td> <td>Automatic (MLLM)</td> </tr> <tr> <td>Model checkpoints</td> <td>2 separate (seg masks, random masks)</td> <td>1 unified model</td> </tr> <tr> <td>Object removal</td> <td>Limited</td> <td>Trained explicitly with removal data</td> </tr> <tr> <td>Multi-round editing</td> <td>No</td> <td>Yes (output becomes next input)</td> </tr> </tbody> </table> <p>The unified model comes from training on <strong>BrushData-v2</strong> — a merged dataset that combines segmentation masks and random masks, plus new removal training pairs where clean-background images are paired with random masks.</p> <h2 id="brushedits-limitations">BrushEdit’s Limitations</h2> <p>No system is perfect. BrushEdit struggles with:</p> <p><strong>Irregular masks.</strong> Very thin, fragmented, or oddly shaped masks can produce artifacts. The model was trained mostly on blob-like masks and object silhouettes.</p> <p><strong>Text-mask misalignment.</strong> If the caption says “a large elephant” but the mask is tiny, the model can’t fit an elephant in there. The MLLM doesn’t always reason well about spatial constraints.</p> <p><strong>Base model ceiling.</strong> BrushEdit uses Stable Diffusion 1.5 as its backbone. Output quality is bounded by what SD 1.5 can generate. It can’t produce FLUX-quality images because the underlying diffusion model isn’t that capable.</p> <p><strong>VLM errors cascade.</strong> If the MLLM misclassifies the edit type (calling a “remove” a “local edit”), the entire downstream pipeline produces wrong results. There’s no error recovery between steps.</p> <h2 id="key-takeaways">Key Takeaways</h2> <p><strong>BrushNet</strong> (Part 1):</p> <ol> <li><strong>Dual-branch design</strong>: Frozen base model + trainable BrushNet branch. Plug-and-play.</li> <li><strong>9-channel input</strong>: Noisy latent (4) + masked image latent (4) + mask (1).</li> <li><strong>Zero convolutions</strong>: Start silent, gradually learn. Stable training.</li> <li><strong>Full U-Net coverage</strong>: Encoder + mid + decoder injection. Not just the encoder (ControlNet-style).</li> <li><strong>No cross-attention in BrushNet</strong>: Text stays in the Base U-Net. BrushNet handles spatial information only.</li> </ol> <p><strong>BrushEdit</strong> (Part 2):</p> <ol> <li><strong>4-model assembly line</strong>: MLLM → GroundingDINO → SAM → BrushNet. Each model does one job well.</li> <li><strong>Training-free VLM</strong>: The MLLM is used off-the-shelf through prompt engineering. No fine-tuning. Swap backends freely.</li> <li><strong>Edit-type-aware masks</strong>: Different edit types get different mask treatments (dilated for removal, inverted for background, bbox for addition).</li> <li><strong>Transparent pipeline</strong>: Every intermediate result is visible and overridable by the user.</li> <li><strong>Unified inpainting model</strong>: One BrushNet checkpoint handles all mask types, trained on BrushData-v2.</li> </ol> <p>The two papers together tell a clean story: BrushNet solves <strong>how to inpaint</strong> (the architecture), and BrushEdit solves <strong>what to inpaint</strong> (the intelligence layer that turns natural language into masks and captions).</p> <hr /> <p><em>This post covers BrushNet (ECCV 2024) and BrushEdit (arXiv 2412.10316). The architecture diagrams come from hands-on experimentation and code analysis of the <a href="https://github.com/TencentARC/BrushEdit">TencentARC/BrushEdit</a> repository.</em></p>

2026/2/7

U-Net Explained: A Visual Guide for Beginners

<p>If you’ve explored image generation, segmentation, or diffusion models, you’ve probably heard of U-Net. But what exactly is it, and why is it so widely used? In this post, I’ll break down U-Net step by step with concrete examples and visual diagrams.</p>  <h2 id="what-is-u-net">What is U-Net?</h2> <p>U-Net is a neural network architecture designed for tasks where you need an <strong>image in</strong> and an <strong>image out</strong> of the same size. It was originally created for medical image segmentation in 2015, but has since become the backbone of many modern AI systems, including Stable Diffusion.</p> <p>The name comes from its shape—when you draw the architecture, it looks like the letter “U”:</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Input Image │ ▼ ┌─────────────────────────────────────────┐ │ ENCODER (Downsampling) │ │ ┌─────┐ ┌─────┐ ┌─────┐ │ │ │64ch │ → │128ch│ → │256ch│ → ... │ │ │128² │ │64² │ │32² │ │ │ └──┬──┘ └──┬──┘ └──┬──┘ │ │ │ skip │ skip │ skip │ │ ▼ ▼ ▼ │ │ ┌──┴──┐ ┌──┴──┐ ┌──┴──┐ │ │ │64ch │ ← │128ch│ ← │256ch│ ← ... │ │ │128² │ │64² │ │32² │ │ │ └─────┘ └─────┘ └─────┘ │ │ DECODER (Upsampling) │ └─────────────────────────────────────────┘ │ ▼ Output Image </code></pre></div></div> <h2 id="the-three-key-parts">The Three Key Parts</h2> <h3 id="1-encoder-the-down-path">1. Encoder (The Down Path)</h3> <p>The encoder compresses the image, making it spatially smaller but with more channels:</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>128×128×3 → 64×64×64 → 32×32×128 → 16×16×256 → 8×8×512 │ │ │ │ │ └──────────────┴─────────────┴─────────────┴────────────┘ Shrinking spatially Growing in channels </code></pre></div></div> <p>At each step:</p> <ul> <li><strong>Spatial size halves</strong> (128 → 64 → 32 → 16 → 8)</li> <li><strong>Channels increase</strong> (3 → 64 → 128 → 256 → 512)</li> </ul> <p>This is like summarizing a book—you lose details but capture the main ideas.</p> <h3 id="2-bottleneck">2. Bottleneck</h3> <p>The bottleneck is the smallest point in the network:</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>┌─────────────────────────────────┐ │ 8×8×512 │ │ │ │ Only 64 spatial positions │ │ but 512 features each │ │ │ │ "Compressed understanding" │ └─────────────────────────────────┘ </code></pre></div></div> <p>At this point, the network has maximum semantic understanding but minimum spatial detail. It knows “what” is in the image but has lost “where” things are precisely.</p> <h3 id="3-decoder-the-up-path">3. Decoder (The Up Path)</h3> <p>The decoder expands the image back to full resolution:</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>8×8×512 → 16×16×256 → 32×32×128 → 64×64×64 → 128×128×3 </code></pre></div></div> <p>But here’s the problem: how do you recover the spatial details that were lost?</p> <h2 id="the-secret-sauce-skip-connections">The Secret Sauce: Skip Connections</h2> <p>This is what makes U-Net special. Skip connections pass information directly from the encoder to the decoder, bypassing the bottleneck:</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>ENCODER DECODER ─────── ─────── 128×128 ─────── skip1 ─────────────→ 128×128 │ ▲ 64×64 ───────── skip2 ───────────→ 64×64 │ ▲ 32×32 ───────── skip3 ─────────→ 32×32 │ ▲ 16×16 ───────── skip4 ───────→ 16×16 │ ▲ └──→ 8×8 BOTTLENECK ──────────────────┘ </code></pre></div></div> <h3 id="why-are-skip-connections-needed">Why Are Skip Connections Needed?</h3> <p>Think of it this way:</p> <table> <thead> <tr> <th>Source</th> <th>Knows</th> <th>Problem</th> </tr> </thead> <tbody> <tr> <td>Bottleneck</td> <td>“What” is in image</td> <td>Lost “where” exactly</td> </tr> <tr> <td>Skip</td> <td>“Where” things are</td> <td>Doesn’t know context</td> </tr> <tr> <td><strong>Combined</strong></td> <td><strong>Both!</strong></td> <td><strong>Sharp + accurate output</strong></td> </tr> </tbody> </table> <h3 id="visual-example">Visual Example</h3> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>WITHOUT skip connections: WITH skip connections: ┌────────────────────┐ ┌────────────────────┐ │ │ │ ● │ │ ◯ │ │ ╲ │ │ (blurry, │ │ ╲ │ │ wrong spot) │ │ ● (sharp, │ │ │ │ ╲ correct!) │ │ │ │ ● │ └────────────────────┘ └────────────────────┘ </code></pre></div></div> <p>The bottleneck knows “there’s a line somewhere” but lost the exact position. The skip connection says “the line edge is at these exact pixels.” Combined, you get a sharp, accurate output.</p> <h2 id="the-building-blocks">The Building Blocks</h2> <h3 id="convblock-the-basic-unit">ConvBlock: The Basic Unit</h3> <p>Every level of the U-Net uses convolutional blocks:</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Input ↓ Conv 3×3 → BatchNorm → ReLU ↓ Conv 3×3 → BatchNorm → ReLU ↓ Output </code></pre></div></div> <p>A 3×3 convolution looks at a pixel and its 8 neighbors to compute each output pixel.</p> <h3 id="understanding-conv2d">Understanding Conv2d</h3> <p>Let’s make this concrete with <code class="language-plaintext highlighter-rouge">Conv2d(2, 3, 3)</code> — 2 input channels, 3 output channels, 3×3 kernel.</p> <p><strong>Key insight:</strong> Each output channel has its own filter, and each filter looks at ALL input channels.</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>INPUT (2 channels) OUTPUT (3 channels) ┌─────────┐ ┌─────────┐ │ Ch 0 │──┬─ Filter 0 ─────→│ Ch 0 │ │ │ │ └─────────┘ └─────────┘ │ ├─ Filter 1 ─────→┌─────────┐ ┌─────────┐ │ │ Ch 1 │ │ Ch 1 │──┤ └─────────┘ │ │ │ └─────────┘ └─ Filter 2 ─────→┌─────────┐ │ Ch 2 │ └─────────┘ </code></pre></div></div> <p>Each filter reads ALL input channels to produce ONE output channel.</p> <h3 id="concrete-conv2d-example">Concrete Conv2d Example</h3> <p>Input (2 channels, 4×4 each):</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Channel 0: Channel 1: ┌────┬────┬────┬────┐ ┌────┬────┬────┬────┐ │ 10 │ 10 │ 0 │ 0 │ │ 5 │ 5 │ 5 │ 5 │ ├────┼────┼────┼────┤ ├────┼────┼────┼────┤ │ 10 │ 10 │ 0 │ 0 │ │ 5 │ 5 │ 5 │ 5 │ ├────┼────┼────┼────┤ ├────┼────┼────┼────┤ │ 10 │ 10 │ 0 │ 0 │ │ 5 │ 5 │ 5 │ 5 │ ├────┼────┼────┼────┤ ├────┼────┼────┼────┤ │ 10 │ 10 │ 0 │ 0 │ │ 5 │ 5 │ 5 │ 5 │ └────┴────┴────┴────┘ └────┴────┴────┴────┘ </code></pre></div></div> <p>Filter 0 (one 3×3 kernel per input channel):</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>For input ch0: For input ch1: ┌────┬────┬────┐ ┌────┬────┬────┐ │ 1 │ 0 │ -1 │ │ 0 │ 0 │ 0 │ ├────┼────┼────┤ ├────┼────┼────┤ │ 1 │ 0 │ -1 │ │ 0 │ 1 │ 0 │ ├────┼────┼────┤ ├────┼────┼────┤ │ 1 │ 0 │ -1 │ │ 0 │ 0 │ 0 │ └────┴────┴────┘ └────┴────┴────┘ </code></pre></div></div> <p>To compute output pixel at (row=1, col=1):</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>From ch0: 10×1 + 10×0 + 0×(-1) + 10×1 + 10×0 + 0×(-1) + 10×1 + 10×0 + 0×(-1) = 30 From ch1: 5×0 + 5×0 + 5×0 + 5×0 + 5×1 + 5×0 + 5×0 + 5×0 + 5×0 = 5 Total: 30 + 5 + bias = 35 </code></pre></div></div> <h3 id="downblock-encoder-step">DownBlock (Encoder Step)</h3> <div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="nf">forward</span><span class="p">(</span><span class="n">self</span><span class="p">,</span> <span class="n">x</span><span class="p">):</span> <span class="n">features</span> <span class="o">=</span> <span class="n">self</span><span class="p">.</span><span class="nf">conv</span><span class="p">(</span><span class="n">x</span><span class="p">)</span> <span class="c1"># Process with ConvBlock </span> <span class="n">pooled</span> <span class="o">=</span> <span class="n">self</span><span class="p">.</span><span class="nf">pool</span><span class="p">(</span><span class="n">features</span><span class="p">)</span> <span class="c1"># Shrink by half </span> <span class="k">return</span> <span class="n">pooled</span><span class="p">,</span> <span class="n">features</span> <span class="c1"># Return BOTH! </span></code></pre></div></div> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Input: (1, 64, 64, 64) │ ConvBlock │ (1, 128, 64, 64) ──→ SAVED as skip connection │ MaxPool2d (shrink) │ Output: (1, 128, 32, 32) </code></pre></div></div> <p>The key: it returns TWO things — the pooled result for the next layer AND the features for the skip connection.</p> <h3 id="upblock-decoder-step">UpBlock (Decoder Step)</h3> <div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="nf">forward</span><span class="p">(</span><span class="n">self</span><span class="p">,</span> <span class="n">x</span><span class="p">,</span> <span class="n">skip</span><span class="p">):</span> <span class="n">x</span> <span class="o">=</span> <span class="n">self</span><span class="p">.</span><span class="nf">up</span><span class="p">(</span><span class="n">x</span><span class="p">)</span> <span class="c1"># Grow spatially (ConvTranspose2d) </span> <span class="n">x</span> <span class="o">=</span> <span class="n">torch</span><span class="p">.</span><span class="nf">cat</span><span class="p">([</span><span class="n">x</span><span class="p">,</span> <span class="n">skip</span><span class="p">],</span> <span class="n">dim</span><span class="o">=</span><span class="mi">1</span><span class="p">)</span> <span class="c1"># Concatenate with skip </span> <span class="n">x</span> <span class="o">=</span> <span class="n">self</span><span class="p">.</span><span class="nf">conv</span><span class="p">(</span><span class="n">x</span><span class="p">)</span> <span class="c1"># Process combined features </span> <span class="k">return</span> <span class="n">x</span> </code></pre></div></div> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Input: (1, 512, 8, 8) Skip: (1, 512, 16, 16) │ ConvTranspose2d (grow 2×) │ (1, 512, 16, 16) │ Concat with skip (channels add) │ (1, 1024, 16, 16) │ ConvBlock (reduce channels) │ Output: (1, 256, 16, 16) </code></pre></div></div> <h3 id="convtranspose2d-growing-images">ConvTranspose2d: Growing Images</h3> <p>ConvTranspose2d is the opposite of Conv2d — it makes images bigger:</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Conv2d (stride=2): ConvTranspose2d (stride=2): 4×4 → 2×2 2×2 → 4×4 (shrink) (grow) </code></pre></div></div> <p>Each input pixel becomes a 2×2 region:</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Input (2×2): Output (4×4): ┌───┬───┐ ┌───┬───┬───┬───┐ │ 1 │ 2 │ │ 1 │ 1 │ 2 │ 2 │ ├───┼───┤ → ├───┼───┼───┼───┤ │ 3 │ 4 │ │ 1 │ 1 │ 2 │ 2 │ └───┴───┘ ├───┼───┼───┼───┤ │ 3 │ 3 │ 4 │ 4 │ ├───┼───┼───┼───┤ │ 3 │ 3 │ 4 │ 4 │ └───┴───┴───┴───┘ </code></pre></div></div> <h2 id="complete-data-flow">Complete Data Flow</h2> <p>Let’s trace through an entire U-Net forward pass:</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>INPUT: (1, 3, 128, 128) "RGB image" ENCODER: enc1: (1, 64, 64, 64) → skip1 saved enc2: (1, 128, 32, 32) → skip2 saved enc3: (1, 256, 16, 16) → skip3 saved enc4: (1, 512, 8, 8) → skip4 saved BOTTLENECK: (1, 512, 8, 8) "Compressed understanding" DECODER: dec4: (1, 256, 16, 16) ← uses skip4 dec3: (1, 128, 32, 32) ← uses skip3 dec2: (1, 64, 64, 64) ← uses skip2 dec1: (1, 64, 128, 128) ← uses skip1 OUTPUT: (1, 3, 128, 128) "Processed image" </code></pre></div></div> <h2 id="what-can-u-net-do">What Can U-Net Do?</h2> <p>U-Net is used for any task requiring pixel-level output:</p> <table> <thead> <tr> <th>Task</th> <th>Input</th> <th>Output</th> </tr> </thead> <tbody> <tr> <td><strong>Medical segmentation</strong></td> <td>CT scan</td> <td>Tumor mask</td> </tr> <tr> <td><strong>Semantic segmentation</strong></td> <td>Photo</td> <td>Labels per pixel</td> </tr> <tr> <td><strong>Image denoising</strong></td> <td>Noisy image</td> <td>Clean image</td> </tr> <tr> <td><strong>Inpainting</strong></td> <td>Image with hole</td> <td>Filled image</td> </tr> <tr> <td><strong>Super resolution</strong></td> <td>Low-res</td> <td>High-res</td> </tr> <tr> <td><strong>Style transfer</strong></td> <td>Photo</td> <td>Stylized image</td> </tr> <tr> <td><strong>Diffusion models</strong></td> <td>Noisy latent</td> <td>Denoised latent</td> </tr> </tbody> </table> <h2 id="when-not-to-use-decoder">When NOT to Use Decoder</h2> <p>Not all tasks need a decoder:</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Classification (no decoder): Image → [shrink, shrink, shrink] → "This is a cat" U-Net (full decoder): Image → [shrink] → [expand] → Processed image </code></pre></div></div> <p>If you only need a label, not a pixel-by-pixel output, skip the decoder.</p> <h2 id="summary">Summary</h2> <p>U-Net’s power comes from three key ideas:</p> <ol> <li><strong>Encoder</strong>: Compress spatially, extract “what” is in the image</li> <li><strong>Decoder</strong>: Expand back to full resolution</li> <li><strong>Skip connections</strong>: Pass “where” information directly from encoder to decoder</li> </ol> <p>This combination allows U-Net to understand both the big picture (global context from bottleneck) and fine details (local information from skips), producing sharp, accurate outputs.</p> <p>Whether you’re segmenting medical images, generating art with Stable Diffusion, or building your own image editing model, U-Net’s elegant architecture is likely at the core.</p> <hr /> <p><em>This post was created while building a text-conditioned image editing model. The examples and diagrams come from hands-on experimentation with PyTorch.</em></p>

2026/2/3

Building an Image Captioning Transformer from Scratch

<p>After building a text-only transformer for name generation, I wanted to tackle something more ambitious: teaching a model to describe images. This post documents my journey building a minimal image captioning transformer that learns to generate captions like “a dog runs through the snow” from raw pixels.</p> <p><strong><a href="/demos/image-captioning/">Try the live demo!</a></strong> - The model runs entirely in your browser using ONNX Runtime Web.</p>  <h2 id="the-architecture-encoder-decoder-with-cross-attention">The Architecture: Encoder-Decoder with Cross-Attention</h2> <p>Unlike the decoder-only transformer from my previous experiment, image captioning requires an <strong>encoder-decoder</strong> architecture. The key insight is that we need to process two different modalities (images and text) and connect them through <strong>cross-attention</strong>.</p> <p><img src="/images/image_caption_architecture.png" alt="Image Captioning Architecture" /></p> <p>The architecture has two parallel paths:</p> <p><strong>Image Path (Blue):</strong> The image goes through patch embedding, then encoder self-attention layers. This produces “image features” — a sequence of patch embeddings that understand spatial relationships.</p> <p><strong>Text Path (Green):</strong> The caption tokens go through token embedding, then decoder layers with both self-attention (causal) and cross-attention to the image features.</p> <p><strong>The Bridge (Purple):</strong> Cross-attention is where the magic happens. It allows each text token to “look at” all image patches and gather relevant visual information.</p> <h2 id="from-pixels-to-patches-the-vision-encoder">From Pixels to Patches: The Vision Encoder</h2> <p>The first challenge is converting an image into something a transformer can process. Transformers work on sequences, but images are 2D grids. The solution: <strong>split the image into patches</strong>.</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>128x128 image → 16x16 grid of 8x8 patches → 256 patch embeddings </code></pre></div></div> <p>Each 8x8 patch contains 64 pixels × 3 colors = 192 values. A linear layer projects this to 128 dimensions:</p> <div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">class</span> <span class="nc">PatchEmbedding</span><span class="p">(</span><span class="n">nn</span><span class="p">.</span><span class="n">Module</span><span class="p">):</span> <span class="k">def</span> <span class="nf">__init__</span><span class="p">(</span><span class="n">self</span><span class="p">,</span> <span class="n">image_size</span><span class="p">,</span> <span class="n">patch_size</span><span class="p">,</span> <span class="n">n_embd</span><span class="p">):</span> <span class="n">patch_dim</span> <span class="o">=</span> <span class="mi">3</span> <span class="o">*</span> <span class="n">patch_size</span> <span class="o">*</span> <span class="n">patch_size</span> <span class="c1"># 192 </span> <span class="n">self</span><span class="p">.</span><span class="n">proj</span> <span class="o">=</span> <span class="n">nn</span><span class="p">.</span><span class="nc">Linear</span><span class="p">(</span><span class="n">patch_dim</span><span class="p">,</span> <span class="n">n_embd</span><span class="p">)</span> <span class="c1"># 192 → 128 </span> <span class="n">self</span><span class="p">.</span><span class="n">pos_embd</span> <span class="o">=</span> <span class="n">nn</span><span class="p">.</span><span class="nc">Parameter</span><span class="p">(</span><span class="n">torch</span><span class="p">.</span><span class="nf">randn</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="n">n_patches</span><span class="p">,</span> <span class="n">n_embd</span><span class="p">))</span> <span class="k">def</span> <span class="nf">forward</span><span class="p">(</span><span class="n">self</span><span class="p">,</span> <span class="n">x</span><span class="p">):</span> <span class="c1"># Split image into patches, flatten, project </span> <span class="n">patches</span> <span class="o">=</span> <span class="nf">extract_patches</span><span class="p">(</span><span class="n">x</span><span class="p">)</span> <span class="c1"># (B, 256, 192) </span> <span class="k">return</span> <span class="n">self</span><span class="p">.</span><span class="nf">proj</span><span class="p">(</span><span class="n">patches</span><span class="p">)</span> <span class="o">+</span> <span class="n">self</span><span class="p">.</span><span class="n">pos_embd</span> <span class="c1"># (B, 256, 128) </span></code></pre></div></div> <p>Now we have 256 “patch tokens” that can go through self-attention, just like text tokens. The encoder self-attention lets patches learn about each other — a patch showing a dog’s head can attend to patches showing its body and legs, building a coherent understanding of “dog”.</p> <h2 id="cross-attention-the-bridge-between-vision-and-language">Cross-Attention: The Bridge Between Vision and Language</h2> <p>This is the key difference from text-only transformers. In self-attention, Q, K, and V all come from the same source. In cross-attention:</p> <ul> <li><strong>Q (Query)</strong> comes from the text decoder: “What visual information do I need?”</li> <li><strong>K, V (Key, Value)</strong> come from the image encoder: “Here’s what each patch contains”</li> </ul> <div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">class</span> <span class="nc">CrossAttention</span><span class="p">:</span> <span class="k">def</span> <span class="nf">forward</span><span class="p">(</span><span class="n">self</span><span class="p">,</span> <span class="n">text_embeddings</span><span class="p">,</span> <span class="n">image_features</span><span class="p">):</span> <span class="n">Q</span> <span class="o">=</span> <span class="n">text_embeddings</span> <span class="o">@</span> <span class="n">W_q</span> <span class="c1"># What am I looking for? </span> <span class="n">K</span> <span class="o">=</span> <span class="n">image_features</span> <span class="o">@</span> <span class="n">W_k</span> <span class="c1"># What does each patch contain? </span> <span class="n">V</span> <span class="o">=</span> <span class="n">image_features</span> <span class="o">@</span> <span class="n">W_v</span> <span class="c1"># What info to retrieve? </span> <span class="n">scores</span> <span class="o">=</span> <span class="n">Q</span> <span class="o">@</span> <span class="n">K</span><span class="p">.</span><span class="n">T</span> <span class="c1"># (text_len, num_patches) </span> <span class="n">weights</span> <span class="o">=</span> <span class="nf">softmax</span><span class="p">(</span><span class="n">scores</span><span class="p">)</span> <span class="k">return</span> <span class="n">weights</span> <span class="o">@</span> <span class="n">V</span> <span class="c1"># Weighted sum of patch info </span></code></pre></div></div> <p>When generating the word “running”, the model learns to attend heavily to patches showing legs in motion. When generating “snow”, it attends to the white ground patches.</p> <h2 id="training-on-flickr8k">Training on Flickr8k</h2> <p>I used the Flickr8k dataset: 8,000 images with 5 human-written captions each. A key insight was using <strong>random caption sampling</strong> — each epoch, randomly select one of the 5 captions per image. This acts as data augmentation and dramatically reduces overfitting.</p> <table> <thead> <tr> <th>Configuration</th> <th>Train Loss</th> <th>Val Loss</th> <th>Notes</th> </tr> </thead> <tbody> <tr> <td>64x64, fixed caption</td> <td>0.78</td> <td>1.10</td> <td>Baseline</td> </tr> <tr> <td>128x128, fixed caption</td> <td>0.58</td> <td>1.38</td> <td>More detail, more overfitting</td> </tr> <tr> <td>128x128, random caption</td> <td>0.90</td> <td>0.99</td> <td>Much better generalization!</td> </tr> </tbody> </table> <p>The random caption sampling closed the train-val gap from 0.80 to just 0.09.</p> <h2 id="results-what-the-model-learned">Results: What the Model Learned</h2> <p>After 30 epochs of training (~17 minutes on M4 Mac), the model generates reasonable captions:</p> <p><strong>Success case:</strong></p> <p><img src="/images/flickr8k_dog_running.jpg" alt="Dog running on grass" /></p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Generated: "a black dog is running through the grass ." Actual: "A black dog running across green grass ." </code></pre></div></div> <p><strong>Failure case:</strong></p> <p><img src="/images/flickr8k_ski_lodge.jpg" alt="Ski lodge scene" /></p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Generated: "a man in a blue shirt is standing in the stree" Actual: "A crowd of people are enjoying a meal with a view of a mountaintop ." </code></pre></div></div> <p>The model handles simple scenes well (dogs, people, basic actions) but struggles with complex scenes (crowds, multiple objects, subtle context).</p> <h2 id="model-statistics">Model Statistics</h2> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Total parameters: ~980,000 (about 1M) Breakdown: - Patch embedding: 32,896 (3%) - Encoder blocks (2): 395,776 (40%) - Token embedding: 8,960 (1%) - Position embedding: 6,144 (1%) - Decoder blocks (2): 527,616 (54%) - Output layer: 9,286 (1%) </code></pre></div></div> <p>The decoder is larger than the encoder because each decoder block has both self-attention AND cross-attention.</p> <h2 id="key-learnings">Key Learnings</h2> <h3 id="1-patches-are-the-tokenizer-for-images">1. Patches are the “tokenizer” for images</h3> <p>Just as we split text into tokens, we split images into patches. This converts the 2D spatial structure into a sequence that transformers can process. The same weight matrix processes every patch, learning a universal “patch reader”.</p> <h3 id="2-cross-attention-is-the-bridge">2. Cross-attention is the bridge</h3> <p>The key architectural difference from text-only transformers. It lets the text generation process “see” the image at every step, attending to relevant patches for each word being generated.</p> <h3 id="3-data-augmentation-matters-enormously">3. Data augmentation matters enormously</h3> <p>Using all 5 captions with random sampling was more impactful than doubling the image resolution. The model learns semantic concepts rather than memorizing specific strings.</p> <h3 id="4-resolution-limits-understanding">4. Resolution limits understanding</h3> <p>At 128x128, a tricycle looks like a blob. The model can distinguish dogs from people, but struggles with fine details. Real vision models use 224x224 or higher.</p> <h3 id="5-this-is-still-a-toy-model">5. This is still a toy model</h3> <p>Production image captioning models use:</p> <ul> <li>Pretrained vision encoders (CLIP, ViT trained on millions of images)</li> <li>Word-level tokenization (shorter sequences)</li> <li>Much larger datasets (COCO has 330k images)</li> <li>Billions of parameters</li> </ul> <h2 id="improvement-using-pretrained-clip-encoder">Improvement: Using Pretrained CLIP Encoder</h2> <p>After training the from-scratch model, I wanted to see how much a pretrained vision encoder could help. I created a second version that uses <strong>CLIP ViT-B/32</strong> as a frozen image encoder, training only the decoder and a projection layer.</p> <h3 id="architecture-changes">Architecture Changes</h3> <p>Instead of learning patch embeddings from scratch:</p> <ul> <li>CLIP’s pretrained ViT processes the image (224x224 input)</li> <li>50 patch embeddings (768-dim) are projected to the decoder dimension</li> <li>Only the decoder (~3.8M params) is trained; CLIP (~87M params) is frozen</li> </ul> <div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">class</span> <span class="nc">CLIPCaptioningModel</span><span class="p">(</span><span class="n">nn</span><span class="p">.</span><span class="n">Module</span><span class="p">):</span> <span class="k">def</span> <span class="nf">encode_image</span><span class="p">(</span><span class="n">self</span><span class="p">,</span> <span class="n">img</span><span class="p">):</span> <span class="c1"># Use CLIP's visual transformer (frozen) </span> <span class="k">with</span> <span class="n">torch</span><span class="p">.</span><span class="nf">no_grad</span><span class="p">():</span> <span class="n">x</span> <span class="o">=</span> <span class="n">clip_model</span><span class="p">.</span><span class="nf">visual</span><span class="p">(</span><span class="n">img</span><span class="p">)</span> <span class="c1"># (B, 50, 768) </span> <span class="k">return</span> <span class="n">self</span><span class="p">.</span><span class="nf">visual_proj</span><span class="p">(</span><span class="n">x</span><span class="p">)</span> <span class="c1"># Project to decoder dim </span></code></pre></div></div> <h3 id="results-comparison">Results Comparison</h3> <table> <thead> <tr> <th>Metric</th> <th>From-Scratch</th> <th>CLIP-based</th> </tr> </thead> <tbody> <tr> <td>Val Loss</td> <td>1.29</td> <td><strong>0.86</strong></td> </tr> <tr> <td>Train Loss</td> <td>1.23</td> <td>0.75</td> </tr> <tr> <td>Epochs</td> <td>30</td> <td>20</td> </tr> <tr> <td>Training Time</td> <td>~17 min</td> <td>~17 min</td> </tr> <tr> <td>Model Size</td> <td>4 MB</td> <td>363 MB</td> </tr> </tbody> </table> <p>The CLIP-based model achieves <strong>33% lower validation loss</strong> with fewer epochs!</p> <h3 id="sample-captions">Sample Captions</h3> <p>For the same test image (two dogs in snow):</p> <table> <thead> <tr> <th>Model</th> <th>Caption</th> </tr> </thead> <tbody> <tr> <td>From-scratch</td> <td>“a black dog and a white dog are in the snow .”</td> </tr> <tr> <td>CLIP-based</td> <td>“two dogs playing in the snow .”</td> </tr> <tr> <td>Ground truth</td> <td>“a black dog is running after a white dog in the snow .”</td> </tr> </tbody> </table> <p>The CLIP-based model produces more natural, concise captions. It benefits from CLIP having been trained on 400 million image-text pairs — it already understands visual concepts like “dogs” and “playing” without needing to learn them from our small 8k image dataset.</p> <h3 id="testing-on-complex-scenes">Testing on Complex Scenes</h3> <p>I tested both models on the validation set, focusing on complex scenes that the from-scratch model struggled with:</p> <table> <thead> <tr> <th>Scene</th> <th>From-Scratch</th> <th>CLIP-based</th> <th>Ground Truth</th> </tr> </thead> <tbody> <tr> <td>Ice skating rink</td> <td>“a man in a blue shirt…”</td> <td>“a group of people standing in the snow .”</td> <td>“A group of people are ice skating in a big city .”</td> </tr> <tr> <td>Rock climbing</td> <td>“a woman is standing…”</td> <td>“a woman in a red shirt is climbing a rock .”</td> <td>“A kid rock climbing against the backdrop of a green valley”</td> </tr> <tr> <td>People at boats</td> <td>“a man is…”</td> <td>“a group of people standing in a rowd of a boat”</td> <td>“A group of people waiting to ride boats .”</td> </tr> <tr> <td>Mountain hikers</td> <td>“a man in…”</td> <td>“two people stand on the side of a mountain .”</td> <td>“Three people facing the mountains .”</td> </tr> </tbody> </table> <p><strong>Key observations:</strong></p> <ol> <li><strong>Better at groups/crowds</strong> — CLIP recognizes “group of people” much better than the from-scratch model which defaults to “a man”</li> <li><strong>Better semantic understanding</strong> — Recognizes concepts like “rock climbing”, “mountain”, “boat” that the small model misses entirely</li> <li><strong>Still struggles with fine details</strong> — Exact counts (two vs three people), specific activities (ice skating vs standing)</li> <li><strong>More robust to complex scenes</strong> — Doesn’t collapse to generic “man in blue shirt” for difficult images</li> </ol> <p>The pretrained visual features give CLIP a huge advantage on scenes requiring real-world knowledge.</p> <h3 id="tradeoff-accuracy-vs-size">Tradeoff: Accuracy vs Size</h3> <p>The improved model is 363MB (vs 4MB), making it impractical for browser deployment. This is the classic accuracy-size tradeoff:</p> <ul> <li><strong>From-scratch model</strong>: Smaller, deployable, but less accurate</li> <li><strong>CLIP-based model</strong>: More accurate, but requires a large pretrained encoder</li> </ul> <p>For production, you’d typically use the large model on a server, or apply techniques like knowledge distillation to compress it.</p> <h2 id="improvement-word-level-tokenization">Improvement: Word-Level Tokenization</h2> <p>The character-level model processes “a black dog” as 11 tokens (including spaces). Word-level tokenization reduces this to just 3 tokens, making sequences shorter and potentially easier to learn.</p> <h3 id="parameter-count-changes">Parameter Count Changes</h3> <p>Switching from character-level to word-level tokenization dramatically changes where the parameters live:</p> <table> <thead> <tr> <th>Component</th> <th>Character-Level</th> <th>Word-Level</th> <th>Change</th> </tr> </thead> <tbody> <tr> <td>Token embedding</td> <td>8,960 (70 × 128)</td> <td>570,240 (4453 × 128)</td> <td>+561K</td> </tr> <tr> <td>Position embedding</td> <td>6,144 (48 × 128)</td> <td>2,560 (20 × 128)</td> <td>-3.5K</td> </tr> <tr> <td>Output layer</td> <td>8,960</td> <td>570,240</td> <td>+561K</td> </tr> <tr> <td><strong>Total model</strong></td> <td>~980K</td> <td>~2.1M</td> <td><strong>+1.1M (2.2×)</strong></td> </tr> </tbody> </table> <p>The vocabulary explodes from ~70 characters to ~4500 words, but sequences shrink from 48 characters to 20 words. The net effect: <strong>2.2× more parameters</strong>, almost entirely in the embedding layers.</p> <h3 id="results-comparison-1">Results Comparison</h3> <table> <thead> <tr> <th>Metric</th> <th>Character-Level</th> <th>Word-Level</th> </tr> </thead> <tbody> <tr> <td>Val Loss</td> <td>0.99</td> <td><strong>2.98</strong></td> </tr> <tr> <td>Train Loss</td> <td>0.90</td> <td>2.42</td> </tr> <tr> <td>Vocab Size</td> <td>70</td> <td>4,453</td> </tr> <tr> <td>Max Seq Length</td> <td>48</td> <td>20</td> </tr> <tr> <td>Model Size</td> <td>4 MB</td> <td>8.2 MB</td> </tr> </tbody> </table> <p>Wait — the word-level loss is <strong>higher</strong>? This is actually expected:</p> <ol> <li><strong>Loss is per-token</strong>: Character-level predicts from 70 options; word-level predicts from 4,453 options</li> <li><strong>Different scales</strong>: A word-level loss of 2.98 means perplexity ~20 (choosing from 4453 words), while character loss 0.99 means perplexity ~2.7 (choosing from 70 chars)</li> <li><strong>The captions are similar quality</strong> despite the different loss values</li> </ol> <h3 id="sample-caption">Sample Caption</h3> <p>For the same test image (two dogs in snow):</p> <table> <thead> <tr> <th>Model</th> <th>Caption</th> </tr> </thead> <tbody> <tr> <td>Character-level</td> <td>“a black dog and a white dog are in the snow .”</td> </tr> <tr> <td>Word-level</td> <td>“a dog is running through the snow .”</td> </tr> <tr> <td>Ground truth</td> <td>“a black dog is running after a white dog in the snow .”</td> </tr> </tbody> </table> <p>The word-level model produces fluent captions but with a smaller effective vocabulary (it saw each word fewer times during training than character-level saw each character).</p> <h3 id="key-insight-vocabulary-size-vs-training-data">Key Insight: Vocabulary Size vs Training Data</h3> <p>Word-level tokenization works better when you have <strong>lots of training data</strong>. With only 8k images:</p> <ul> <li>Character-level sees each character thousands of times → learns robust patterns</li> <li>Word-level sees many words only a few times → harder to learn good embeddings</li> </ul> <p>This is why production models use:</p> <ul> <li><strong>Subword tokenization</strong> (BPE, WordPiece): Best of both worlds</li> <li><strong>Much larger datasets</strong>: COCO (330k), Conceptual Captions (3M+)</li> <li><strong>Pretrained word embeddings</strong>: GloVe, Word2Vec, etc.</li> </ul> <h2 id="improvement-clip--glove-pretrained-embeddings">Improvement: CLIP + GloVe Pretrained Embeddings</h2> <p>Since the word-level model struggled with limited training data, I tried combining the best of both worlds: <strong>CLIP’s pretrained vision encoder</strong> with <strong>GloVe pretrained word embeddings</strong>.</p> <h3 id="the-idea">The Idea</h3> <p>Instead of learning word embeddings from scratch with only 8k images, why not use GloVe embeddings trained on 6 billion words? This gives the model a head start on understanding word relationships.</p> <div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">class</span> <span class="nc">CLIPGloVeCaptioningModel</span><span class="p">(</span><span class="n">nn</span><span class="p">.</span><span class="n">Module</span><span class="p">):</span> <span class="k">def</span> <span class="nf">__init__</span><span class="p">(</span><span class="n">self</span><span class="p">,</span> <span class="n">vocab_size</span><span class="p">,</span> <span class="n">clip_model</span><span class="p">,</span> <span class="n">glove_embeddings</span><span class="p">,</span> <span class="p">...):</span> <span class="c1"># Use CLIP for vision (frozen) </span> <span class="n">self</span><span class="p">.</span><span class="n">clip_model</span> <span class="o">=</span> <span class="n">clip_model</span> <span class="c1"># Use GloVe for word embeddings (fine-tuned) </span> <span class="n">self</span><span class="p">.</span><span class="n">token_embed</span> <span class="o">=</span> <span class="n">nn</span><span class="p">.</span><span class="nc">Embedding</span><span class="p">(</span><span class="n">vocab_size</span><span class="p">,</span> <span class="n">glove_dim</span><span class="p">)</span> <span class="n">self</span><span class="p">.</span><span class="n">token_embed</span><span class="p">.</span><span class="n">weight</span><span class="p">.</span><span class="n">data</span><span class="p">.</span><span class="nf">copy_</span><span class="p">(</span><span class="n">glove_embeddings</span><span class="p">)</span> <span class="c1"># Project GloVe dim (100) to decoder dim (256) </span> <span class="n">self</span><span class="p">.</span><span class="n">glove_proj</span> <span class="o">=</span> <span class="n">nn</span><span class="p">.</span><span class="nc">Linear</span><span class="p">(</span><span class="n">glove_dim</span><span class="p">,</span> <span class="n">n_embd</span><span class="p">)</span> </code></pre></div></div> <h3 id="glove-coverage">GloVe Coverage</h3> <p>Using GloVe 6B 100d (100-dimensional embeddings trained on 6 billion tokens):</p> <ul> <li><strong>4441 out of 4517 words</strong> (98.3%) found in GloVe</li> <li>Only 76 words missing (mostly rare or domain-specific terms)</li> <li>Missing words initialized with small random values</li> </ul> <h3 id="results">Results</h3> <table> <thead> <tr> <th>Metric</th> <th>Word-Level (random)</th> <th>CLIP + GloVe</th> </tr> </thead> <tbody> <tr> <td>Val Loss</td> <td>2.98</td> <td><strong>2.55</strong></td> </tr> <tr> <td>Train Loss</td> <td>2.42</td> <td>1.78</td> </tr> <tr> <td>Epochs</td> <td>30</td> <td>30</td> </tr> <tr> <td>GloVe Coverage</td> <td>N/A</td> <td>98.3%</td> </tr> </tbody> </table> <p>The GloVe embeddings give a <strong>14% improvement</strong> in validation loss!</p> <h3 id="sample-caption-1">Sample Caption</h3> <p>For the same test image (two dogs in snow):</p> <table> <thead> <tr> <th>Model</th> <th>Caption</th> </tr> </thead> <tbody> <tr> <td>Word-level (random init)</td> <td>“a dog is running through the snow .”</td> </tr> <tr> <td><strong>CLIP + GloVe</strong></td> <td>“two dogs are playing in the snow .”</td> </tr> <tr> <td>Ground truth</td> <td>“a black dog is running after a white dog in the snow .”</td> </tr> </tbody> </table> <p>The GloVe model correctly identifies “two dogs” rather than “a dog”, suggesting the pretrained embeddings help with understanding quantities and relationships.</p> <h3 id="key-insight-transfer-learning-stacks">Key Insight: Transfer Learning Stacks</h3> <p>This experiment shows that <strong>transfer learning compounds</strong>:</p> <ol> <li>CLIP brings pretrained visual understanding (400M image-text pairs)</li> <li>GloVe brings pretrained word relationships (6B tokens)</li> <li>Only the decoder and projection layers need to learn task-specific mappings</li> </ol> <p>Even with just 8k training images, combining two pretrained components achieves significantly better results than training from scratch.</p> <h2 id="whats-next">What’s Next</h2> <p>Remaining improvements to explore:</p> <ol> <li><del><strong>Pretrained vision encoder</strong>: Use CLIP or ViT instead of learning from scratch</del> ✅ Done!</li> <li><del><strong>Word-level tokenization</strong>: “a black dog” as 3 tokens instead of 11 characters</del> ✅ Done!</li> <li><del><strong>Pretrained word embeddings</strong>: Use GloVe for better word representations</del> ✅ Done!</li> <li><strong>Subword tokenization</strong>: Use BPE for better vocab coverage</li> <li><strong>More data</strong>: COCO dataset (330k images) instead of Flickr8k (8k)</li> <li><strong>Knowledge distillation</strong>: Train a small model to mimic the CLIP-based one</li> </ol> <p>But even the minimal from-scratch implementation demonstrates the core concepts: patch embeddings, encoder-decoder architecture, and cross-attention as the bridge between vision and language.</p> <h2 id="code">Code</h2> <p>The complete training script is available in my <a href="https://github.com/Jeswang/learn-llm">learn-llm</a> repository as <code class="language-plaintext highlighter-rouge">train-image-caption.py</code>.</p>

2026/1/30

Building a Language Transformer Step by Step

<p>After months of reading about transformers and LLMs, I finally decided to build one from scratch. Not by copy-pasting code, but by incrementally adding each architectural component and measuring its impact. The result was a character-level name generator trained on 32,033 names, and the journey taught me more than any paper or tutorial could.</p>  <h2 id="preparation-standing-on-the-shoulders-of-giants">Preparation: Standing on the Shoulders of Giants</h2> <p>Before diving into code, I spent time building intuition through two excellent resources:</p> <p><strong>“Build a Large Language Model (From Scratch)” by Sebastian Raschka</strong> was my theoretical foundation. The book walks through every component of a transformer with clear explanations and diagrams. Reading it gave me a mental model of how attention, embeddings, and layer normalization fit together — knowledge that proved essential when debugging my own implementation.</p> <p><strong>Andrej Karpathy’s YouTube series</strong> (<a href="https://www.youtube.com/playlist?list=PLAqhIrjkxbuWI23v9cThsA9GvCAUhRvKZ">Neural Networks: Zero to Hero</a>) was equally valuable. His “Let’s build GPT” video demystified the architecture by building it live on screen. Watching someone think through the design decisions — why we use residual connections, how attention matrices work, what LayerNorm actually does — made the concepts stick in a way that reading alone couldn’t. His <a href="https://github.com/karpathy/makemore">makemore</a> repository became the dataset and benchmark for my experiments.</p> <p>With this foundation, I was ready to build.</p> <h2 id="the-experiment">The Experiment</h2> <p>I incrementally built a character-level transformer for name generation. Each step adds one architectural improvement. All models were trained with batch size 32, AdamW optimizer, and per-name padding with masked loss.</p> <h2 id="results---architecture-comparison-5000-steps">Results - Architecture Comparison (5,000 steps)</h2> <table> <thead> <tr> <th>Config</th> <th>N_EMBD</th> <th>Heads</th> <th>Layers</th> <th>Params</th> <th>Train</th> <th>Test</th> </tr> </thead> <tbody> <tr> <td>baseline</td> <td>32</td> <td>1</td> <td>1</td> <td>2,908</td> <td>2.35</td> <td>2.35</td> </tr> <tr> <td>double embd</td> <td>64</td> <td>1</td> <td>1</td> <td>8,860</td> <td>2.34</td> <td>2.34</td> </tr> <tr> <td>2 heads</td> <td>32</td> <td>2</td> <td>1</td> <td>5,948</td> <td>2.25</td> <td>2.23</td> </tr> <tr> <td>4 layers</td> <td>32</td> <td>2</td> <td>4</td> <td>18,332</td> <td>2.00</td> <td>2.04</td> </tr> <tr> <td>+ MLP</td> <td>32</td> <td>2</td> <td>4</td> <td>51,740</td> <td>1.97</td> <td>2.02</td> </tr> <tr> <td>+ LayerNorm</td> <td>32</td> <td>2</td> <td>4</td> <td>52,252</td> <td>1.96</td> <td>1.99</td> </tr> <tr> <td>+ RoPE</td> <td>32</td> <td>2</td> <td>4</td> <td>52,252</td> <td>1.94</td> <td>1.98</td> </tr> <tr> <td>+ GELU</td> <td>32</td> <td>2</td> <td>4</td> <td>52,252</td> <td>1.94</td> <td>1.94</td> </tr> </tbody> </table> <h2 id="results---scaling-up">Results - Scaling Up</h2> <table> <thead> <tr> <th>Config</th> <th>Steps</th> <th>Train</th> <th>Test</th> <th>Notes</th> </tr> </thead> <tbody> <tr> <td>N_EMBD=32, 2 heads</td> <td>5,000</td> <td>1.94</td> <td>1.94</td> <td>Baseline final model</td> </tr> <tr> <td>N_EMBD=64, 4 heads</td> <td>5,000</td> <td>1.84</td> <td>1.92</td> <td>Matches makemore architecture</td> </tr> <tr> <td>N_EMBD=64, 4 heads + dropout</td> <td>5,000</td> <td>1.95</td> <td>2.00</td> <td>Dropout slows convergence</td> </tr> <tr> <td>N_EMBD=64, 4 heads + dropout</td> <td>20,000</td> <td>1.75</td> <td>1.85</td> <td>Longer training helps</td> </tr> <tr> <td>+ LR schedule, weight decay, grad clip</td> <td>20,000</td> <td>1.72</td> <td>1.86</td> <td>Training improvements</td> </tr> </tbody> </table> <p>Makemore’s default transformer achieves ~1.92 test loss with N_EMBD=64, 4 heads, 4 layers.</p> <h2 id="generated-names">Generated Names</h2> <p>Sample outputs from the final model (N_EMBD=64, 4 heads, 20k steps with all training improvements):</p> <blockquote> <p>kaelynn, aileigh, elyce, yadi, ovani, derella, nyailee, ranyah, niaa, sett</p> </blockquote> <h2 id="key-findings">Key Findings</h2> <h3 id="depth-beats-width">Depth beats width</h3> <p>Doubling embedding size from 32 to 64 (3x params) gave almost no improvement (2.35 -> 2.34). Adding a second attention head with fewer total params (5,948 vs 8,860) dropped loss by 0.12. Stacking 4 layers was the single biggest improvement, dropping test loss from 2.23 to 2.04. The model benefits far more from multiple layers of processing than from wider representations at a single layer.</p> <h3 id="data-handling-matters-most">Data handling matters most</h3> <p>Before adding per-name padding, our best model achieved 2.36 test loss. After switching to per-name padding with masked loss (same architecture), it dropped to 1.94. This was a larger improvement than all architectural changes combined. The reason: without padding, the model wasted capacity trying to predict across name boundaries — an impossible task that added noise to every gradient update.</p> <h3 id="mlp-adds-capacity-but-needs-regularization">MLP adds capacity but needs regularization</h3> <p>Adding the feed-forward network (MLP) to each layer tripled the parameter count (18k -> 52k) but only modestly improved results. It also widened the train-test gap (2.00/2.04 -> 1.97/2.02), suggesting mild overfitting. The MLP lets the model transform representations nonlinearly after attention gathers information, but at this small scale the effect is limited.</p> <h3 id="layernorm-and-rope-help-incrementally">LayerNorm and RoPE help incrementally</h3> <p>LayerNorm stabilized training and closed the train-test gap slightly. RoPE (Rotary Position Embeddings) gave the model awareness of character positions without adding any parameters. Neither was dramatic at this scale, but both are essential for larger models — LayerNorm enables training deep networks, and RoPE enables generalization to longer sequences.</p> <h3 id="gelu-vs-relu-is-negligible-at-small-scale">GELU vs ReLU is negligible at small scale</h3> <p>Switching from ReLU to GELU activation in the MLP had no measurable effect. The smoother gradient flow matters more when networks are deeper and wider.</p> <h3 id="scaling-up-helps-significantly">Scaling up helps significantly</h3> <p>Doubling N_EMBD to 64 and using 4 heads (matching makemore’s architecture) dropped test loss from 1.94 to 1.92 at 5k steps. With longer training (20k steps), the model reached 1.85 test loss — surpassing makemore’s default.</p> <h3 id="dropout-trades-speed-for-generalization">Dropout trades speed for generalization</h3> <p>Adding 20% dropout increased the train-test gap initially and slowed convergence. At 5k steps, it actually hurt test loss (1.92 -> 2.00). But it prevents overfitting during longer training runs, allowing the model to keep improving past where it would otherwise plateau.</p> <h3 id="training-improvements-compound">Training improvements compound</h3> <p>Learning rate scheduling (warmup + cosine decay), weight decay (0.01), and gradient clipping (max_norm=1.0) together produced smoother training curves. The cosine decay prevents the learning rate from being too high in later steps when fine-tuning. Weight decay acts as regularization. Gradient clipping prevents instability from occasional large gradients.</p> <h2 id="architecture-summary">Architecture Summary</h2> <p>The final model is a proper transformer decoder:</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Input tokens -> Token Embedding (28 vocab -> 64 dim) -> 4x Transformer Blocks: -> LayerNorm -> Multi-Head Attention (4 heads, RoPE, dropout) -> Residual -> LayerNorm -> MLP (64 -> 256 -> 64, GELU, dropout) -> Residual -> Linear (64 -> 28 vocab) -> Cross-entropy loss (masked on PAD tokens) </code></pre></div></div> <p>Training config:</p> <ul> <li>20,000 steps</li> <li>Batch size 32</li> <li>AdamW optimizer with weight decay 0.01</li> <li>Learning rate: warmup to 1e-3 over 200 steps, cosine decay to 1e-4</li> <li>Gradient clipping: max_norm=1.0</li> <li>Dropout: 0.2</li> </ul> <h2 id="what-the-loss-means">What the Loss Means</h2> <p><img src="/images/cross_entropy.png" alt="Cross Entropy Loss" /></p> <p>A loss of 1.86 means the model assigns ~15.6% probability on average to the correct next character (<code class="language-plaintext highlighter-rouge">e^(-1.86)</code>). Random guessing over 27 characters would give ~3.7% (loss = 3.30). Perfect prediction is impossible because many positions are genuinely ambiguous — after “ma”, the next character could be r, d, k, x, t, and many others.</p> <p>Progress through this project:</p> <ul> <li>Start: 2.35 test loss (~9.5% confidence)</li> <li>Final: 1.86 test loss (~15.6% confidence)</li> <li>Improvement: ~1.6x more confident on the correct character</li> </ul> <h2 id="conclusion">Conclusion</h2> <p>Building a transformer incrementally taught me that the magic isn’t in any single component — it’s in how they work together. Data preprocessing had the biggest impact. Depth mattered more than width. And the “modern” improvements (LayerNorm, RoPE, GELU) are less about dramatic gains and more about enabling scale.</p>

2026/1/28

Reverse Engineering Guitar Pro 8's Locked Files

<p>Have you ever worked on a Guitar Pro tab, saved it, and then realized you couldn’t edit it anymore because it was “locked”? Or perhaps you downloaded a tab that was perfect but needed just one small tweak, and the author had locked it?</p> <p>I recently went down a rabbit hole reverse-engineering this “protection” mechanism in Guitar Pro 8. What I found was a classic case of “security through obscurity” — and not very deep obscurity at that.</p>  <h2 id="the-problem">The Problem</h2> <p>Guitar Pro has a feature to “lock” a file. When locked, the file can be opened and played, but the editing features are disabled. If you peek inside the <code class="language-plaintext highlighter-rouge">.gp</code> file (which is just a ZIP archive), you’ll see a few interesting things:</p> <ol> <li>A file named <code class="language-plaintext highlighter-rouge">editLocked</code>.</li> <li>The main content <code class="language-plaintext highlighter-rouge">Content/score.gpif</code> is encrypted (it doesn’t have the standard XML header).</li> </ol> <p>Removing <code class="language-plaintext highlighter-rouge">editLocked</code> isn’t enough. The app sees it’s missing, but the content remains encrypted and unreadable.</p> <h2 id="the-breakthrough">The Breakthrough</h2> <p>As Guitar Pro can open and play the file without ever prompting for a password, it was clear that the key to decrypt the content must be available to the application without user input. This realization led me to investigate how the application handles these files internally.</p> <p>I analyzed the <code class="language-plaintext highlighter-rouge">GuitarPro</code> binary and its libraries, specifically <code class="language-plaintext highlighter-rouge">libGPIO.dylib</code>.</p> <h3 id="1-the-salt">1. The Salt</h3> <p>Deep in the binary, I found a reference to a static salt used in the encryption routine. <code class="language-plaintext highlighter-rouge">da40cc64900b617a0f72ad4e6ef42f9c</code></p> <h3 id="2-the-password">2. The Password</h3> <p>Tracing the assembly code for <code class="language-plaintext highlighter-rouge">Score::setLockPwd</code>, I found something surprising. The application reads the <strong>entire content</strong> of the <code class="language-plaintext highlighter-rouge">editLocked</code> file (which contains a salt and a hash of the user’s original password) and sets <em>that string</em> as the internal password for decryption.</p> <p>So, the “password” to decrypt audio and score data isn’t what you typed. It’s the metadata file itself.</p> <h2 id="the-solution">The Solution</h2> <p>Putting it all together, the encryption scheme is:</p> <ul> <li><strong>Algorithm</strong>: AES-256-CBC</li> <li><strong>Key Derivation</strong>: PBKDF2-HMAC-SHA1 (4096 iterations)</li> <li><strong>Password</strong>: The content of <code class="language-plaintext highlighter-rouge">editLocked</code> (e.g., <code class="language-plaintext highlighter-rouge">salt$hash</code>)</li> <li><strong>Salt</strong>: The static binary salt (<code class="language-plaintext highlighter-rouge">da40cc...</code>)</li> </ul> <p>With this information, I wrote a Python script <code class="language-plaintext highlighter-rouge">unlock_score.py</code> that fully unlocks these files.</p> <h3 id="the-script">The Script</h3> <p>Here is the core logic of the unlocker:</p> <div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">STATIC_SALT_HEX</span> <span class="o">=</span> <span class="sh">"</span><span class="s">da40cc64900b617a0f72ad4e6ef42f9c</span><span class="sh">"</span> <span class="k">def</span> <span class="nf">decrypt_gpif</span><span class="p">(</span><span class="n">encrypted_data</span><span class="p">,</span> <span class="n">password</span><span class="p">):</span> <span class="n">salt</span> <span class="o">=</span> <span class="n">binascii</span><span class="p">.</span><span class="nf">unhexlify</span><span class="p">(</span><span class="n">STATIC_SALT_HEX</span><span class="p">)</span> <span class="c1"># PBKDF2 with 4096 iterations </span> <span class="n">key</span> <span class="o">=</span> <span class="n">hashlib</span><span class="p">.</span><span class="nf">pbkdf2_hmac</span><span class="p">(</span><span class="sh">"</span><span class="s">sha1</span><span class="sh">"</span><span class="p">,</span> <span class="n">password</span><span class="p">.</span><span class="nf">encode</span><span class="p">(),</span> <span class="n">salt</span><span class="p">,</span> <span class="mi">4096</span><span class="p">,</span> <span class="mi">32</span><span class="p">)</span> <span class="n">iv</span> <span class="o">=</span> <span class="n">encrypted_data</span><span class="p">[:</span><span class="mi">16</span><span class="p">]</span> <span class="n">ciphertext</span> <span class="o">=</span> <span class="n">encrypted_data</span><span class="p">[</span><span class="mi">16</span><span class="p">:]</span> <span class="n">cipher</span> <span class="o">=</span> <span class="nc">Cipher</span><span class="p">(</span><span class="n">algorithms</span><span class="p">.</span><span class="nc">AES</span><span class="p">(</span><span class="n">key</span><span class="p">),</span> <span class="n">modes</span><span class="p">.</span><span class="nc">CBC</span><span class="p">(</span><span class="n">iv</span><span class="p">),</span> <span class="n">backend</span><span class="o">=</span><span class="nf">default_backend</span><span class="p">())</span> <span class="n">decryptor</span> <span class="o">=</span> <span class="n">cipher</span><span class="p">.</span><span class="nf">decryptor</span><span class="p">()</span> <span class="n">decrypted</span> <span class="o">=</span> <span class="n">decryptor</span><span class="p">.</span><span class="nf">update</span><span class="p">(</span><span class="n">ciphertext</span><span class="p">)</span> <span class="o">+</span> <span class="n">decryptor</span><span class="p">.</span><span class="nf">finalize</span><span class="p">()</span> <span class="c1"># Decompress zlib payload </span> <span class="k">return</span> <span class="n">zlib</span><span class="p">.</span><span class="nf">decompress</span><span class="p">(</span><span class="n">decrypted</span><span class="p">)</span> </code></pre></div></div> <p>You can find the full tool on <a href="https://gist.github.com/Jeswang/eeac3eb0977dee490814926e74538c9a">GitHub Gist</a>.</p> <h2 id="the-role-of-llms-in-reverse-engineering">The Role of LLMs in Reverse Engineering</h2> <p>A fascinating part of this project was using an LLM to accelerate the reverse engineering process. While tools like <code class="language-plaintext highlighter-rouge">otool</code> and <code class="language-plaintext highlighter-rouge">grep</code> provided the raw data, the AI acted as a “force multiplier”:</p> <ul> <li><strong>Reading Code at Scale</strong>: The most daunting part of reverse engineering is the sheer volume of information. A binary dump can contain millions of lines of assembly instructions. For a human, “reading” this to build a mental model of the software’s behavior is a task that takes days or weeks. The LLM, however, could digest these massive text dumps instantly.</li> <li><strong>Semantic Understanding</strong>: It didn’t just match patterns; it understood the <em>intent</em> of the low-level code. By analyzing the context around function calls (like <code class="language-plaintext highlighter-rouge">AES_encrypt</code> or <code class="language-plaintext highlighter-rouge">setLockPwd</code>), the AI could infer high-level logic—such as identifying that the password was being sourced from file metadata—without us having to manually trace every register.</li> <li><strong>Time Compression</strong>: This ability to essentially “read” the binary allowed us to bypass the tedious manual tracing phase entirely. We could ask high-level questions about the software’s behavior and get answers derived from the raw assembly, compressing what would be an “forever” task for a human into a quick conversation.</li> </ul> <p>This collaboration turned what could have been a multi-day debugging session into a targeted, systematic investigation.</p> <h2 id="conclusion">Conclusion</h2> <p>This exercise showed that the “lock” feature in Guitar Pro is effectively just a UI flag backed by a fixed-key obfuscation. It prevents casual editing but offers no real security against someone determined to access the data.</p> <p><em>Disclaimer: This information is for educational purposes only. Always respect copyright and the wishes of content creators.</em></p>

2026/1/17

Vibe Coding - Extracting Pet Sprites from Cross Gate

<p><img src="/images/cross-gate-pet-viewer.png" alt="Cross Gate Pet Viewer" /></p> <p>Cross Gate (魔力宝贝) was one of the most influential MMORPGs in Taiwan and China during the early 2000s. As someone who spent countless hours collecting pets in this game during my childhood, I recently embarked on a nostalgia-driven project: extracting all the pet sprites from the game files and building a modern web viewer to browse them.</p>  <h2 id="the-challenge">The Challenge</h2> <p>Game resources from the early 2000s are notoriously difficult to work with. Cross Gate uses proprietary binary formats for its graphics and animation data:</p> <ul> <li><strong>GraphicInfo_*.bin</strong> (40 bytes per entry) - Metadata for each graphic including dimensions, offsets, and addresses</li> <li><strong>Graphic_*.bin</strong> - RLE-compressed 8-bit indexed images with transparency</li> <li><strong>AnimeInfo_*.bin</strong> (12 bytes per entry) - Animation metadata linking pet IDs to frame sequences</li> <li><strong>Anime_*.bin</strong> - Animation frame data with actions and directions</li> <li><strong>Palette files (.cgp)</strong> - 224-color palettes mapping indices 16-239</li> </ul> <p>The compression format is a custom RLE implementation with multiple encoding modes (literal, repeat, transparent) and variable-length counters.</p> <h2 id="the-solution">The Solution</h2> <p>Using AI-assisted development (Claude Code and Antigravity), I built a Python extraction pipeline:</p> <ol> <li><strong>Parse the binary formats</strong> - Read the structured binary files, extracting metadata and addresses</li> <li><strong>Decompress RLE graphics</strong> - Implement the full RLE decompression algorithm with all encoding modes</li> <li><strong>Apply palettes</strong> - Map 8-bit indexed pixels to RGB colors using the game’s palette files</li> <li><strong>Generate animated GIFs</strong> - Combine frames into animated GIFs for each pet’s actions and directions</li> </ol> <p>Each pet has up to 10 actions (Idle, Walk, Attack, Defend, Cast, etc.) and 8 directions, resulting in potentially 80 GIF animations per pet.</p> <h2 id="the-frontend">The Frontend</h2> <p>I built a Next.js web application to browse the extracted pets:</p> <ul> <li><strong>Grid view</strong> displaying all available pets</li> <li><strong>Detail view</strong> with interactive controls for actions and directions</li> <li><strong>Drag-to-rotate</strong> functionality for intuitive direction changes</li> <li><strong>Pixel-perfect rendering</strong> with <code class="language-plaintext highlighter-rouge">image-rendering: pixelated</code> to preserve the retro aesthetic</li> </ul> <h2 id="lessons-learned">Lessons Learned</h2> <ol> <li><strong>Binary format reverse engineering is time-consuming</strong> - Even with AI assistance, understanding undocumented binary formats requires careful experimentation and validation</li> <li><strong>Progress persistence is essential</strong> - With 1000+ pets to process, the batch generator needed to skip already-processed pets and handle timeouts gracefully</li> <li><strong>Test with edge cases early</strong> - Some pets had unusual frame counts or missing animations that caused the initial implementation to fail</li> </ol> <h2 id="references">References</h2> <p>This project was made possible by the <a href="https://github.com/x2048/cgg-viewer">cgg-viewer</a> project, which provided the foundational understanding of Cross Gate’s binary file formats and RLE decompression algorithm. The original Python implementation by the cgg-viewer author was invaluable for understanding how to correctly parse GraphicInfo, AnimeInfo, and palette files.</p> <h2 id="whats-next">What’s Next</h2> <ul class="task-list"> <li class="task-list-item"><input type="checkbox" class="task-list-item-checkbox" disabled="disabled" />Try <a href="https://3d.hunyuan.tencent.com/">Tencent Hunyuan 3D</a> to convert 2D sprites into 3D models</li> </ul> <p>You can try it out at <a href="https://1203906e.cross-gate-pets.pages.dev/">https://1203906e.cross-gate-pets.pages.dev/</a>.</p>

2026/1/16

Breaking Up with Evernote: Building a Custom Migration Tool for Apple Notes

<p>After 15+ years of note-taking, I finally said goodbye to Evernote. Here’s the technical journey of migrating 4,330 notes—with all their attachments, tables, and formatting—to Apple Notes.</p>  <h2 id="the-problem">The Problem</h2> <p>Evernote had been my digital brain since the late 2000s. But with each passing version, the app became slower, more bloated, and increasingly expensive. Apple Notes, meanwhile, has quietly evolved into a capable, fast, and free alternative that syncs seamlessly across my devices.</p> <p>The catch? <strong>There’s no official migration path.</strong> Evernote’s export format (ENEX) doesn’t preserve everything, and Apple Notes doesn’t have any bulk import feature. Manual copy-paste wasn’t an option.</p> <p>So I built my own migration tool.</p> <h2 id="what-made-this-hard">What Made This Hard</h2> <p>This wasn’t a simple file conversion:</p> <ul> <li><strong>Rich text formatting</strong> including tables, checklists, and styled text</li> <li><strong>Embedded attachments</strong> (images, PDFs, documents) referenced by MD5 hashes in Evernote’s proprietary ENML format</li> <li><strong>Creation and modification dates</strong> that needed to be preserved</li> <li><strong>Duplicate detection</strong> to allow resumable, interruptible migrations</li> <li><strong>Apple Notes’ limitations</strong>—no public API, only AppleScript access</li> </ul> <p>Evernote v10 made things even more complicated. Unlike older versions that stored everything in a straightforward SQLite database, v10 uses a hybrid system with:</p> <ul> <li>A SQLite database for metadata</li> <li>Separate <code class="language-plaintext highlighter-rouge">.dat</code> files containing rich text content (tables/formatting)</li> <li>Protobuf-encoded binary structures</li> <li>Server-side attachment storage requiring authenticated downloads</li> </ul> <h2 id="the-solution-a-two-phase-migration-system">The Solution: A Two-Phase Migration System</h2> <p>I built a Python-based migration pipeline that handles all of this complexity.</p> <h3 id="phase-1-parallel-preparation">Phase 1: Parallel Preparation</h3> <p>The first phase downloads attachments and generates PDFs in parallel using 10 worker threads. For notes with embedded images or files, I render the complete content (HTML + attachments) into a PDF using headless Chrome. This preserves formatting perfectly.</p> <h3 id="phase-2-sequential-import">Phase 2: Sequential Import</h3> <p>The second phase imports to Apple Notes via AppleScript—sequentially, because Apple Notes doesn’t handle concurrent modifications well.</p> <h3 id="solving-the-attachment-problem">Solving the Attachment Problem</h3> <p>Evernote embeds attachments using <code class="language-plaintext highlighter-rouge"><en-media></code> tags with MD5 hashes. To resolve these to actual files, I:</p> <ol> <li>Query Evernote’s local database for attachment metadata</li> <li>Download from Evernote’s servers using captured auth tokens</li> <li>Embed them as base64 in generated PDFs</li> <li>Attach the PDF to the Apple Notes entry</li> </ol> <h3 id="deduplication-done-right">Deduplication Done Right</h3> <p>My initial attempt at duplicate detection was fragile—comparing dates via AppleScript often failed. The fix was simple: track Evernote note IDs in a log file. This makes the migration <strong>fully resumable</strong>.</p> <h2 id="bonus-ai-powered-organization">Bonus: AI-Powered Organization</h2> <p>Once notes were in Apple Notes, I used Gemini AI to automatically categorize them into folders based on content.</p> <h2 id="lessons-learned">Lessons Learned</h2> <ol> <li> <p><strong>AppleScript is slow but reliable</strong> — Building a cache at startup dropped duplicate checks from 0.5s to 0.001s per note.</p> </li> <li> <p><strong>Parallelism for I/O, sequential for mutations</strong> — Downloading attachments scales linearly with workers. Writing to Apple Notes must be sequential.</p> </li> <li> <p><strong>Auth tokens expire</strong> — Evernote’s tokens last about an hour. I kept Proxyman ready to capture fresh tokens.</p> </li> <li> <p><strong>PDF is a universal container</strong> — When your target doesn’t support rich formatting or attachments, bundle everything into a PDF.</p> </li> </ol> <h2 id="the-code">The Code</h2> <p>The entire migration toolkit is available on GitHub: <a href="https://github.com/Jeswang/apple-notes-toolkit">apple-notes-toolkit</a></p> <p>⚠️ Note: This repo is fully vibe coded. Use with caution.</p> <h2 id="final-thoughts">Final Thoughts</h2> <p>What started as a weekend project turned into a deep dive into Evernote’s internals, Apple’s Scripting Bridge, and the art of data migration. But the result is worth it: my 15 years of notes are now in Apple Notes, fully searchable, syncing across devices, and—most importantly—mine to keep.</p> <p>If you’re considering leaving Evernote, know that it’s possible. It just takes a bit of engineering.</p>

2026/1/16

《世上为什么要有图书馆》读书笔记

<p>最近读到的一本文字流畅，内容清爽的小书。书里描述了大学教授杨素秋，在西安市碑林区文化旅游局挂职一年，筹办区图书馆的经历。这是个繁杂、具体，有时甚至需要挑战权威的工作：</p> <ul> <li>区里提供的馆址是个地下空间，需要在有限的预算内，找到合适的装修公司，把这个地下空间改造成舒适的阅读空间。</li> <li>在图书采购过程中，供应商惯于提供劣质的，滥竽充数的图书，为采购者支付回扣。作者不屑于收受回扣，一心为公，希望图书馆里都是经历了时间检验的好书。</li> <li>为一个图书馆选书，工程浩大，无法仅凭一己之力完成。作者发动自己的人脉，联系了诸多好友帮忙选书。选书缘由，荐者心路，作者缓缓道来，推卷而述，好不痛快。</li> </ul> <p>尽管困难重重，作者心有所往，逆流而上，不畏险阻，最终得偿所愿。主线之余，作者夹叙一年挂职生活所遇的形色人等，有的让人牙关紧咬，有的让人唏嘘感慨，说尽人情冷暖。西安的美食，官场中的本色不改，选书朋友们的人生故事，对弱势群体的关照，五味杂陈，乒乓作响，读者吃到的是酸辣爽口的一餐。</p>  <h1 id="附录里的书单">附录里的书单</h1> <h3 id="童书含漫画">童书（含漫画）</h3> <table> <thead> <tr> <th style="text-align: left">书名</th> <th style="text-align: left">作者</th> <th style="text-align: left">出版年份</th> <th style="text-align: left">豆瓣评分</th> <th style="text-align: left">豆瓣链接</th> </tr> </thead> <tbody> <tr> <td style="text-align: left">《安徒生童话》</td> <td style="text-align: left">[丹麦] 汉斯·克里斯蒂安·安徒生</td> <td style="text-align: left">1835年</td> <td style="text-align: left">9.2</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E5%AE%89%E5%BE%92%E7%94%9F%E7%AB%A5%E8%AF%9D">链接</a></td> </tr> <tr> <td style="text-align: left">《镖人》</td> <td style="text-align: left">许先哲</td> <td style="text-align: left">2015年</td> <td style="text-align: left">9.0</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E9%95%96%E4%BA%BA">链接</a></td> </tr> <tr> <td style="text-align: left">《冰菓》</td> <td style="text-align: left">[日] 米澤穂信</td> <td style="text-align: left">2001年</td> <td style="text-align: left">8.6</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E5%86%B0%E8%8F%93">链接</a></td> </tr> <tr> <td style="text-align: left">《查理和巧克力工厂》</td> <td style="text-align: left">[英] 罗尔德·达尔</td> <td style="text-align: left">1964年</td> <td style="text-align: left">8.9</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E6%9F%A5%E7%90%86%E5%92%8C%E5%B7%A7%E5%85%8B%E5%8A%9B%E5%B7%A5%E5%8E%82">链接</a></td> </tr> <tr> <td style="text-align: left">《虫师》</td> <td style="text-align: left">[日] 漆原友纪</td> <td style="text-align: left">1999年</td> <td style="text-align: left">9.4</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E8%99%AB%E5%B8%88">链接</a></td> </tr> <tr> <td style="text-align: left">《宝可梦（宠物小精灵）》</td> <td style="text-align: left">[日] 日下秀宪 / 真斗</td> <td style="text-align: left">1997年</td> <td style="text-align: left">9.0</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E5%AE%9D%E5%8F%AF%E6%A2%A6%EF%BC%88%E5%AE%A0%E7%89%A9%E5%B0%8F%E7%B2%BE%E7%81%B5%EF%BC%89">链接</a></td> </tr> <tr> <td style="text-align: left">《窗边的小豆豆》</td> <td style="text-align: left">[日] 黑柳彻子</td> <td style="text-align: left">1981年</td> <td style="text-align: left">8.8</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E7%AA%97%E8%BE%B9%E7%9A%84%E5%B0%8F%E8%B1%86%E8%B1%86">链接</a></td> </tr> <tr> <td style="text-align: left">《吹小号的天鹅》</td> <td style="text-align: left">[美] E.B. 怀特</td> <td style="text-align: left">1970年</td> <td style="text-align: left">8.9</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E5%90%B9%E5%B0%8F%E5%8F%B7%E7%9A%84%E5%A4%A9%E9%B9%85">链接</a></td> </tr> <tr> <td style="text-align: left">《丁丁历险记》</td> <td style="text-align: left">[比利时] 埃尔热</td> <td style="text-align: left">1929年</td> <td style="text-align: left">9.4</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E4%B8%81%E4%B8%81%E5%8E%86%E9%99%A9%E8%AE%B0">链接</a></td> </tr> <tr> <td style="text-align: left">《机动战士高达》</td> <td style="text-align: left">[日] 富野由悠季 / 矢立肇</td> <td style="text-align: left">1979年</td> <td style="text-align: left">9.2</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E6%9C%BA%E5%8A%A8%E6%88%98%E5%A3%AB%E9%AB%98%E8%BE%BE">链接</a></td> </tr> <tr> <td style="text-align: left">《给孩子的故事》</td> <td style="text-align: left">黄永玉</td> <td style="text-align: left">2015年</td> <td style="text-align: left">8.2</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E7%BB%99%E5%AD%A9%E5%AD%90%E7%9A%84%E6%95%85%E4%BA%8B">链接</a></td> </tr> <tr> <td style="text-align: left">《灌篮高手》</td> <td style="text-align: left">[日] 井上雄彦</td> <td style="text-align: left">1990年</td> <td style="text-align: left">9.7</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E7%81%8C%E7%AF%AE%E9%AB%98%E6%89%8B">链接</a></td> </tr> <tr> <td style="text-align: left">《哈利·波特》</td> <td style="text-align: left">[英] J.K. 罗琳</td> <td style="text-align: left">1997年</td> <td style="text-align: left">9.2</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E5%93%88%E5%88%A9%C2%B7%E6%B3%A2%E7%89%B9">链接</a></td> </tr> <tr> <td style="text-align: left">《海贼王》</td> <td style="text-align: left">[日] 尾田荣一郎</td> <td style="text-align: left">1997年</td> <td style="text-align: left">9.6</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E6%B5%B7%E8%B4%BC%E7%8E%8B">链接</a></td> </tr> <tr> <td style="text-align: left">《汉声中国童话》</td> <td style="text-align: left">汉声杂志社</td> <td style="text-align: left">1982年</td> <td style="text-align: left">9.5</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E6%B1%89%E5%A3%B0%E4%B8%AD%E5%9B%BD%E7%AB%A5%E8%AF%9D">链接</a></td> </tr> <tr> <td style="text-align: left">《荷花镇的早市》</td> <td style="text-align: left">周翔</td> <td style="text-align: left">2014年</td> <td style="text-align: left">8.8</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E8%8D%B7%E8%8A%B1%E9%95%87%E7%9A%84%E6%97%A9%E5%B8%82">链接</a></td> </tr> <tr> <td style="text-align: left">《黑子的篮球》</td> <td style="text-align: left">[日] 藤卷忠俊</td> <td style="text-align: left">2008年</td> <td style="text-align: left">8.1</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E9%BB%91%E5%AD%90%E7%9A%84%E7%AF%AE%E7%90%83">链接</a></td> </tr> <tr> <td style="text-align: left">《护生画集》</td> <td style="text-align: left">丰子恺 / 弘一法师</td> <td style="text-align: left">1929年</td> <td style="text-align: left">9.4</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E6%8A%A4%E7%94%9F%E7%94%BB%E9%9B%86">链接</a></td> </tr> <tr> <td style="text-align: left">《火影忍者》</td> <td style="text-align: left">[日] 岸本齐史</td> <td style="text-align: left">1999年</td> <td style="text-align: left">9.3</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E7%81%AB%E5%BD%B1%E5%BF%8D%E8%80%85">链接</a></td> </tr> <tr> <td style="text-align: left">《精灵鼠小弟》</td> <td style="text-align: left">[美] E.B. 怀特</td> <td style="text-align: left">1945年</td> <td style="text-align: left">8.6</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E7%B2%BE%E7%81%B5%E9%BC%A0%E5%B0%8F%E5%BC%9F">链接</a></td> </tr> <tr> <td style="text-align: left">《可怕的科学》</td> <td style="text-align: left">[英] 尼克·阿诺德</td> <td style="text-align: left">1996年</td> <td style="text-align: left">9.3</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E5%8F%AF%E6%80%95%E7%9A%84%E7%A7%91%E5%AD%A6">链接</a></td> </tr> <tr> <td style="text-align: left">《拉比的猫》</td> <td style="text-align: left">[法] 尤安·斯法</td> <td style="text-align: left">2002年</td> <td style="text-align: left">8.8</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E6%8B%89%E6%AF%94%E7%9A%84%E7%8C%AB">链接</a></td> </tr> <tr> <td style="text-align: left">《了不起的狐狸爸爸》</td> <td style="text-align: left">[英] 罗尔德·达尔</td> <td style="text-align: left">1970年</td> <td style="text-align: left">8.8</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E4%BA%86%E4%B8%8D%E8%B5%B7%E7%9A%84%E7%8B%90%E7%8B%B8%E7%88%B8%E7%88%B8">链接</a></td> </tr> <tr> <td style="text-align: left">《龙珠Z》 (漫画原作)</td> <td style="text-align: left">[日] 鸟山明</td> <td style="text-align: left">1984年</td> <td style="text-align: left">9.7</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E9%BE%99%E7%8F%A0Z">链接</a></td> </tr> <tr> <td style="text-align: left">《玛蒂尔达》</td> <td style="text-align: left">[英] 罗尔德·达尔</td> <td style="text-align: left">1988年</td> <td style="text-align: left">9.1</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E7%8E%9B%E8%92%82%E5%B0%94%E8%BE%BE">链接</a></td> </tr> <tr> <td style="text-align: left">《玛法达》</td> <td style="text-align: left">[阿根廷] 季诺</td> <td style="text-align: left">1964年</td> <td style="text-align: left">9.4</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E7%8E%9B%E6%B3%95%E8%BE%BE">链接</a></td> </tr> <tr> <td style="text-align: left">《名侦探柯南》</td> <td style="text-align: left">[日] 青山刚昌</td> <td style="text-align: left">1994年</td> <td style="text-align: left">9.3</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E5%90%8D%E4%BE%A6%E6%8E%A2%E6%9F%AF%E5%8D%97">链接</a></td> </tr> <tr> <td style="text-align: left">《排球少年》</td> <td style="text-align: left">[日] 古馆春一</td> <td style="text-align: left">2012年</td> <td style="text-align: left">9.7</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E6%8E%92%E7%90%83%E5%B0%91%E5%B9%B4">链接</a></td> </tr> <tr> <td style="text-align: left">《七龙珠》</td> <td style="text-align: left">[日] 鸟山明</td> <td style="text-align: left">1984年</td> <td style="text-align: left">9.7</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E4%B8%83%E9%BE%99%E7%8F%A0">链接</a></td> </tr> <tr> <td style="text-align: left">《棋魂》</td> <td style="text-align: left">[日] 堀田由美 / 小畑健</td> <td style="text-align: left">1999年</td> <td style="text-align: left">9.5</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E6%A3%8B%E9%AD%82">链接</a></td> </tr> <tr> <td style="text-align: left">《犬夜叉》</td> <td style="text-align: left">[日] 高桥留美子</td> <td style="text-align: left">1996年</td> <td style="text-align: left">9.1</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E7%8A%AC%E5%A4%9C%E5%8F%89">链接</a></td> </tr> <tr> <td style="text-align: left">《三毛流浪记》</td> <td style="text-align: left">张乐平</td> <td style="text-align: left">1947年</td> <td style="text-align: left">9.1</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E4%B8%89%E6%AF%9B%E6%B5%81%E6%B5%AA%E8%AE%B0">链接</a></td> </tr> <tr> <td style="text-align: left">《圣斗士星矢》</td> <td style="text-align: left">[日] 车田正美</td> <td style="text-align: left">1986年</td> <td style="text-align: left">9.2</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E5%9C%A3%E6%96%97%E5%A3%AB%E6%98%9F%E7%9F%A2">链接</a></td> </tr> <tr> <td style="text-align: left">《死神》 (BLEACH)</td> <td style="text-align: left">[日] 久保带人</td> <td style="text-align: left">2001年</td> <td style="text-align: left">9.0</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E6%AD%BB%E7%A5%9E">链接</a></td> </tr> <tr> <td style="text-align: left">《死亡笔记》</td> <td style="text-align: left">[日] 大场鸫 / 小畑健</td> <td style="text-align: left">2003年</td> <td style="text-align: left">9.2</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E6%AD%BB%E4%BA%A1%E7%AC%94%E8%AE%B0">链接</a></td> </tr> <tr> <td style="text-align: left">《四月是你的谎言》</td> <td style="text-align: left">[日] 新川直司</td> <td style="text-align: left">2011年</td> <td style="text-align: left">8.7</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E5%9B%9B%E6%9C%88%E6%98%AF%E4%BD%A0%E7%9A%84%E8%B0%8E%E8%A8%80">链接</a></td> </tr> <tr> <td style="text-align: left">《太空》</td> <td style="text-align: left">[美] H.A. 雷</td> <td style="text-align: left">1957年</td> <td style="text-align: left">9.1</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E5%A4%AA%E7%A9%BA">链接</a></td> </tr> <tr> <td style="text-align: left">《网球王子》</td> <td style="text-align: left">[日] 许斐刚</td> <td style="text-align: left">1999年</td> <td style="text-align: left">8.8</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E7%BD%91%E7%90%83%E7%8E%8B%E5%AD%90">链接</a></td> </tr> <tr> <td style="text-align: left">《文豪野犬》</td> <td style="text-align: left">[日] 朝雾卡夫卡 / 春河35</td> <td style="text-align: left">2012年</td> <td style="text-align: left">8.4</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E6%96%87%E8%B1%AA%E9%87%8E%E7%8A%AC">链接</a></td> </tr> <tr> <td style="text-align: left">《希利尔讲艺术史》</td> <td style="text-align: left">[美] V.M. 希利尔</td> <td style="text-align: left">1924年</td> <td style="text-align: left">8.8</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E5%B8%8C%E5%88%A9%E5%B0%94%E8%AE%B2%E8%89%BA%E6%9C%AF%E5%8F%B2">链接</a></td> </tr> <tr> <td style="text-align: left">《夏洛的网》</td> <td style="text-align: left">[美] E.B. 怀特</td> <td style="text-align: left">1952年</td> <td style="text-align: left">8.6</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E5%A4%8F%E6%B4%9B%E7%9A%84%E7%BD%91">链接</a></td> </tr> <tr> <td style="text-align: left">《夏目友人帐》</td> <td style="text-align: left">[日] 绿川幸</td> <td style="text-align: left">2005年</td> <td style="text-align: left">9.4</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E5%A4%8F%E7%9B%AE%E5%8F%8B%E4%BA%BA%E5%B8%90">链接</a></td> </tr> <tr> <td style="text-align: left">《写给孩子的哲学启蒙书》</td> <td style="text-align: left">[法] 布里吉特·拉贝等</td> <td style="text-align: left">2001年</td> <td style="text-align: left">8.8</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E5%86%99%E7%BB%99%E5%AD%A9%E5%AD%90%E7%9A%84%E5%93%B2%E5%AD%A6%E5%90%AF%E8%92%99%E4%B9%A6">链接</a></td> </tr> <tr> <td style="text-align: left">《银魂》</td> <td style="text-align: left">[日] 空知英秋</td> <td style="text-align: left">2003年</td> <td style="text-align: left">9.5</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E9%93%B6%E9%AD%82">链接</a></td> </tr> <tr> <td style="text-align: left">《幽游白书》</td> <td style="text-align: left">[日] 冨㭴义博</td> <td style="text-align: left">1990年</td> <td style="text-align: left">9.5</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E5%B9%BD%E6%B8%B8%E7%99%BD%E4%B9%A6">链接</a></td> </tr> <tr> <td style="text-align: left">《月刊少女野崎君》</td> <td style="text-align: left">[日] 椿泉</td> <td style="text-align: left">2011年</td> <td style="text-align: left">9.2</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E6%9C%88%E5%88%8A%E5%B1%91%E5%A5%B3%E9%87%8E%E5%B4%8E%E5%90%9B">链接</a></td> </tr> </tbody> </table> <h3 id="文学类">文学类</h3> <table> <thead> <tr> <th style="text-align: left">书名</th> <th style="text-align: left">作者</th> <th style="text-align: left">出版年份</th> <th style="text-align: left">豆瓣评分</th> <th style="text-align: left">豆瓣链接</th> </tr> </thead> <tbody> <tr> <td style="text-align: left">《奥德赛》</td> <td style="text-align: left">[古希腊] 荷马</td> <td style="text-align: left">公元前8世纪</td> <td style="text-align: left">8.7</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E5%A5%A5%E5%BE%B7%E8%B5%9B">链接</a></td> </tr> <tr> <td style="text-align: left">《白鹿原》</td> <td style="text-align: left">陈忠实</td> <td style="text-align: left">1993年</td> <td style="text-align: left">9.3</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E7%99%BD%E9%B9%BF%E5%8E%9F">链接</a></td> </tr> <tr> <td style="text-align: left">《冰与火之歌》</td> <td style="text-align: left">[美] 乔治·R.R. 马丁</td> <td style="text-align: left">1996年</td> <td style="text-align: left">9.4</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E5%86%B0%E4%B8%8E%E7%81%AB%E4%B9%8B%E6%AD%8C">链接</a></td> </tr> <tr> <td style="text-align: left">《查令十字街84号》</td> <td style="text-align: left">[美] 海莲·汉芙</td> <td style="text-align: left">1970年</td> <td style="text-align: left">8.5</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E6%9F%A5%E4%BB%A4%E5%8D%81%E5%AD%97%E8%A1%9784%E5%8F%B7">链接</a></td> </tr> <tr> <td style="text-align: left">《传习录》</td> <td style="text-align: left">王阳明</td> <td style="text-align: left">约1518年</td> <td style="text-align: left">9.1</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E4%BC%A0%E4%B9%A0%E5%BD%95">链接</a></td> </tr> <tr> <td style="text-align: left">《东周列国志》</td> <td style="text-align: left">[明] 冯梦龙</td> <td style="text-align: left">约1620年代</td> <td style="text-align: left">9.3</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E4%B8%9C%E5%91%A8%E5%88%97%E5%9B%BD%E5%BF%97">链接</a></td> </tr> <tr> <td style="text-align: left">《读库》</td> <td style="text-align: left">张立宪 (主编)</td> <td style="text-align: left">2006年</td> <td style="text-align: left">9.3</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E8%AF%BB%E5%BA%93">链接</a></td> </tr> <tr> <td style="text-align: left">《儿女英雄传》</td> <td style="text-align: left">[清] 文康</td> <td style="text-align: left">约1878年</td> <td style="text-align: left">7.6</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E5%84%BF%E5%A5%B3%E8%8B%B1%E9%9B%84%E4%BC%A0">链接</a></td> </tr> <tr> <td style="text-align: left">《反骨仔》</td> <td style="text-align: left">王朔</td> <td style="text-align: left">2007年</td> <td style="text-align: left">7.0</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E5%8F%8D%E9%AA%A8%E4%BB%94">链接</a></td> </tr> <tr> <td style="text-align: left">《废都》</td> <td style="text-align: left">贾平凹</td> <td style="text-align: left">1993年</td> <td style="text-align: left">8.2</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E5%BA%9F%E9%83%BD">链接</a></td> </tr> <tr> <td style="text-align: left">《古文观止》</td> <td style="text-align: left">[清] 吴楚材 / 吴调侯</td> <td style="text-align: left">1695年</td> <td style="text-align: left">9.4</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E5%8F%A4%E6%96%87%E8%A7%82%E6%AD%A2">链接</a></td> </tr> <tr> <td style="text-align: left">《哈克贝利·费恩历险记》</td> <td style="text-align: left">[美] 马克·吐温</td> <td style="text-align: left">1884年</td> <td style="text-align: left">8.7</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E5%93%88%E5%85%8B%E8%B4%9D%E5%88%A9%C2%B7%E8%B4%B9%E6%81%A9%E5%8E%86%E9%99%A9%E8%AE%B0">链接</a></td> </tr> <tr> <td style="text-align: left">《海边的卡夫卡》</td> <td style="text-align: left">[日] 村上春树</td> <td style="text-align: left">2002年</td> <td style="text-align: left">8.2</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E6%B5%B7%E8%BE%B9%E7%9A%84%E5%8D%A1%E5%A4%AB%E5%8D%A1">链接</a></td> </tr> <tr> <td style="text-align: left">《海底两万里》</td> <td style="text-align: left">[法] 儒勒·凡尔纳</td> <td style="text-align: left">1870年</td> <td style="text-align: left">8.6</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E6%B5%B7%E5%BA%95%E4%B8%A4%E4%B8%87%E9%87%8C">链接</a></td> </tr> <tr> <td style="text-align: left">《汉字王国》</td> <td style="text-align: left">[瑞典] 林西莉</td> <td style="text-align: left">1989年</td> <td style="text-align: left">9.0</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E6%B1%89%E5%AD%97%E7%8E%8B%E5%9B%BD">链接</a></td> </tr> <tr> <td style="text-align: left">《红楼梦》</td> <td style="text-align: left">[清] 曹雪芹</td> <td style="text-align: left">约1791年</td> <td style="text-align: left">9.6</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E7%BA%A2%E6%A5%BC%E6%A2%A6">链接</a></td> </tr> <tr> <td style="text-align: left">《活着》</td> <td style="text-align: left">余华</td> <td style="text-align: left">1993年</td> <td style="text-align: left">9.4</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E6%B4%BB%E7%9D%80">链接</a></td> </tr> <tr> <td style="text-align: left">《基督山伯爵》</td> <td style="text-align: left">[法] 大仲马</td> <td style="text-align: left">1844年</td> <td style="text-align: left">9.2</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E5%9F%BA%E7%9D%A3%E5%B1%B1%E4%BC%AF%E7%88%B5">链接</a></td> </tr> <tr> <td style="text-align: left">《卡拉马佐夫兄弟》</td> <td style="text-align: left">[俄] 陀思妥耶夫斯基</td> <td style="text-align: left">1880年</td> <td style="text-align: left">9.7</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E5%8D%A1%E6%8B%89%E9%A9%AC%E4%BD%90%E5%A4%AB%E5%85%84%E5%BC%9F">链接</a></td> </tr> <tr> <td style="text-align: left">《克林索尔的最后夏天》</td> <td style="text-align: left">[德] 赫尔曼·黑塞</td> <td style="text-align: left">1920年</td> <td style="text-align: left">8.8</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E5%85%8B%E6%9E%97%E7%B4%A2%E5%B0%94%E7%9A%84%E6%9C%80%E5%90%8E%E5%A4%8F%E5%A4%A9">链接</a></td> </tr> <tr> <td style="text-align: left">《老人与海》</td> <td style="text-align: left">[美] 欧内斯特·海明威</td> <td style="text-align: left">1952年</td> <td style="text-align: left">8.5</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E8%80%81%E4%BA%BA%E4%B8%8E%E6%B5%B7">链接</a></td> </tr> <tr> <td style="text-align: left">《礼物》</td> <td style="text-align: left">[美] 弗拉基米尔·纳博科夫</td> <td style="text-align: left">1938年</td> <td style="text-align: left">8.8</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E7%A4%BC%E7%89%A9">链接</a></td> </tr> <tr> <td style="text-align: left">《裂缝》</td> <td style="text-align: left">[英] 多丽丝·莱辛</td> <td style="text-align: left">2007年</td> <td style="text-align: left">7.9</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E8%A3%82%E7%BC%9D">链接</a></td> </tr> <tr> <td style="text-align: left">《流言》</td> <td style="text-align: left">张爱玲</td> <td style="text-align: left">1944年</td> <td style="text-align: left">8.8</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E6%B5%81%E8%A8%80">链接</a></td> </tr> <tr> <td style="text-align: left">《鲁滨孙漂流记》</td> <td style="text-align: left">[英] 丹尼尔·笛福</td> <td style="text-align: left">1719年</td> <td style="text-align: left">8.4</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E9%B2%81%E6%BB%A8%E5%AD%99%E6%BC%82%E6%B5%81%E8%AE%B0">链接</a></td> </tr> <tr> <td style="text-align: left">《鲁迅全集》</td> <td style="text-align: left">鲁迅</td> <td style="text-align: left">1938年</td> <td style="text-align: left">9.7</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E9%B2%81%E8%BF%85%E5%85%A8%E9%9B%86">链接</a></td> </tr> <tr> <td style="text-align: left">《论语》</td> <td style="text-align: left">孔子弟子及再传弟子</td> <td style="text-align: left">战国时期</td> <td style="text-align: left">9.4</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E8%AE%BA%E8%AF%AD">链接</a></td> </tr> <tr> <td style="text-align: left">《罗生门》</td> <td style="text-align: left">[日] 芥川龙之介</td> <td style="text-align: left">1915年</td> <td style="text-align: left">8.7</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E7%BD%97%E7%94%9F%E9%97%A8">链接</a></td> </tr> <tr> <td style="text-align: left">《麦田里的守望者》</td> <td style="text-align: left">[美] J.D. 塞林格</td> <td style="text-align: left">1951年</td> <td style="text-align: left">8.2</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E9%BA%A6%E7%94%B0%E9%87%8C%E7%9A%84%E5%AE%88%E6%9C%9B%E8%80%85">链接</a></td> </tr> <tr> <td style="text-align: left">《魔戒》</td> <td style="text-align: left">[英] J.R.R. 托尔金</td> <td style="text-align: left">1954年</td> <td style="text-align: left">9.4</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E9%AD%94%E6%88%92">链接</a></td> </tr> <tr> <td style="text-align: left">《墓法墓天》</td> <td style="text-align: left">不带剑</td> <td style="text-align: left">2017年</td> <td style="text-align: left">7.9</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E5%A2%93%E6%B3%95%E5%A2%93%E5%A4%A9">链接</a></td> </tr> <tr> <td style="text-align: left">《那不勒斯四部曲》</td> <td style="text-align: left">[意] 埃莱娜·费兰特</td> <td style="text-align: left">2011年</td> <td style="text-align: left">8.8</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E9%82%A3%E4%B8%8D%E5%8B%92%E6%96%AF%E5%9B%9B%E9%83%A8%E6%9B%B2">链接</a></td> </tr> <tr> <td style="text-align: left">《挪威的森林》</td> <td style="text-align: left">[日] 村上春树</td> <td style="text-align: left">1987年</td> <td style="text-align: left">8.1</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E6%8C%AA%E5%A8%81%E7%9A%84%E6%A3%AE%E6%9E%97">链接</a></td> </tr> <tr> <td style="text-align: left">《胚胎奇谭》</td> <td style="text-align: left">[英] 朱利安·巴恩斯</td> <td style="text-align: left">1984年</td> <td style="text-align: left">8.5</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E8%83%9A%E8%83%8E%E5%A5%87%E8%B0%AD">链接</a></td> </tr> <tr> <td style="text-align: left">《契诃夫文集》</td> <td style="text-align: left">[俄] 安东·巴甫洛维奇·契诃夫</td> <td style="text-align: left">19世纪末</td> <td style="text-align: left">9.6</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E5%A5%91%E8%AF%83%E5%A4%AB%E6%96%87%E9%9B%86">链接</a></td> </tr> <tr> <td style="text-align: left">《人间词话》</td> <td style="text-align: left">王国维</td> <td style="text-align: left">1910年</td> <td style="text-align: left">9.0</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E4%BA%BA%E9%97%B4%E8%AF%8D%E8%AF%9D">链接</a></td> </tr> <tr> <td style="text-align: left">《人间喜剧》</td> <td style="text-align: left">[法] 奥诺雷·德·巴尔扎克</td> <td style="text-align: left">1829-1848年</td> <td style="text-align: left">9.2</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E4%BA%BA%E9%97%B4%E5%96%9C%E5%89%A7">链接</a></td> </tr> <tr> <td style="text-align: left">《三国演义》</td> <td style="text-align: left">[明] 罗贯中</td> <td style="text-align: left">14世纪</td> <td style="text-align: left">9.2</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E4%B8%89%E5%9B%BD%E6%BC%94%E4%B9%89">链接</a></td> </tr> <tr> <td style="text-align: left">《三体》</td> <td style="text-align: left">刘慈欣</td> <td style="text-align: left">2006年</td> <td style="text-align: left">8.9</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E4%B8%89%E4%BD%93">链接</a></td> </tr> <tr> <td style="text-align: left">《诗的八堂课》</td> <td style="text-align: left">张晓风</td> <td style="text-align: left">2011年</td> <td style="text-align: left">8.3</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E8%AF%97%E7%9A%84%E5%85%AB%E5%A0%82%E8%AF%BE">链接</a></td> </tr> <tr> <td style="text-align: left">《诗歌手册》</td> <td style="text-align: left">[法] 保尔·瓦雷里</td> <td style="text-align: left">1942年</td> <td style="text-align: left">8.7</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E8%AF%97%E6%AD%8C%E6%89%8B%E5%86%8C">链接</a></td> </tr> <tr> <td style="text-align: left">《诗经》</td> <td style="text-align: left">佚名</td> <td style="text-align: left">公元前11-7世纪</td> <td style="text-align: left">9.0</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E8%AF%97%E7%BB%8F">链接</a></td> </tr> <tr> <td style="text-align: left">《史记》</td> <td style="text-align: left">[汉] 司马迁</td> <td style="text-align: left">约公元前94年</td> <td style="text-align: left">9.6</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E5%8F%B2%E8%AE%B0">链接</a></td> </tr> <tr> <td style="text-align: left">《世说新语》</td> <td style="text-align: left">[南朝宋] 刘义庆</td> <td style="text-align: left">约430年</td> <td style="text-align: left">9.1</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E4%B8%96%E8%AF%B4%E6%96%B0%E8%AF%AD">链接</a></td> </tr> <tr> <td style="text-align: left">《鼠疫》</td> <td style="text-align: left">[法] 阿尔贝·加缪</td> <td style="text-align: left">1947年</td> <td style="text-align: left">9.1</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E9%BC%A0%E7%96%AB">链接</a></td> </tr> <tr> <td style="text-align: left">《太平广记》</td> <td style="text-align: left">[宋] 李昉等</td> <td style="text-align: left">978年</td> <td style="text-align: left">9.5</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E5%A4%AA%E5%B9%B3%E5%B9%BF%E8%AE%B0">链接</a></td> </tr> <tr> <td style="text-align: left">《汤姆·索亚历险记》</td> <td style="text-align: left">[美] 马克·吐温</td> <td style="text-align: left">1876年</td> <td style="text-align: left">8.5</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E6%B1%A4%E5%A7%86%C2%B7%E7%B4%A2%E4%BA%9A%E5%8E%86%E9%99%A9%E8%AE%B0">链接</a></td> </tr> <tr> <td style="text-align: left">《唐诗别裁集》</td> <td style="text-align: left">[清] 沈德潜</td> <td style="text-align: left">1717年</td> <td style="text-align: left">9.0</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E5%94%90%E8%AF%97%E5%88%AB%E8%A3%81%E9%9B%86">链接</a></td> </tr> <tr> <td style="text-align: left">《唐诗三百首》</td> <td style="text-align: left">[清] 蘅塘退士</td> <td style="text-align: left">约1763年</td> <td style="text-align: left">9.2</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E5%94%90%E8%AF%97%E4%B8%89%E7%99%BE%E9%A6%96">链接</a></td> </tr> <tr> <td style="text-align: left">《天龙八部》</td> <td style="text-align: left">金庸</td> <td style="text-align: left">1963年</td> <td style="text-align: left">9.2</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E5%A4%A9%E9%BE%99%E5%85%AB%E9%83%A8">链接</a></td> </tr> <tr> <td style="text-align: left">《推拿》</td> <td style="text-align: left">毕飞宇</td> <td style="text-align: left">2008年</td> <td style="text-align: left">8.7</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E6%8E%A8%E6%8B%BF">链接</a></td> </tr> <tr> <td style="text-align: left">《文苑英华》</td> <td style="text-align: left">[宋] 李昉等</td> <td style="text-align: left">987年</td> <td style="text-align: left">9.7</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E6%96%87%E8%8B%91%E8%8B%B1%E5%8D%8E">链接</a></td> </tr> <tr> <td style="text-align: left">《我弥留之际》</td> <td style="text-align: left">[美] 威廉·福克纳</td> <td style="text-align: left">1930年</td> <td style="text-align: left">8.8</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E6%88%91%E5%BC%A5%E7%95%99%E4%B9%8B%E9%99%85">链接</a></td> </tr> <tr> <td style="text-align: left">《西南联大国文课》</td> <td style="text-align: left">闻一多 / 朱自清等</td> <td style="text-align: left">-</td> <td style="text-align: left">8.4</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E8%A5%BF%E5%8D%97%E8%81%94%E5%A4%A7%E5%9B%BD%E6%96%87%E8%AF%BE">链接</a></td> </tr> <tr> <td style="text-align: left">《献给阿尔吉侬的花束》</td> <td style="text-align: left">[美] 丹尼尔·凯斯</td> <td style="text-align: left">1966年</td> <td style="text-align: left">9.1</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E7%8C%AE%E7%BB%99%E9%98%BF%E5%B0%94%E5%90%89%E4%BE%AC%E7%9A%84%E8%8A%B1%E6%9D%9F">链接</a></td> </tr> <tr> <td style="text-align: left">《小城之恋》</td> <td style="text-align: left">[英] L.P. 哈特利</td> <td style="text-align: left">1953年</td> <td style="text-align: left">8.1</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E5%B0%8F%E5%9F%8E%E4%B9%8B%E6%81%8B">链接</a></td> </tr> <tr> <td style="text-align: left">《小说课》</td> <td style="text-align: left">毕飞宇</td> <td style="text-align: left">2017年</td> <td style="text-align: left">8.6</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E5%B0%8F%E8%AF%B4%E8%AF%BE">链接</a></td> </tr> <tr> <td style="text-align: left">《写作法宝》</td> <td style="text-align: left">[美] 斯蒂芬·金</td> <td style="text-align: left">2000年</td> <td style="text-align: left">8.9</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E5%86%99%E4%BD%9C%E6%B3%95%E5%AE%9D">链接</a></td> </tr> <tr> <td style="text-align: left">《伊利亚特》</td> <td style="text-align: left">[古希腊] 荷马</td> <td style="text-align: left">公元前8世纪</td> <td style="text-align: left">8.8</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E4%BC%8A%E5%88%A9%E4%BA%9A%E7%89%B9">链接</a></td> </tr> <tr> <td style="text-align: left">《阴阳师》</td> <td style="text-align: left">[日] 梦枕貘</td> <td style="text-align: left">1986年</td> <td style="text-align: left">8.6</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E9%98%B4%E9%98%B3%E5%B8%88">链接</a></td> </tr> <tr> <td style="text-align: left">《银河帝国》</td> <td style="text-align: left">[美] 艾萨克·阿西莫夫</td> <td style="text-align: left">1951年</td> <td style="text-align: left">9.4</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E9%93%B6%E6%B2%B3%E5%B8%9D%E5%9B%BD">链接</a></td> </tr> <tr> <td style="text-align: left">《酉阳杂俎》</td> <td style="text-align: left">[唐] 段成式</td> <td style="text-align: left">9世纪</td> <td style="text-align: left">9.2</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E9%85%89%E9%98%B3%E6%9D%82%E9%98%BB">链接</a></td> </tr> <tr> <td style="text-align: left">《战国争鸣记》</td> <td style="text-align: left">[日] 宫崎市定</td> <td style="text-align: left">1947年</td> <td style="text-align: left">8.5</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E6%88%98%E5%9B%BD%E4%BA%89%E9%B8%A3%E8%AE%B0">链接</a></td> </tr> <tr> <td style="text-align: left">《朝花夕拾》</td> <td style="text-align: left">鲁迅</td> <td style="text-align: left">1928年</td> <td style="text-align: left">8.8</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E6%9C%9D%E8%8A%B1%E5%A4%95%E6%8B%BE">链接</a></td> </tr> <tr> <td style="text-align: left">《正常人》</td> <td style="text-align: left">[爱尔兰] 萨莉·鲁尼</td> <td style="text-align: left">2018年</td> <td style="text-align: left">8.0</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E6%AD%A3%E5%B8%B8%E4%BA%BA">链接</a></td> </tr> <tr> <td style="text-align: left">《纸牌屋》</td> <td style="text-align: left">[英] 迈克尔·多布斯</td> <td style="text-align: left">1989年</td> <td style="text-align: left">8.6</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E7%BA%B8%E7%89%8C%E5%B1%8B">链接</a></td> </tr> <tr> <td style="text-align: left">《最后一个匈奴》</td> <td style="text-align: left">高建群</td> <td style="text-align: left">1993年</td> <td style="text-align: left">8.1</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E6%9C%80%E5%90%8E%E4%B8%80%E4%B8%AA%E5%8C%88%E5%A5%B4">链接</a></td> </tr> <tr> <td style="text-align: left">《左传》</td> <td style="text-align: left">[春秋] 左丘明 (传)</td> <td style="text-align: left">战国时期</td> <td style="text-align: left">9.4</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E5%B7%A6%E4%BC%A0">链接</a></td> </tr> <tr> <td style="text-align: left">《作文七巧》</td> <td style="text-align: left">夏丏尊 / 叶圣陶</td> <td style="text-align: left">1980年</td> <td style="text-align: left">8.0</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E4%BD%9C%E6%96%87%E4%B8%83%E5%B7%A7">链接</a></td> </tr> </tbody> </table> <h3 id="人文社科">人文社科</h3> <table> <thead> <tr> <th style="text-align: left">书名</th> <th style="text-align: left">作者</th> <th style="text-align: left">出版年份</th> <th style="text-align: left">豆瓣评分</th> <th style="text-align: left">豆瓣链接</th> </tr> </thead> <tbody> <tr> <td style="text-align: left">《1844年经济学哲学手稿》</td> <td style="text-align: left">[德] 卡尔·马克思</td> <td style="text-align: left">1932年</td> <td style="text-align: left">9.2</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+1844%E5%B9%B4%E7%BB%8F%E6%B5%8E%E5%AD%A6%E5%93%B2%E5%AD%A6%E6%89%8B%E7%A8%BF">链接</a></td> </tr> <tr> <td style="text-align: left">《奥斯威辛：一部历史》</td> <td style="text-align: left">[英] 劳伦斯·里斯</td> <td style="text-align: left">2005年</td> <td style="text-align: left">9.3</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E5%A5%A5%E6%96%AF%E5%A8%81%E8%BE%9B%EF%BC%9A%E4%B8%80%E9%83%A8%E5%8E%86%E5%8F%B2">链接</a></td> </tr> <tr> <td style="text-align: left">《奥义书》</td> <td style="text-align: left">佚名</td> <td style="text-align: left">公元前800-500年</td> <td style="text-align: left">9.1</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E5%A5%A5%E4%B9%89%E4%B9%A6">链接</a></td> </tr> <tr> <td style="text-align: left">《巴尔扎克传》</td> <td style="text-align: left">[奥] 斯蒂芬·茨威格</td> <td style="text-align: left">1946年</td> <td style="text-align: left">9.1</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E5%B7%B4%E5%B0%94%E6%89%8E%E5%85%8B%E4%BC%A0">链接</a></td> </tr> <tr> <td style="text-align: left">《保卫马克思》</td> <td style="text-align: left">[法] 路易·阿尔都塞</td> <td style="text-align: left">1965年</td> <td style="text-align: left">8.8</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E4%BF%9D%E5%8D%AB%E9%A9%AC%E5%85%8B%E6%80%9D">链接</a></td> </tr> <tr> <td style="text-align: left">《藏在碑林里的国宝》</td> <td style="text-align: left">郭志呈 / 郭强</td> <td style="text-align: left">2019年</td> <td style="text-align: left">8.5</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E8%97%8F%E5%9C%A8%E7%A2%91%E6%9E%97%E9%87%8C%E7%9A%84%E5%9B%BD%E5%AE%9D">链接</a></td> </tr> <tr> <td style="text-align: left">《册府元龟》</td> <td style="text-align: left">[宋] 王钦若 / 杨亿</td> <td style="text-align: left">1013年</td> <td style="text-align: left">9.8</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E5%86%8C%E5%BA%9C%E5%85%83%E9%BE%9F">链接</a></td> </tr> <tr> <td style="text-align: left">《纯粹理性批判》</td> <td style="text-align: left">[德] 伊曼努尔·康德</td> <td style="text-align: left">1781年</td> <td style="text-align: left">9.2</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E7%BA%AF%E7%B2%B9%E7%90%86%E6%80%A7%E6%89%B9%E5%88%A4">链接</a></td> </tr> <tr> <td style="text-align: left">《丛书集成》</td> <td style="text-align: left">王云五 (主编)</td> <td style="text-align: left">1935年</td> <td style="text-align: left">9.7</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E4%B8%9B%E4%B9%A6%E9%9B%86%E6%88%90">链接</a></td> </tr> <tr> <td style="text-align: left">《大藏经》</td> <td style="text-align: left">历代高僧</td> <td style="text-align: left">历代</td> <td style="text-align: left">9.8</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E5%A4%A7%E8%97%8F%E7%BB%8F">链接</a></td> </tr> <tr> <td style="text-align: left">《抵抗的群体》</td> <td style="text-align: left">[美] 王人英</td> <td style="text-align: left">2011年</td> <td style="text-align: left">8.8</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E6%8A%B5%E6%8A%97%E7%9A%84%E7%BE%A4%E4%BD%93">链接</a></td> </tr> <tr> <td style="text-align: left">《第二性》</td> <td style="text-align: left">[法] 西蒙·娜·德·波伏娃</td> <td style="text-align: left">1949年</td> <td style="text-align: left">8.8</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E7%AC%AC%E4%BA%8C%E6%80%A7">链接</a></td> </tr> <tr> <td style="text-align: left">《洞穴奇案》</td> <td style="text-align: left">[美] 彼得·萨伯</td> <td style="text-align: left">1998年</td> <td style="text-align: left">9.4</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E6%B4%9E%E7%A9%B4%E5%A5%87%E6%A1%88">链接</a></td> </tr> <tr> <td style="text-align: left">《对影胡说》</td> <td style="text-align: left">胡兰成</td> <td style="text-align: left">1980年</td> <td style="text-align: left">7.2</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E5%AF%B9%E5%BD%B1%E8%83%A1%E8%AF%B4">链接</a></td> </tr> <tr> <td style="text-align: left">《二十四史》</td> <td style="text-align: left">历代史学家</td> <td style="text-align: left">历代</td> <td style="text-align: left">9.7</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E4%BA%8C%E5%8D%81%E5%9B%9B%E5%8F%B2">链接</a></td> </tr> <tr> <td style="text-align: left">《二手时间》</td> <td style="text-align: left">[白俄] S.A.阿列克谢耶维奇</td> <td style="text-align: left">2013年</td> <td style="text-align: left">9.2</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E4%BA%8C%E6%89%8B%E6%97%B6%E9%97%B4">链接</a></td> </tr> <tr> <td style="text-align: left">《佛家名相通释》</td> <td style="text-align: left">熊十力</td> <td style="text-align: left">1937年</td> <td style="text-align: left">9.1</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E4%BD%9B%E5%AE%B6%E5%90%8D%E7%9B%B8%E9%80%9A%E9%87%8A">链接</a></td> </tr> <tr> <td style="text-align: left">《傅山的世界》</td> <td style="text-align: left">[美] 白谦慎</td> <td style="text-align: left">2006年</td> <td style="text-align: left">9.1</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E5%82%85%E5%B1%B1%E7%9A%84%E4%B8%96%E7%95%8C">链接</a></td> </tr> <tr> <td style="text-align: left">《伽利略传》</td> <td style="text-align: left">[德] 贝托尔特·布莱希特</td> <td style="text-align: left">1943年</td> <td style="text-align: left">8.9</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E4%BC%BD%E5%88%A9%E7%95%A5%E4%BC%A0">链接</a></td> </tr> <tr> <td style="text-align: left">《关于他人的痛苦》</td> <td style="text-align: left">[美] 苏珊·桑塔格</td> <td style="text-align: left">2003年</td> <td style="text-align: left">8.5</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E5%85%B3%E4%BA%8E%E4%BB%96%E4%BA%BA%E7%9A%84%E7%97%9B%E8%8B%A6">链接</a></td> </tr> <tr> <td style="text-align: left">《观看之道》</td> <td style="text-align: left">[英] 约翰·伯格</td> <td style="text-align: left">1972年</td> <td style="text-align: left">8.5</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E8%A7%82%E7%9C%8B%E4%B9%8B%E9%81%93">链接</a></td> </tr> <tr> <td style="text-align: left">《汉字书法之美》</td> <td style="text-align: left">蒋勋</td> <td style="text-align: left">2009年</td> <td style="text-align: left">8.5</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E6%B1%89%E5%AD%97%E4%B9%A6%E6%B3%95%E4%B9%8B%E7%BE%8E">链接</a></td> </tr> <tr> <td style="text-align: left">《汉字与文物的故事》</td> <td style="text-align: left">孙机</td> <td style="text-align: left">2021年</td> <td style="text-align: left">9.2</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E6%B1%89%E5%AD%97%E4%B8%8E%E6%96%87%E7%89%A9%E7%9A%84%E6%95%85%E4%BA%8B">链接</a></td> </tr> <tr> <td style="text-align: left">《黑镜头》</td> <td style="text-align: left">[美] 罗伯特·普雷基</td> <td style="text-align: left">2002年</td> <td style="text-align: left">8.8</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E9%BB%91%E9%95%9C%E5%A4%B4">链接</a></td> </tr> <tr> <td style="text-align: left">《黄泉下的美术》</td> <td style="text-align: left">巫鸿</td> <td style="text-align: left">2005年</td> <td style="text-align: left">8.6</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E9%BB%84%E6%B3%89%E4%B8%8B%E7%9A%84%E7%BE%8E%E6%9C%AF">链接</a></td> </tr> <tr> <td style="text-align: left">《火车上的中国人》</td> <td style="text-align: left">王福春</td> <td style="text-align: left">2001年</td> <td style="text-align: left">8.8</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E7%81%AB%E8%BD%A6%E4%B8%8A%E7%9A%84%E4%B8%AD%E5%9B%BD%E4%BA%BA">链接</a></td> </tr> <tr> <td style="text-align: left">《基督教神学原理》</td> <td style="text-align: left">[美] 奥尔森</td> <td style="text-align: left">1992年</td> <td style="text-align: left">8.9</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E5%9F%BA%E7%9D%A3%E6%95%99%E7%A5%9E%E5%AD%A6%E5%8E%9F%E7%90%86">链接</a></td> </tr> <tr> <td style="text-align: left">《基督教要义》</td> <td style="text-align: left">[法] 约翰·加尔文</td> <td style="text-align: left">1536年</td> <td style="text-align: left">9.5</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E5%9F%BA%E7%9D%A3%E6%95%99%E8%A6%81%E4%B9%89">链接</a></td> </tr> <tr> <td style="text-align: left">《加德纳艺术通史》</td> <td style="text-align: left">[美] 弗雷德·S. 克莱纳</td> <td style="text-align: left">1926年</td> <td style="text-align: left">9.4</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E5%8A%A0%E5%BE%B7%E7%BA%B3%E8%89%BA%E6%9C%AF%E9%80%9A%E5%8F%B2">链接</a></td> </tr> <tr> <td style="text-align: left">《剑桥中国史》</td> <td style="text-align: left">[英] 费正清等</td> <td style="text-align: left">1978年</td> <td style="text-align: left">9.4</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E5%89%91%E6%A1%A5%E4%B8%AD%E5%9B%BD%E5%8F%B2">链接</a></td> </tr> <tr> <td style="text-align: left">《咖啡厅、餐馆内景实例》</td> <td style="text-align: left">-</td> <td style="text-align: left">-</td> <td style="text-align: left">6.7</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E5%92%96%E5%95%A1%E5%8E%85%E3%80%81%E9%A4%90%E9%A6%86%E5%86%85%E6%99%AF%E5%AE%9E%E4%BE%8B">链接</a></td> </tr> <tr> <td style="text-align: left">《康德传》</td> <td style="text-align: left">[德] 曼弗雷德·库恩</td> <td style="text-align: left">2001年</td> <td style="text-align: left">9.1</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E5%BA%B7%E5%BE%B7%E4%BC%A0">链接</a></td> </tr> <tr> <td style="text-align: left">《旷野呼告》</td> <td style="text-align: left">[美] 杰克·伦敦</td> <td style="text-align: left">1903年</td> <td style="text-align: left">8.8</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E6%97%B7%E9%87%8E%E5%91%BC%E5%91%8A">链接</a></td> </tr> <tr> <td style="text-align: left">《拉丁美洲被切开的血管》</td> <td style="text-align: left">[乌拉圭] 爱德华多·加莱亚诺</td> <td style="text-align: left">1971年</td> <td style="text-align: left">9.3</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E6%8B%89%E4%B8%81%E7%BE%8E%E6%B4%B2%E8%A2%AB%E5%88%87%E5%BC%80%E7%9A%84%E8%A1%80%E7%AE%A1">链接</a></td> </tr> <tr> <td style="text-align: left">《蓝色血脉》</td> <td style="text-align: left">朱大可</td> <td style="text-align: left">1991年</td> <td style="text-align: left">8.1</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E8%93%9D%E8%89%B2%E8%A1%80%E8%84%89">链接</a></td> </tr> <tr> <td style="text-align: left">《劳特利奇哲学史》</td> <td style="text-align: left">G.H.R.帕金森 (主编)</td> <td style="text-align: left">1993年</td> <td style="text-align: left">9.3</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E5%8A%B3%E7%89%B9%E5%88%A9%E5%A5%87%E5%93%B2%E5%AD%A6%E5%8F%B2">链接</a></td> </tr> <tr> <td style="text-align: left">《理解一张照片》</td> <td style="text-align: left">[英] 约翰·伯格</td> <td style="text-align: left">2013年</td> <td style="text-align: left">8.3</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E7%90%86%E8%A7%A3%E4%B8%80%E5%BC%A0%E7%85%A7%E7%89%87">链接</a></td> </tr> <tr> <td style="text-align: left">《理想城市》</td> <td style="text-align: left">[美] 简·雅各布斯</td> <td style="text-align: left">1961年</td> <td style="text-align: left">9.4</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E7%90%86%E6%83%B3%E5%9F%8E%E5%B8%82">链接</a></td> </tr> <tr> <td style="text-align: left">《另一种讲述的方式》</td> <td style="text-align: left">[英] 约翰·伯格</td> <td style="text-align: left">1982年</td> <td style="text-align: left">8.8</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E5%8F%A6%E4%B8%80%E7%A7%8D%E8%AE%B2%E8%BF%B0%E7%9A%84%E6%96%B9%E5%BC%8F">链接</a></td> </tr> <tr> <td style="text-align: left">《伦理学》</td> <td style="text-align: left">[荷] 巴鲁赫·斯宾诺莎</td> <td style="text-align: left">1677年</td> <td style="text-align: left">9.2</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E4%BC%A6%E7%90%86%E5%AD%A6">链接</a></td> </tr> <tr> <td style="text-align: left">《论摄影》</td> <td style="text-align: left">[美] 苏珊·桑塔格</td> <td style="text-align: left">1977年</td> <td style="text-align: left">8.7</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E8%AE%BA%E6%91%84%E5%BD%B1">链接</a></td> </tr> <tr> <td style="text-align: left">《毛以后的中国》</td> <td style="text-align: left">[美] 罗德里克·麦克法夸尔</td> <td style="text-align: left">2008年</td> <td style="text-align: left">9.3</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E6%AF%9B%E4%BB%A5%E5%90%8E%E7%9A%84%E4%B8%AD%E5%9B%BD">链接</a></td> </tr> <tr> <td style="text-align: left">《美术、神话与祭祀》</td> <td style="text-align: left">张光直</td> <td style="text-align: left">1988年</td> <td style="text-align: left">9.0</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E7%BE%8E%E6%9C%AF%E3%80%81%E7%A5%9E%E8%AF%9D%E4%B8%8E%E7%A5%AD%E7%A5%80">链接</a></td> </tr> <tr> <td style="text-align: left">《明朝那些事儿》</td> <td style="text-align: left">当年明月</td> <td style="text-align: left">2006年</td> <td style="text-align: left">9.2</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E6%98%8E%E6%9C%9D%E9%82%A3%E4%BA%9B%E4%BA%8B%E5%84%BF">链接</a></td> </tr> <tr> <td style="text-align: left">《墨庄漫录》</td> <td style="text-align: left">[宋] 张邦基</td> <td style="text-align: left">南宋</td> <td style="text-align: left">8.6</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E5%A2%A8%E5%BA%84%E6%BC%AB%E5%BD%95">链接</a></td> </tr> <tr> <td style="text-align: left">《纽约摄影学院摄影教材》</td> <td style="text-align: left">[美] Don Sheff</td> <td style="text-align: left">1970年</td> <td style="text-align: left">8.7</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E7%BA%BD%E7%BA%A6%E6%91%84%E5%BD%B1%E5%AD%A6%E9%99%A2%E6%91%84%E5%BD%B1%E6%95%99%E6%9D%90">链接</a></td> </tr> <tr> <td style="text-align: left">《欧洲大学史》</td> <td style="text-align: left">[法] 克里斯托夫·夏尔勒</td> <td style="text-align: left">2002年</td> <td style="text-align: left">8.3</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E6%AC%A7%E6%B4%B2%E5%A4%A7%E5%AD%A6%E5%8F%B2">链接</a></td> </tr> <tr> <td style="text-align: left">《破〈破新唯识论〉》</td> <td style="text-align: left">熊十力</td> <td style="text-align: left">1923年</td> <td style="text-align: left">8.6</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E7%A0%B4%E3%80%88%E7%A0%B4%E6%96%B0%E5%94%AF%E8%AF%86%E8%AE%BA%E3%80%89">链接</a></td> </tr> <tr> <td style="text-align: left">《囚徒的困境》</td> <td style="text-align: left">[美] 威廉·庞德斯通</td> <td style="text-align: left">1992年</td> <td style="text-align: left">8.4</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E5%9B%9A%E5%BE%92%E7%9A%84%E5%9B%B0%E5%A2%83">链接</a></td> </tr> <tr> <td style="text-align: left">《让房子与你的灵魂契合》</td> <td style="text-align: left">[美] 克莱尔·库珀·马库斯</td> <td style="text-align: left">1995年</td> <td style="text-align: left">8.0</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E8%AE%A9%E6%88%BF%E5%AD%90%E4%B8%8E%E4%BD%A0%E7%9A%84%E7%81%B5%E9%AD%82%E5%A5%91%E5%90%88">链接</a></td> </tr> <tr> <td style="text-align: left">《人类简史》</td> <td style="text-align: left">[以色列] 尤瓦尔·赫拉利</td> <td style="text-align: left">2011年</td> <td style="text-align: left">9.1</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E4%BA%BA%E7%B1%BB%E7%AE%80%E5%8F%B2">链接</a></td> </tr> <tr> <td style="text-align: left">《如何建造美好家园》</td> <td style="text-align: left">[英] 约翰·布鲁克斯</td> <td style="text-align: left">1984年</td> <td style="text-align: left">8.6</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E5%A6%82%E4%BD%95%E5%BB%BA%E9%80%A0%E7%BE%8E%E5%A5%BD%E5%AE%B6%E5%9B%AD">链接</a></td> </tr> <tr> <td style="text-align: left">《撒马尔罕的金桃》</td> <td style="text-align: left">[美] 薛爱华</td> <td style="text-align: left">1963年</td> <td style="text-align: left">9.2</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E6%92%92%E9%A9%AC%E5%B0%94%E7%BD%95%E7%9A%84%E9%87%91%E6%A1%83">链接</a></td> </tr> <tr> <td style="text-align: left">《僧侣与哲学家》</td> <td style="text-align: left">[法] 让-弗朗索瓦·勒维尔</td> <td style="text-align: left">1997年</td> <td style="text-align: left">8.5</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E5%83%A7%E4%BE%A3%E4%B8%8E%E5%93%B2%E5%AD%A6%E5%AE%B6">链接</a></td> </tr> <tr> <td style="text-align: left">《送法下乡》</td> <td style="text-align: left">苏力</td> <td style="text-align: left">2000年</td> <td style="text-align: left">8.7</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E9%80%81%E6%B3%95%E4%B8%8B%E4%B9%A1">链接</a></td> </tr> <tr> <td style="text-align: left">《山川悠远》</td> <td style="text-align: left">方闻</td> <td style="text-align: left">2004年</td> <td style="text-align: left">8.5</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E5%B1%B1%E5%B7%9D%E6%82%A0%E8%BF%9C">链接</a></td> </tr> <tr> <td style="text-align: left">《设计中的设计》</td> <td style="text-align: left">[日] 原研哉</td> <td style="text-align: left">2003年</td> <td style="text-align: left">8.5</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E8%AE%BE%E8%AE%A1%E4%B8%AD%E7%9A%84%E8%AE%BE%E8%AE%A1">链接</a></td> </tr> <tr> <td style="text-align: left">《摄影哲学的思考》</td> <td style="text-align: left">[捷] 维兰·傅拉瑟</td> <td style="text-align: left">1983年</td> <td style="text-align: left">8.5</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E6%91%84%E5%BD%B1%E5%93%B2%E5%AD%A6%E7%9A%84%E6%80%9D%E8%80%83">链接</a></td> </tr> <tr> <td style="text-align: left">《身体·性别·摄影》</td> <td style="text-align: left">[日] 笠原美智子</td> <td style="text-align: left">2003年</td> <td style="text-align: left">8.0</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E8%BA%AB%E4%BD%93%C2%B7%E6%80%A7%E5%88%AB%C2%B7%E6%91%84%E5%BD%B1">链接</a></td> </tr> <tr> <td style="text-align: left">《神话学》</td> <td style="text-align: left">[法] 罗兰·巴特</td> <td style="text-align: left">1957年</td> <td style="text-align: left">8.4</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E7%A5%9E%E8%AF%9D%E5%AD%A6">链接</a></td> </tr> <tr> <td style="text-align: left">《生活与命运》</td> <td style="text-align: left">[苏] 瓦西里·格罗斯曼</td> <td style="text-align: left">1980年</td> <td style="text-align: left">9.6</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E7%94%9F%E6%B4%BB%E4%B8%8E%E5%91%BD%E8%BF%90">链接</a></td> </tr> <tr> <td style="text-align: left">《圣经·旧约》</td> <td style="text-align: left">摩西等</td> <td style="text-align: left">公元前13世纪-前2世纪</td> <td style="text-align: left">9.2</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E5%9C%A3%E7%BB%8F%C2%B7%E6%97%A7%E7%BA%A6">链接</a></td> </tr> <tr> <td style="text-align: left">《圣经·新约》</td> <td style="text-align: left">马太 / 马可 / 路加等</td> <td style="text-align: left">公元1世纪</td> <td style="text-align: left">9.2</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E5%9C%A3%E7%BB%8F%C2%B7%E6%96%B0%E7%BA%A6">链接</a></td> </tr> <tr> <td style="text-align: left">《世界摄影史》</td> <td style="text-align: left">[美] 内奥米·罗森布拉姆</td> <td style="text-align: left">1984年</td> <td style="text-align: left">8.8</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E4%B8%96%E7%95%8C%E6%91%84%E5%BD%B1%E5%8F%B2">链接</a></td> </tr> <tr> <td style="text-align: left">《世界摄影艺术史》</td> <td style="text-align: left">[法] 安德烈·胡耶</td> <td style="text-align: left">2005年</td> <td style="text-align: left">8.3</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E4%B8%96%E7%95%8C%E6%91%84%E5%BD%B1%E8%89%BA%E6%9C%AF%E5%8F%B2">链接</a></td> </tr> <tr> <td style="text-align: left">《世界通史》</td> <td style="text-align: left">[美] 斯塔夫里阿诺斯</td> <td style="text-align: left">1970年</td> <td style="text-align: left">9.1</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E4%B8%96%E7%95%8C%E9%80%9A%E5%8F%B2">链接</a></td> </tr> <tr> <td style="text-align: left">《市井西仓》</td> <td style="text-align: left">胡武功</td> <td style="text-align: left">2006年</td> <td style="text-align: left">8.1</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E5%B8%82%E4%BA%95%E8%A5%BF%E4%BB%93">链接</a></td> </tr> <tr> <td style="text-align: left">《私人生活史》</td> <td style="text-align: left">[法] 菲利普·阿里埃斯等</td> <td style="text-align: left">1985年</td> <td style="text-align: left">8.7</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E7%A7%81%E4%BA%BA%E7%94%9F%E6%B4%BB%E5%8F%B2">链接</a></td> </tr> <tr> <td style="text-align: left">《斯宾诺莎导读》</td> <td style="text-align: left">[美] 史蒂文·纳德勒</td> <td style="text-align: left">2006年</td> <td style="text-align: left">8.7</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E6%96%AF%E5%AE%BE%E8%AF%BA%E8%8E%8E%E5%AF%BC%E8%AF%BB">链接</a></td> </tr> <tr> <td style="text-align: left">《四库全书》</td> <td style="text-align: left">[清] 纪昀等</td> <td style="text-align: left">1782年</td> <td style="text-align: left">9.9</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E5%9B%9B%E5%BA%93%E5%85%A8%E4%B9%A6">链接</a></td> </tr> <tr> <td style="text-align: left">《俗世威尔》</td> <td style="text-align: left">[英] 特里·伊格尔顿</td> <td style="text-align: left">2008年</td> <td style="text-align: left">8.5</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E4%BF%97%E4%B8%96%E5%A8%81%E5%B0%94">链接</a></td> </tr> <tr> <td style="text-align: left">《涑水记闻》</td> <td style="text-align: left">[宋] 司马光</td> <td style="text-align: left">北宋</td> <td style="text-align: left">8.7</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E6%BA%A3%E6%B0%B4%E8%AE%B0%E9%97%BB">链接</a></td> </tr> <tr> <td style="text-align: left">《太平御览》</td> <td style="text-align: left">[宋] 李昉等</td> <td style="text-align: left">983年</td> <td style="text-align: left">9.8</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E5%A4%AA%E5%B9%B3%E5%BE%A1%E8%A7%88">链接</a></td> </tr> <tr> <td style="text-align: left">《天真的人类学家》</td> <td style="text-align: left">[英] 奈吉尔·巴利</td> <td style="text-align: left">1983年</td> <td style="text-align: left">8.4</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E5%A4%A9%E7%9C%9F%E7%9A%84%E4%BA%BA%E7%B1%BB%E5%AD%A6%E5%AE%B6">链接</a></td> </tr> <tr> <td style="text-align: left">《同性恋亚文化》</td> <td style="text-align: left">李银河 / 王小波</td> <td style="text-align: left">1998年</td> <td style="text-align: left">8.5</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E5%90%8C%E6%80%A7%E6%81%8B%E4%BA%9A%E6%96%87%E5%8C%96">链接</a></td> </tr> <tr> <td style="text-align: left">《图书馆入门》</td> <td style="text-align: left">[日] 若松英辅</td> <td style="text-align: left">2013年</td> <td style="text-align: left">8.1</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E5%9B%BE%E4%B9%A6%E9%A6%86%E5%85%A5%E9%97%A8">链接</a></td> </tr> <tr> <td style="text-align: left">《完美店铺设计指南》</td> <td style="text-align: left">-</td> <td style="text-align: left">-</td> <td style="text-align: left">7.0</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E5%AE%8C%E7%BE%8E%E5%BA%97%E9%93%BA%E8%AE%BE%E8%AE%A1%E6%8C%87%E5%8D%97">链接</a></td> </tr> <tr> <td style="text-align: left">《唯识二十论》</td> <td style="text-align: left">[古印度] 世亲</td> <td style="text-align: left">约4世纪</td> <td style="text-align: left">9.2</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E5%94%AF%E8%AF%86%E4%BA%8C%E5%8D%81%E8%AE%BA">链接</a></td> </tr> <tr> <td style="text-align: left">《为什么我不是基督教徒》</td> <td style="text-align: left">[英] 伯特兰·罗素</td> <td style="text-align: left">1927年</td> <td style="text-align: left">8.7</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E4%B8%BA%E4%BB%80%E4%B9%88%E6%88%91%E4%B8%8D%E6%98%AF%E5%9F%BA%E7%9D%A3%E6%95%99%E5%BE%92">链接</a></td> </tr> <tr> <td style="text-align: left">《未来简史》</td> <td style="text-align: left">[以色列] 尤瓦尔·赫拉利</td> <td style="text-align: left">2015年</td> <td style="text-align: left">8.4</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E6%9C%AA%E6%9D%A5%E7%AE%80%E5%8F%B2">链接</a></td> </tr> <tr> <td style="text-align: left">《文字的力与美》</td> <td style="text-align: left">[日] 杉浦康平</td> <td style="text-align: left">2002年</td> <td style="text-align: left">8.7</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E6%96%87%E5%AD%97%E7%9A%84%E5%8A%9B%E4%B8%8E%E7%BE%8E">链接</a></td> </tr> <tr> <td style="text-align: left">《无知的教师》</td> <td style="text-align: left">[法] 雅克·朗西埃</td> <td style="text-align: left">1987年</td> <td style="text-align: left">8.5</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E6%97%A0%E7%9F%A5%E7%9A%84%E6%95%99%E5%B8%88">链接</a></td> </tr> <tr> <td style="text-align: left">《乡土中国》</td> <td style="text-align: left">费孝通</td> <td style="text-align: left">1947年</td> <td style="text-align: left">9.3</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E4%B9%A1%E5%9C%9F%E4%B8%AD%E5%9B%BD">链接</a></td> </tr> <tr> <td style="text-align: left">《湘山野录》</td> <td style="text-align: left">[宋] 释文莹</td> <td style="text-align: left">北宋</td> <td style="text-align: left">8.2</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E6%B9%98%E5%B1%B1%E9%87%8E%E5%BD%95">链接</a></td> </tr> <tr> <td style="text-align: left">《新教伦理与资本主义精神》</td> <td style="text-align: left">[德] 马克斯·韦伯</td> <td style="text-align: left">1905年</td> <td style="text-align: left">8.9</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E6%96%B0%E6%95%99%E4%BC%A6%E7%90%86%E4%B8%8E%E8%B5%84%E6%9C%AC%E4%B8%BB%E4%B9%89%E7%B2%BE%E7%A5%9E">链接</a></td> </tr> <tr> <td style="text-align: left">《新唯识论》</td> <td style="text-align: left">熊十力</td> <td style="text-align: left">1932年</td> <td style="text-align: left">9.1</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E6%96%B0%E5%94%AF%E8%AF%86%E8%AE%BA">链接</a></td> </tr> <tr> <td style="text-align: left">《新游牧民》</td> <td style="text-align: left">[日] 四方田犬彦</td> <td style="text-align: left">2002年</td> <td style="text-align: left">7.9</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E6%96%B0%E6%B8%B8%E7%89%A7%E6%B0%91">链接</a></td> </tr> <tr> <td style="text-align: left">《幸运者》</td> <td style="text-align: left">[英] 约翰·伯格</td> <td style="text-align: left">1967年</td> <td style="text-align: left">8.8</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E5%B9%B8%E8%BF%90%E8%80%85">链接</a></td> </tr> <tr> <td style="text-align: left">《修剪菩提树》</td> <td style="text-align: left">[美] 唐纳德·S.洛佩兹</td> <td style="text-align: left">1995年</td> <td style="text-align: left">8.7</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E4%BF%AE%E5%89%AA%E8%8F%A9%E6%8F%90%E6%A0%91">链接</a></td> </tr> <tr> <td style="text-align: left">《雅典与耶路撒冷》</td> <td style="text-align: left">[俄] 列夫·舍斯托夫</td> <td style="text-align: left">1938年</td> <td style="text-align: left">9.1</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E9%9B%85%E5%85%B8%E4%B8%8E%E8%80%B6%E8%B7%AF%E6%92%92%E5%86%B7">链接</a></td> </tr> <tr> <td style="text-align: left">《艺术哲学》</td> <td style="text-align: left">[法] 丹纳</td> <td style="text-align: left">1865年</td> <td style="text-align: left">9.1</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E8%89%BA%E6%9C%AF%E5%93%B2%E5%AD%A6">链接</a></td> </tr> <tr> <td style="text-align: left">《隐士建筑》</td> <td style="text-align: left">[日] 中村好文</td> <td style="text-align: left">2011年</td> <td style="text-align: left">8.6</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E9%9A%90%E5%A3%AB%E5%BB%BA%E7%AD%91">链接</a></td> </tr> <tr> <td style="text-align: left">《永字八法》</td> <td style="text-align: left">佚名</td> <td style="text-align: left">唐代</td> <td style="text-align: left">8.3</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E6%B0%B8%E5%AD%97%E5%85%AB%E6%B3%95">链接</a></td> </tr> <tr> <td style="text-align: left">《犹太教》</td> <td style="text-align: left">[英] 诺曼·所罗门</td> <td style="text-align: left">1996年</td> <td style="text-align: left">8.3</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E7%8A%B9%E5%A4%AA%E6%95%99">链接</a></td> </tr> <tr> <td style="text-align: left">《与古为徒和娟娟发屋》</td> <td style="text-align: left">巫鸿</td> <td style="text-align: left">2005年</td> <td style="text-align: left">9.0</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E4%B8%8E%E5%8F%A4%E4%B8%BA%E5%BE%92%E5%92%8C%E5%A8%9F%E5%A8%9F%E5%8F%91%E5%B1%8B">链接</a></td> </tr> <tr> <td style="text-align: left">《与小泽征尔共度的午后音乐时光》</td> <td style="text-align: left">[日] 村上春树 / 小泽征尔</td> <td style="text-align: left">2011年</td> <td style="text-align: left">8.7</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E4%B8%8E%E5%B0%8F%E6%B3%BD%E5%BE%81%E5%B0%94%E5%85%B1%E5%BA%A6%E7%9A%84%E5%8D%88%E5%90%8E%E9%9F%B3%E4%B9%90%E6%97%B6%E5%85%89">链接</a></td> </tr> <tr> <td style="text-align: left">《造型的诞生》</td> <td style="text-align: left">[日] 杉浦康平</td> <td style="text-align: left">1999年</td> <td style="text-align: left">9.1</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E9%80%A0%E5%9E%8B%E7%9A%84%E8%AF%9E%E7%94%9F">链接</a></td> </tr> <tr> <td style="text-align: left">《怎样阅读照片》</td> <td style="text-align: left">[英] 伊安·杰夫里</td> <td style="text-align: left">1981年</td> <td style="text-align: left">8.4</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E6%80%8E%E6%A0%B7%E9%98%85%E8%AF%BB%E7%85%A7%E7%89%87">链接</a></td> </tr> <tr> <td style="text-align: left">《詹森艺术史》</td> <td style="text-align: left">[美] H.W. 詹森</td> <td style="text-align: left">1962年</td> <td style="text-align: left">9.4</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E8%A9%B9%E6%A3%AE%E8%89%BA%E6%9C%AF%E5%8F%B2">链接</a></td> </tr> <tr> <td style="text-align: left">《正面管教》</td> <td style="text-align: left">[美] 简·尼尔森</td> <td style="text-align: left">1981年</td> <td style="text-align: left">8.4</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E6%AD%A3%E9%9D%A2%E7%AE%A1%E6%95%99">链接</a></td> </tr> <tr> <td style="text-align: left">《知日》</td> <td style="text-align: left">苏静 (主编)</td> <td style="text-align: left">2011年</td> <td style="text-align: left">7.5</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E7%9F%A5%E6%97%A5">链接</a></td> </tr> <tr> <td style="text-align: left">《直角之诗》</td> <td style="text-align: left">[法] 勒·柯布西耶</td> <td style="text-align: left">1955年</td> <td style="text-align: left">8.9</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E7%9B%B4%E8%A7%92%E4%B9%8B%E8%AF%97">链接</a></td> </tr> <tr> <td style="text-align: left">《纸上纪录片》</td> <td style="text-align: left">崔永元 (主编)</td> <td style="text-align: left">2002年</td> <td style="text-align: left">8.7</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E7%BA%B8%E4%B8%8A%E7%BA%AA%E5%BD%95%E7%89%87">链接</a></td> </tr> <tr> <td style="text-align: left">《中国碑帖名品》</td> <td style="text-align: left">-</td> <td style="text-align: left">-</td> <td style="text-align: left">9.2</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E4%B8%AD%E5%9B%BD%E7%A2%91%E5%B8%96%E5%90%8D%E5%93%81">链接</a></td> </tr> <tr> <td style="text-align: left">《中国摄影史》</td> <td style="text-align: left">陈申 / 徐希景</td> <td style="text-align: left">1987年</td> <td style="text-align: left">8.4</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E4%B8%AD%E5%9B%BD%E6%91%84%E5%BD%B1%E5%8F%B2">链接</a></td> </tr> <tr> <td style="text-align: left">《中国照相馆史》</td> <td style="text-align: left">[美] 泰瑞·贝内特</td> <td style="text-align: left">2013年</td> <td style="text-align: left">8.9</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E4%B8%AD%E5%9B%BD%E7%85%A7%E7%9B%B8%E9%A6%86%E5%8F%B2">链接</a></td> </tr> <tr> <td style="text-align: left">《宗教生活的基本形式》</td> <td style="text-align: left">[法] 埃米尔·涂尔干</td> <td style="text-align: left">1912年</td> <td style="text-align: left">9.0</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E5%AE%97%E6%95%99%E7%94%9F%E6%B4%BB%E7%9A%84%E5%9F%BA%E6%9C%AC%E5%BD%A2%E5%BC%8F">链接</a></td> </tr> <tr> <td style="text-align: left">《走向新建筑》</td> <td style="text-align: left">[法] 勒·柯布西耶</td> <td style="text-align: left">1923年</td> <td style="text-align: left">8.6</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E8%B5%B0%E5%90%91%E6%96%B0%E5%BB%BA%E7%AD%91">链接</a></td> </tr> </tbody> </table> <h3 id="自然科学">自然科学</h3> <table> <thead> <tr> <th style="text-align: left">书名</th> <th style="text-align: left">作者</th> <th style="text-align: left">出版年份</th> <th style="text-align: left">豆瓣评分</th> <th style="text-align: left">豆瓣链接</th> </tr> </thead> <tbody> <tr> <td style="text-align: left">《别闹了，费曼先生》</td> <td style="text-align: left">[美] 理查德·费曼</td> <td style="text-align: left">1985年</td> <td style="text-align: left">9.3</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E5%88%AB%E9%97%B9%E4%BA%86%EF%BC%8C%E8%B4%B9%E6%9B%BC%E5%85%88%E7%94%9F">链接</a></td> </tr> <tr> <td style="text-align: left">《城市自然故事》</td> <td style="text-align: left">张瑜</td> <td style="text-align: left">2021年</td> <td style="text-align: left">8.9</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E5%9F%8E%E5%B8%82%E8%87%AA%E7%84%B6%E6%95%85%E4%BA%8B">链接</a></td> </tr> <tr> <td style="text-align: left">《从一到无穷大》</td> <td style="text-align: left">[美] G. 伽莫夫</td> <td style="text-align: left">1947年</td> <td style="text-align: left">9.2</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E4%BB%8E%E4%B8%80%E5%88%B0%E6%97%A0%E7%A9%B7%E5%A4%A7">链接</a></td> </tr> <tr> <td style="text-align: left">《地球编年史》</td> <td style="text-align: left">[美] 撒迦利亚·西琴</td> <td style="text-align: left">1976年</td> <td style="text-align: left">8.1</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E5%9C%B0%E7%90%83%E7%BC%96%E5%B9%B4%E5%8F%B2">链接</a></td> </tr> <tr> <td style="text-align: left">《第三种黑猩猩》</td> <td style="text-align: left">[美] 贾雷德·戴蒙德</td> <td style="text-align: left">1991年</td> <td style="text-align: left">8.5</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E7%AC%AC%E4%B8%89%E7%A7%8D%E9%BB%91%E7%8C%A9%E7%8C%A9">链接</a></td> </tr> <tr> <td style="text-align: left">《哥德尔、艾舍尔、巴赫》</td> <td style="text-align: left">[美] 侯世达</td> <td style="text-align: left">1979年</td> <td style="text-align: left">9.4</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E5%93%A5%E5%BE%B7%E5%B0%94%E3%80%81%E8%89%BE%E8%88%8D%E5%B0%94%E3%80%81%E5%B7%B4%E8%B5%AB">链接</a></td> </tr> <tr> <td style="text-align: left">《给忙碌者的天体物理学》</td> <td style="text-align: left">[美] 奈尔·德葛拉司·泰森</td> <td style="text-align: left">2017年</td> <td style="text-align: left">8.6</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E7%BB%99%E5%BF%99%E7%A2%8C%E8%80%85%E7%9A%84%E5%A4%A9%E4%BD%93%E7%89%A9%E7%90%86%E5%AD%A6">链接</a></td> </tr> <tr> <td style="text-align: left">《给青年科学家的信》</td> <td style="text-align: left">[美] 爱德华·威尔逊</td> <td style="text-align: left">2013年</td> <td style="text-align: left">8.4</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E7%BB%99%E9%9D%92%E5%B9%B4%E7%A7%91%E5%AD%A6%E5%AE%B6%E7%9A%84%E4%BF%A1">链接</a></td> </tr> <tr> <td style="text-align: left">《果壳中的宇宙》</td> <td style="text-align: left">[英] 斯蒂芬·霍金</td> <td style="text-align: left">2001年</td> <td style="text-align: left">9.0</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E6%9E%9C%E5%A3%B3%E4%B8%AD%E7%9A%84%E5%AE%87%E5%AE%99">链接</a></td> </tr> <tr> <td style="text-align: left">《剑桥科学史》</td> <td style="text-align: left">[英] 科林·A.罗南</td> <td style="text-align: left">1983年</td> <td style="text-align: left">8.9</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E5%89%91%E6%A1%A5%E7%A7%91%E5%AD%A6%E5%8F%B2">链接</a></td> </tr> <tr> <td style="text-align: left">《科学的历程》</td> <td style="text-align: left">吴国盛</td> <td style="text-align: left">1995年</td> <td style="text-align: left">9.1</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E7%A7%91%E5%AD%A6%E7%9A%84%E5%8E%86%E7%A8%8B">链接</a></td> </tr> <tr> <td style="text-align: left">《盲眼钟表匠》</td> <td style="text-align: left">[英] 理查德·道金斯</td> <td style="text-align: left">1986年</td> <td style="text-align: left">9.0</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E7%9B%B2%E7%9C%BC%E9%92%9F%E8%A1%A8%E5%8C%A0">链接</a></td> </tr> <tr> <td style="text-align: left">《上帝掷骰子吗？》</td> <td style="text-align: left">曹天元</td> <td style="text-align: left">2006年</td> <td style="text-align: left">9.3</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E4%B8%8A%E5%B8%9D%E6%8E%B7%E9%AA%B0%E5%AD%90%E5%90%97%EF%BC%9F">链接</a></td> </tr> <tr> <td style="text-align: left">《什么是科学》</td> <td style="text-align: left">吴国盛</td> <td style="text-align: left">2016年</td> <td style="text-align: left">8.6</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E4%BB%80%E4%B9%88%E6%98%AF%E7%A7%91%E5%AD%A6">链接</a></td> </tr> <tr> <td style="text-align: left">《实验室女孩》</td> <td style="text-align: left">[美] 霍普·洁伦</td> <td style="text-align: left">2016年</td> <td style="text-align: left">8.6</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E5%AE%9E%E9%AA%8C%E5%AE%A4%E5%A5%B3%E5%AD%A9">链接</a></td> </tr> <tr> <td style="text-align: left">《贪婪的多巴胺》</td> <td style="text-align: left">[美] 丹尼尔·利伯曼等</td> <td style="text-align: left">2018年</td> <td style="text-align: left">7.9</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E8%B4%AA%E5%A9%AA%E7%9A%84%E5%A4%9A%E5%B7%B4%E8%83%BA">链接</a></td> </tr> <tr> <td style="text-align: left">《物理世界奇遇记》</td> <td style="text-align: left">[美] G. 伽莫夫</td> <td style="text-align: left">1940年</td> <td style="text-align: left">9.1</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E7%89%A9%E7%90%86%E4%B8%96%E7%95%8C%E5%A5%87%E9%81%87%E8%AE%B0">链接</a></td> </tr> <tr> <td style="text-align: left">《现实不似你所见》</td> <td style="text-align: left">[意] 卡洛·罗韦利</td> <td style="text-align: left">2014年</td> <td style="text-align: left">8.9</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E7%8E%B0%E5%AE%9E%E4%B8%8D%E4%BC%BC%E4%BD%A0%E6%89%80%E8%A7%81">链接</a></td> </tr> <tr> <td style="text-align: left">《园丁的一年》</td> <td style="text-align: left">[捷克] 卡雷尔·恰佩克</td> <td style="text-align: left">1929年</td> <td style="text-align: left">8.7</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E5%9B%AD%E4%B8%81%E7%9A%84%E4%B8%80%E5%B9%B4">链接</a></td> </tr> <tr> <td style="text-align: left">《云彩收集者手册》</td> <td style="text-align: left">[英] 加文·弗雷特-平尼</td> <td style="text-align: left">2006年</td> <td style="text-align: left">8.0</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E4%BA%91%E5%BD%A9%E6%94%B6%E9%9B%86%E8%80%85%E6%89%8B%E5%86%8C">链接</a></td> </tr> <tr> <td style="text-align: left">《杂草的故事》</td> <td style="text-align: left">[英] 理查德·梅比</td> <td style="text-align: left">2012年</td> <td style="text-align: left">8.8</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E6%9D%82%E8%8D%89%E7%9A%84%E6%95%85%E4%BA%8B">链接</a></td> </tr> <tr> <td style="text-align: left">《怎样观察一棵树》</td> <td style="text-align: left">[美] 南希·罗斯·哈格</td> <td style="text-align: left">2005年</td> <td style="text-align: left">8.5</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E6%80%8E%E6%A0%B7%E8%A7%82%E5%AF%9F%E4%B8%80%E6%A3%B5%E6%A0%91">链接</a></td> </tr> <tr> <td style="text-align: left">《这里是中国》</td> <td style="text-align: left">星球研究所 / 中国青藏高原研究会</td> <td style="text-align: left">2018年</td> <td style="text-align: left">9.3</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E8%BF%99%E9%87%8C%E6%98%AF%E4%B8%AD%E5%9B%BD">链接</a></td> </tr> <tr> <td style="text-align: left">《自私的基因》</td> <td style="text-align: left">[英] 理查德·道金斯</td> <td style="text-align: left">1976年</td> <td style="text-align: left">8.9</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E8%87%AA%E7%A7%81%E7%9A%84%E5%9F%BA%E5%9B%A0">链接</a></td> </tr> </tbody> </table> <h3 id="其他系列书">其他系列书</h3> <table> <thead> <tr> <th style="text-align: left">书名</th> <th style="text-align: left">作者</th> <th style="text-align: left">出版年份</th> <th style="text-align: left">豆瓣评分</th> <th style="text-align: left">豆瓣链接</th> </tr> </thead> <tbody> <tr> <td style="text-align: left">《中国在梁庄》(“梁庄”系列)</td> <td style="text-align: left">梁鸿</td> <td style="text-align: left">2010年</td> <td style="text-align: left">8.9</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E4%B8%AD%E5%9B%BD%E5%9C%A8%E6%A2%81%E5%BA%84">链接</a></td> </tr> <tr> <td style="text-align: left">《玛格南世纪》(“玛格南”系列)</td> <td style="text-align: left">玛格南图片社</td> <td style="text-align: left">1999年</td> <td style="text-align: left">9.4</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E7%8E%9B%E6%A0%BC%E5%8D%97%E4%B8%96%E7%BA%AA">链接</a></td> </tr> <tr> <td style="text-align: left">“牛津树”系列</td> <td style="text-align: left">[英] Roderick Hunt 等</td> <td style="text-align: left">1986年</td> <td style="text-align: left">9.7</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E7%89%9B%E6%B4%A5%E6%A0%91">链接</a></td> </tr> <tr> <td style="text-align: left">“培生”系列</td> <td style="text-align: left">培生教育集团</td> <td style="text-align: left">-</td> <td style="text-align: left">9.1</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E5%9F%B9%E7%94%9F">链接</a></td> </tr> <tr> <td style="text-align: left">《失落的一代》(“中国纪实三部曲”)</td> <td style="text-align: left">[法] 潘鸣啸</td> <td style="text-align: left">1994年</td> <td style="text-align: left">9.2</td> <td style="text-align: left"><a href="https://www.google.com/search?q=site%3Adouban.com+%E5%A4%B1%E8%90%BD%E7%9A%84%E4%B8%80%E4%BB%A3">链接</a></td> </tr> </tbody> </table>

2025/9/28

Yi's Blog

Solving Jane Street's 'Dropped a Neural Net' Puzzle

HRM Explained: A 27M Parameter Model That Reasons Without Chain-of-Thought

BrushNet & BrushEdit Explained: From Inpainting Architecture to Intelligent Editing

U-Net Explained: A Visual Guide for Beginners

Building an Image Captioning Transformer from Scratch

Building a Language Transformer Step by Step

Reverse Engineering Guitar Pro 8's Locked Files

Vibe Coding - Extracting Pet Sprites from Cross Gate

Breaking Up with Evernote: Building a Custom Migration Tool for Apple Notes

《世上为什么要有图书馆》读书笔记

推荐订阅

news view

Chawye Hsu

Oracle Blog for Java

Yuko's Blog

GoodBoyboy 's Blog｜惬意小屋-点滴记忆