From reading papers to reading the field

Why a paper is not enough

Every paper is a snapshot of a moving conversation. The transformer paper (Vaswani 2017) sits between the attention papers that came before it (Bahdanau 2014, Cheng 2016) and the avalanche of follow-up work that built on it (BERT, GPT, T5, ViT, the entire foundation-model era). Reading the transformer paper in isolation tells you what its authors thought in 2017. Reading the conversation around it — what came before, what came after, what challenged it, what extended it — tells you what the field thinks now.

The two readings are different acts. Both matter. Most readers default to the first and never make it to the second.

Following the citation graph

Two directions of citation, both useful.

Backward citations — what this paper cites. Usually the related-work and method sections. These tell you the lineage: what the authors are building on, who they consider prior art. Reading three backward citations of a paper you found important gives you the context the authors assumed you had.

Forward citations — what cites this paper. The "Cited by" feature on Google Scholar and Semantic Scholar is the basic tool. For an important paper, forward citations show you the conversation that followed: follow-up work, criticism, refinements, applications. This is how you find out whether the original paper's claims have held up.

Concrete workflow: when you finish a Pass 2 read of a paper you cared about, open Semantic Scholar and scan its forward citations. Look at the highest-cited follow-ups. Note any that challenge or correct the original. Then decide whether to read those.

Reproducibility and code release

An AI paper today is incomplete without code. Increasingly, major venues require code release as a condition of acceptance. When reading a paper, check:

Is there a code link? Usually in the abstract, the first footnote, or the experimental section. If yes, the work is more reproducible and more credible.

Has the code been independently reproduced? Papers With Code aggregates community reproductions; for major papers there is often a third-party PyTorch or JAX implementation. If multiple independent reproductions exist, the result is robust.

What is in the repository? A README, training scripts, model weights, evaluation harness, and a clear list of dependencies is the standard for serious work. A "code coming soon" placeholder, two years after publication, is a quality signal in the other direction.

The negative-result asymmetry. Code release helps positive claims more than negative ones. If a paper says "X works," releasing the code lets others verify. If a paper says "X does not work" (e.g. "scaling Y does not help"), the negative claim is harder to verify externally because you can never prove the authors did not miss a hyperparameter that would have changed the result. Read negative claims with extra care.

Knowing the field — not just the papers

Three things help you know the field rather than a stack of papers.

Survey papers. A good survey, published roughly once every two years per sub-field, organises the landscape. The "Foundation Models" survey from Stanford (Bommasani et al. 2021) was an attempt at this for the LLM era. The "Mechanistic Interpretability" overview papers from Anthropic and others are doing the same for that sub-field. Read a survey before going deep on individual papers; it tells you which papers are pivotal.

Notable blogs, talks, and podcasts. Researchers write blog posts that are often more honest than their papers. Andrew Ng's The Batch, Sebastian Raschka's blog, Lilian Weng's blog, Chip Huyen's blog, and the technical posts on the major labs' websites (Anthropic, DeepMind, OpenAI) all sit between papers and news. A 30-minute conference talk often communicates more about what a paper actually contributes than the paper itself.

Reproducibility crises and corrections. The field has had several. The "Are Emergent Abilities a Mirage?" paper challenged the emergence claim from the original chain-of-thought work. The Reformer paper turned out not to scale the way it claimed. The "Galactica" launch was withdrawn after critical reception. These corrections are part of the field's knowledge. Read them.

The sustainable reading practice

Three principles from researchers who keep this up across decades, not weeks.

Concentrate, do not scatter. Pick a sub-field — alignment, vision-language models, RL, mechanistic interpretability, retrieval-augmented generation, sport AI, AI for code — and concentrate on it for at least three months at a time. Cross-discipline reading is shallower; deep reading in one area compounds.

Read most days, briefly. Twenty minutes of triage per day, every day, beats a four-hour Saturday session. The habit is more important than the volume.

Write what you read. Even one paragraph. The act of summarising is what makes the reading stick. Many researchers keep a personal "reading journal" — a single file or notebook of one-paragraph notes on every paper they Pass-2 read. Five years in, the journal is more valuable than any individual paper in it.

Picking a paper to read this week

The final exercise of the course. Find a paper from the last six months that is connected to a problem you actually care about. Apply everything from this course to it.

Where to go next

If you want to go further with research reading in particular, three suggestions:

• Srinivasan Keshav's "How to Read a Paper" (2007). The classic, two pages, free online. The source of the three-pass technique.
• The Distill journal archives (distill.pub). Although the journal stopped publishing in 2021, its archive is the gold standard for accessible AI research writing. Read three articles to see what good explanation looks like.
• The Anthropic research blog, the DeepMind blog, the OpenAI research page. The papers from these labs are accompanied by accessible companion posts. Reading the post first, then the paper, is a fast way to internalise a result.

What we covered

Five lessons. Why papers are hard. The standard anatomy. How to read the method. How to read experiments critically. How to read the field, not just papers. You now have a sustainable framework for keeping up with one of the fastest-moving fields in the world — without burning out and without confusing reading for understanding.