The papers that actually helped (and the ones I bounced off)
A reading path into modern AI for people who did not do a PhD: what to read, in which order, and what to skip on the first pass.
January was my month of papers. Some of them earned their reputation. Some of them I bounced off twice before I understood what they were even claiming. This is the path that worked for me, in order, with honest notes.
Start before the famous one. Everyone will tell you to read Attention Is All You Need first. I tried. I printed it, took it on the train to Vienna, and got exactly nowhere, because the paper assumes you know what problem attention was solving in the first place. It was written for people who were already fighting with recurrent networks in 2017. My own AI years, 2019 onward, were spent learning the neighboring village: tabular models, anomaly detection, forecasting. NLP was the place across the river, and I had only ever visited as a tourist: using the models daily, never understanding them.
What unlocked it was reading about the problem first. Older translation models had to squeeze a whole sentence into one fixed summary before producing the translation, and long sentences did not survive the squeeze. Attention was the patch: let the model glance back at the original words while it writes, instead of working from memory. Once you see it that way, the transformer paper stops being mysterious. Its claim becomes almost cheeky: what if we keep only the glancing-back part and drop everything else.
Then the on-ramps, not the paper. Jay Alammar’s The Illustrated Transformer and Peter Bloem’s Transformers from scratch did more for me than the original PDF. This felt like cheating until I noticed that people who do this professionally recommend the same route. Read the blog posts, then reread the paper. Second time through, on the same train route in fact, it read like a summary of something I already knew.
Then three papers that each add one idea. BERT: you can pre-train on unlabeled text by masking words, and the result transfers to almost everything. GPT-3: if you make the model big enough, it starts doing tasks nobody trained it for, from just a few examples in the prompt. And InstructGPT, the RLHF paper: raw language models are alien, and the chat behavior we all take for granted is a separate training step where humans rank outputs. That last one quietly explains half of the weird behavior you see in chatbots, including the agreeable tone and the confident nonsense.
One more, because everyone runs into it eventually: the original RAG paper by Lewis et al. from 2020. The idea is one sentence: instead of hoping the model memorized your facts, retrieve the relevant documents and paste them into the prompt. Six years on there are entire conference tracks about doing this well, but the 2020 paper is still the cleanest statement of why you would bother.
What I skipped, deliberately: the math-heavy papers, anything about training infrastructure, and the scaling-law papers beyond their one-line summary (bigger models get better, in a surprisingly predictable way). Not because they are bad. Because at my stage they answered questions I did not have yet. A paper you read before you have the question is just an expensive way to feel tired.
The meta-lesson took me the whole month to admit, and six years of reading ML papers had taught me the opposite habit. I used to think reading papers was the serious way to learn and blog posts were the junk-food way. It is closer to the opposite: the blog posts give you the shape of the idea, and then the paper gives you the precision. Doing it in the other order means decoding notation with no picture in your head.
Total damage for January: five papers actually read, two abandoned, one read twice because the first time did not count. And a lesson I did not expect: every question I answered spawned two new ones. I am told this ratio never improves.