On reading research papers

We’ll be reading a lot of research papers in this course, so over the semester you will naturally develop your own way to read a research paper. Use these guides to get started.

First, some basic points; then, concrete reading guides.

A research paper encapsulates enormous effort

A good research paper distills significant work by its authors. My group has worked for years on tens of thousands of lines of code and huge numbers of experiments, most of which got thrown away, to produce a single 12-page paper. A paper’s authors are trying to boil down everything they learned into something you can digest. A single sentence may represent a full year of misdirected effort.

Of course, there are many kinds of effort, and a great research paper may represent a flash of insight rather than toil. That insight still required real work, namely everything the researcher did to prepare themselves to receive it.

A research paper encapsulates a moment in history

Any research paper is a picture of its time. The paper was written in a context shaped by technology and society. What hardware was available? What kinds of research were exciting to the community at the time? What kinds of research were being funded? An open reading can teach you a lot about what people were thinking in the months and years before publication.

A research paper deserves critical attention

As of 2019, more than 7 million scientific papers are published per year. They are not all equally good; some are outright fraudulent. We will try to avoid the really bad ones, but you will still find that all research papers have weaknesses, and for some papers the weaknesses may overwhelm the strengths. Some papers fail on their own terms when read carefully; some will fail to interest you because of your own cast of mind. Nevertheless, you can and should learn something from each paper.

You owe a research paper nothing

When you read a paper, your goal is extractive: What can this paper teach you, now? A paper is not a precious artwork demanding a respectful, hushed approach. Skim it, skip around in it, disagree with it, rip it apart—whatever it takes to learn what you can—and when you’ve learned what you can, drop it. Despite the effort and history that formed the paper, you owe the paper nothing.

I read best when I read with curiosity, openness, and skepticism. The skepticism keeps me curious: What’s really going on in these experiments? The openness keeps me interested: even if I’m not interested in the topic, maybe there’s some trick I could learn from; and maybe the paper will show me why I should care about the topic after all.

Concrete reading guides

These guides have concrete advice on the reading process. Keshav’s is especially well known in the systems community.

These guides are great. They also contradict. (Keshav says to read the abstract first; Raff says to never read the abstract until the end.) There’s no one right way to read a paper. The literatures in different sciences have different qualities (for example, unfortunately for you, computer systems papers tend to be verbose), and our minds are different. We also read for different goals. Reading a paper in order to review it requires more antagonism than reading a long-published, well-cited work. And particularly at the beginning of this course, when we are reading multiple historical papers per course meeting rather than one current paper, I expect you to spend less than “three to four hours” per paper!

Public reading groups

You may be interested in these public examples of reading research papers.