Project – CS 2610

Your final assignment, due May 14 (really the morning of May 15), is a research project and paper involving computer software systems. This is the biggest work product of the course.

The project has these components:

A five-to-ten-minute presentation on Friday May 9 of your results so far.
A research paper of sufficient density and polish that you would feel comfortable submitting it for scientific peer review (at least 6 pages).
(For group projects only) An individual contribution document detailing your contribution to the work, and your perceptions of the contributions of others (≤1 page).
A short self-evaluation (1–2 paragraphs). You may assign yourself a grade, or you may simply discuss your self-perceived level of effort and investment over the course.

Self-evaluations are a common feature of performance reviews in industry and academia. These are pretty annoying to write, but they also enforce valuable introspection, and help people realize just how much work they’ve actually accomplished.
A link to code and data used for the project. No need for superb software engineering, but ideally the code should be accompanied by enough documentation that a motivated user could attempt to replicate your results.

All course material is due on our final deadline, Thursday May 15 at 7:00am. There’s no flexibility on this date.

You will turn in your work on Canvas. Upload the paper there, and enter your self-evaluation in the text field.

Paper outline

The final project writeup should be a PDF that follows the format of a typical research paper: 11pt font, two-column, single-spaced. Do not double-space.

Writeups will be shared with the class and posted publicly unless you explicitly withhold permission to do so when submitting your paper.

Your contribution is a key aspect of the project writeup. What did you discover?

A typical writeup will follow this format, but the format is mostly advisory rather than mandatory.

Title. Something grabby that references a part of the contribution.
Abstract. One to three paragraphs that concisely describe the motivation (the problem the work addresses), the contribution of the work, and a highlight of the results. The abstract tends to target an already-expert audience.
Introduction. A page or so that expands on the abstract. The introduction is written for a broader audience than the abstract; its purpose is to situate the technically adept, but not necessarily expert, reader with the research problem as you define it. Your introduction, for example, might start out with a brief explanation of serverless computing, followed by a description of the issue with serverless computing that you address.

Often an introduction will end with a list of contributions, and an “outline” paragraph that says “The rest of this paper is organized as follows. Section 2, etc.”
Related work (this may also appear just before the conclusion). A description of related research, especially research closely related to your own work. The purposes of this section are citation and comparison. Foundational work requires citation only; “Amazon Web Services introduced modern serverless computing with AWS Lambda in 2014 [19].” More recent work requires comparison. Describe for each group of citations (1) the core idea, (2) what is complementary with your work, (3) what is more advanced than your work, and (4) what is advanced upon by your work. (2)–(4) are optional—some papers will be entirely complementary with or orthogonal to your work. (Hopefully no work will be entirely more advanced than yours!)

Note that related work is mandatory.
Design. The system you built, or the experiments you designed, described in enough relevant detail that a skilled system builder could replicate your work. Design is not about the names of your classes or your software engineering prowess, though you should include these details when they are relevant. Multiple pages.
Implementation. Aspects of your system that are not relevant to the design, but might be relevant to understanding your evaluation. Typically a page or less.
Evaluation. For systems work, this will often include the following subsections:
- Experimental setup. Describe how you ran your experiments. What kinds of machine? How much memory? How many trials? How did you prepare the machine before each trial? (Interesting reading: Systems Benchmarking Crimes, by Gernot Heiser.)
- The experiments themselves, grouped by purpose. Include figures.
- A summary of the experimental results.
A project hypothesis is often stated in the evaluation section.

Not all projects will have systems evaluations, but every project should be evaluated. If you are introducing a new computing model, for example, you could evaluate whether existing systems would fit the model.
Discussion. This optional section can speculate about your work. Feel free to put stuff here that is based on intuition and that you haven’t backed up with experiment. Do you think, in the end, that the project was a good idea? What would you do next? What should readers learn?
Conclusion. A paragraph or so.

Course advice

Writing quality is important to me. Do read over your paper before turning it in. Do not leave the writing for midnight on May 14! You may use AI to polish, but do not rely on it to do the writing (to overrely on AI is to impoverish yourself). Speak naturally and colloquially, but precisely, like some of the older papers we read.

Related work is mandatory. You must include a related work section in your writeup. Your related work section must demonstrate that you’ve done a literature search, found relevant prior efforts, and adjusted your project accordingly.

A good way to organize your project is to ask yourself how your ideas will be evaluated. What is your project hypothesis? This is a testable, falsifiable statement about your project’s core ideas. A good project hypothesis can guide and simplify your implementation and your design. For instance, you can use the hypothesis to decide which features to skip. (“I have an idea about how to build a better file system for non-persistent data. Since I can evaluate my hypothesis entirely using RAM disks, I don’t need to implement any file system consistency features or the fsync system call.”)

Some papers put their project hypotheses prominently in the abstract, the introduction, or the introduction to the evaluation section. Here’s a made-up project hypothesis (for which paper?). Note that it has several testable parts:

“File system read workloads will increasingly fit in memory caches. Therefore, file system performance will be dominated by writes. A file system that stores all data in a circular log will offer better write performance, and therefore better performance overall.”

A project hypothesis may require several iterations. It is OK to change your project hypothesis as you figure out which system to build.

You may be interested in writeups from prior classes:

General advice

Michael Ernst has some interesting advice on how to write a technical paper. (He has other advice too.) Some key points:

“The goal of writing a paper is to change people’s behavior: for instance, to change the way they think about a research problem or to convince them to use a new approach. … As a general rule, your paper needs to convince the audience of three key points: that the problem is interesting, that it is hard, and that you solved it.”

“A common mistake is to focus on what you spent the most time on. Do not write your paper as a chronological narrative of all the things that you tried, and do not devote space in the paper proportionately to the amount of time you spent on each task. Most work that you do will never show up in any paper; the purpose of infrastructure-building and exploration of blind alleys is to enable you to do the small amount of work that is worth writing about. Another way of stating this is that the purpose of the paper is not to describe what you have done, but to inform readers of the successful outcome or significant results, and to convince readers of the validity of those conclusions.”

“A related work section should not only explain what research others have done, but in each case should compare and contrast that to your work and also to other related work. After reading your related work section, a reader should understand the key idea and contribution of each significant piece of related work, how they fit together (what are the common themes or approaches in the research community?), and how your work differs.”

But don’t get too hung up on these points, especially in the context of a class project. There are many ways to write a good research paper, and different communities have different standards. Regarding the quotes above:

Negative results are important research, but negative results may not change people’s behavior.
“Interesting” and “hard” are subjective and depend on audience receptivity and authorial skill. A problem can be interesting for many reasons, including that the problem has intrinsically interesting features, that other people care about the problem (whether or not the authors do), or that a solution to the problem would open up a new class of problem. And a problem can be hard for many reasons, including that the problem has simply never been noticed before: a hard problem need not have a hard solution.
A chronological narrative can be interesting if done elegantly and if uninteresting details are skipped.