Lecture 11

(Including notes by Thomas Lively)

Front matter

Interesting bugs in the pset?

• adding struct field causes deadlock - uninitialized memory!

Logistics

• Next pset out soon
• Syllabus out soon(!)
• Please fill out survey on Piazza

File system disk structures

• Performance
• Hardware characteristics
• Consistency

Disks

• Block addressable
• 512 = sector size (original block size)
• x86 inb and outb instructions read/write from disk by sector
• Modern disks have larger block sizes
• Controllers translate between CPU commands and device commands

How to tell how big block size is?

• Timing reads (except sequential access is too fast!)
• Timing writes (turns into read then write for larger block sizes)

Write amplification = writes turn into bigger writes

• due to block size mismatches, garbage collection, etc.

Disk Contents

• Boot loader in block 0
• Files!

How to find "hello.txt"?

// on disk
struct filesystem {
    struct file {
        char name[200];
        char data[1000000];
        size_t size;
    } f[10000];
};

Issues:

• Not great utilization (high internal fragmentation)
• Slow lookup
• Small max file size

Good things:

• Gets the job done

Suggestions:

• Make the file system a tree (e.g. ReiserFS)
• Store names together separately from data

Fix slow lookup: add a directory file type

• Directory contains names and pointers to files
• Place directory in fixed place to make it findable

Fix low utilization: 1. Extent based FS

• Every file is contiguous range of blocks
• But leads to high external fragmentation
• Needs defragging/garbage collection
• Solution: let files have multiple extents (e.g. ext4)

Let's invent FAT (File Allocation Table)

• Buddy allocator on disk is not FAT
• linked list of blocks is FAT (yay!)

How to find free blocks?

Suggestions:

• Traverse all linked lists (this is super slow)
• Store all next pointers in array at beginning (this is FAT)
• Need SUPER BLOCK to hold constants for file system

New idea: Free block bitmap + indirect blocks

• First few blocks stored contiguously in structure
• Indirect block contains array of block pointers
• As files get bigger, add more pointers to indirect blocks
• Add doubly indirect block containing array of indirect blocks, etc.
• Real systems have triply indirect blocks!
• Now file size limits are huge and small files are super fast

Next class: Correctness in file systems!

Further reading

Modern Operating Systems by Andrew S. Tanenbaum. §4.3 and §4.4.1 cover file system structure and design. Early sections in Chapter 5 make a good introduction to buses, controllers, and general hardware devices. Tanenbaum does not really cover SSDs (flash memory) or write amplification.

Operating Systems in Depth by Thomas W. Doeppner. Today’s material is covered in Chapter 6. §6.5 briefly covers write amplification.

Reading for Thursday

Tanenbaum: §4.3.6 (Journaling File Systems) and §4.4.3 (File System Consistency).

Doeppner: §6.2.