(Including notes by Thomas Lively)
Front matter
Interesting bugs in the pset?
- adding struct field causes deadlock - uninitialized memory!
Logistics
- Next pset out soon
- Syllabus out soon(!)
- Please fill out survey on Piazza
File system disk structures
- Performance
- Hardware characteristics
- Consistency
Disks
- Block addressable
- 512 = sector size (original block size)
- x86
inb
andoutb
instructions read/write from disk by sector - Modern disks have larger block sizes
- Controllers translate between CPU commands and device commands
How to tell how big block size is?
- Timing reads (except sequential access is too fast!)
- Timing writes (turns into read then write for larger block sizes)
Write amplification = writes turn into bigger writes
- due to block size mismatches, garbage collection, etc.
Disk Contents
- Boot loader in block 0
- Files!
How to find "hello.txt"?
// on disk
struct filesystem {
struct file {
char name[200];
char data[1000000];
size_t size;
} f[10000];
};
Issues:
- Not great utilization (high internal fragmentation)
- Slow lookup
- Small max file size
Good things:
- Gets the job done
Suggestions:
- Make the file system a tree (e.g. ReiserFS)
- Store names together separately from data
Fix slow lookup: add a directory file type
- Directory contains names and pointers to files
- Place directory in fixed place to make it findable
Fix low utilization: 1. Extent based FS
- Every file is contiguous range of blocks
- But leads to high external fragmentation
- Needs defragging/garbage collection
- Solution: let files have multiple extents (e.g. ext4)
Let's invent FAT (File Allocation Table)
- Buddy allocator on disk is not FAT
- linked list of blocks is FAT (yay!)
How to find free blocks?
Suggestions:
- Traverse all linked lists (this is super slow)
- Store all next pointers in array at beginning (this is FAT)
- Need SUPER BLOCK to hold constants for file system
New idea: Free block bitmap + indirect blocks
- First few blocks stored contiguously in structure
- Indirect block contains array of block pointers
- As files get bigger, add more pointers to indirect blocks
- Add doubly indirect block containing array of indirect blocks, etc.
- Real systems have triply indirect blocks!
- Now file size limits are huge and small files are super fast
Next class: Correctness in file systems!
Further reading
Modern Operating Systems by Andrew S. Tanenbaum. §4.3 and §4.4.1 cover file system structure and design. Early sections in Chapter 5 make a good introduction to buses, controllers, and general hardware devices. Tanenbaum does not really cover SSDs (flash memory) or write amplification.
Operating Systems in Depth by Thomas W. Doeppner. Today’s material is covered in Chapter 6. §6.5 briefly covers write amplification.
Reading for Thursday
Tanenbaum: §4.3.6 (Journaling File Systems) and §4.4.3 (File System Consistency).
Doeppner: §6.2.