Skip to content

Sizes on Disk

Scope: the one idea the whole guide stands on — a file has two sizes, the content and the space — proven on three specimens: the one-byte file, the sparse file, and the file with two names.

Blocks: the disk's smallest coin

A filesystem does not hand out space byte by byte. It allocates blocks — on the build machine, 4096 bytes at a time — and every file occupies a whole number of them. That rounding is the disk usage. The apparent size is the content's own length, unrounded. ncdu reads both from the file's metadata (the same numbers stat prints) and never opens the file.

The lab's smallest specimen makes the rounding visible. tiny.txt holds a single character:

$ stat -c '%n size=%s blocks=%b (%B-byte units)' homedir/tiny.txt
homedir/tiny.txt size=1 blocks=8 (512-byte units)
$ du -B1 homedir/tiny.txt
4096    homedir/tiny.txt

One byte of content, 8×512 = 4096 bytes of disk. stat counts blocks in 512-byte units whatever the real block size; du and ncdu do the multiplication for you. In the browser, tiny.txt shows 4.0 KiB by disk usage and 1.0 B by apparent size (the a key swaps; chapter 2). The whole Documents folder is the same lesson at scale: 3.2 KiB of text costing 12.0 KiB of disk, three files and three whole blocks, plus change.

Neither number lies. Disk usage is what the disk feels; apparent size is what a program reading the file sees. A directory of ten thousand tiny files is small in every editor and large on every disk — which is why chapter 2's C sort, by item count, exists.

Directories are entries too: ncdu shows a directory's row as the total beneath it, and its own entry takes space on some filesystems (a block of its own on ext4, none on the build machine's overlay — which is why the lab's du and ncdu totals agree to the byte).

The sparse file: 100 MiB of nothing

vm-images/old-vm.img was made with truncate, which sets a length without writing anything. The unwritten stretches are holes — the filesystem records "zeros here" and allocates no blocks:

$ stat -c '%n size=%s blocks=%b' homedir/vm-images/old-vm.img
homedir/vm-images/old-vm.img size=104857600 blocks=0
$ du -B1 homedir/vm-images
0       homedir/vm-images

The info popup (i, chapter 2) says it in ncdu's own words:

│  Name: old-vm.img                               │
│     Disk usage:   0.0   B (0 B)                 │
│  Apparent size: 100.0 MiB (104.857.600 B)       │

Virtual-machine images, scientific data files and half-written downloads are often sparse, deliberately or by accident. Two consequences worth keeping:

  • A disk can be "full" of apparent size and empty of disk usage, or the reverse. The lab's footer — 68.9 MiB disk, 168.8 MiB apparent — differs by almost exactly this one file.
  • Copying can change a file's weight. A copy tool that reads content and writes it plainly turns holes into real blocks (the copy of old-vm.img would occupy a real 100 MiB); tools with a sparse option (cp --sparse=auto, rsync --sparse) keep the holes holes. Stated, not demonstrated — see criteria.md.

Pictures/holiday/ holds img-002.jpg and img-002-copy.jpg. They are not a file and its copy. They are two directory entries for one inode — one body of data with two names, a hard link:

$ stat -c '%n links=%h inode=%i' homedir/Pictures/holiday/img-002*
homedir/Pictures/holiday/img-002.jpg links=2 inode=8029818
homedir/Pictures/holiday/img-002-copy.jpg links=2 inode=8029818

(Inode numbers differ on your build; the point is that the two names share one.)

Watch how each tool accounts for it:

  • The browser shows both rows at 3.0 MiB, each flagged H — and the directory total is 5.0 MiB (with img-001.jpg's 2 MiB), counting the shared data once. The rows of a directory with hard links add up to more than its total, on purpose: the H is ncdu warning you the double-count is in the display, not in the total.
  • du agrees: du -B1 on the directory reports 5,242,880 bytes = 2 MiB + 3 MiB, the shared data once.
  • The info popup's Links tab (chapter 2) names the partner path, and 2.9's popup shows Link count and Inode outright.
  • The export, chapter 6 will show, lists both entries at full size, marked hlnkc — sum an export naively and you overcount by exactly one copy.

The deletion consequence is important enough to preview here and prove in chapter 5: removing one name of a hardlinked file frees nothing — the data lives until the last name goes. People "delete the big file" and watch the disk stay full; this is one of the two classic reasons (chapter 9 has the other).

Reconciling the totals

Three honest ways to add up the same tree, from the build machine:

Method Bytes Why
du -B1 homedir 72,208,384 disk usage, hardlinks once
ncdu footer 68.9 MiB (= the same) disk usage, hardlinks once
Sum of dsize over the export 75,354,112 both hardlink names at full size

The difference between the last row and the first two is 3,145,728 bytes — one copy of the photograph, to the byte. When two tools disagree about a disk, do not ask which is broken; ask what each one counted. That question, plus "how old is this scan" (chapter 2) and "what was excluded" (chapter 4), resolves nearly every size mystery in chapter 9.

Check: reproduce the table's first row yourself — du -B1 homedir | tail -1 against your own build of the lab. Your byte count should match exactly; the lab's sizes are fixed by the script, not by the machine.