Sizes on Disk
Scope: the one idea the whole guide stands on — a file has two sizes, the content and the space — proven on three specimens: the one-byte file, the sparse file, and the file with two names.
Blocks: the disk's smallest coin
A filesystem does not hand out space byte
by byte. It allocates blocks — on the
build machine, 4096 bytes at a time — and
every file occupies a whole number of
them. That rounding is the disk usage.
The apparent size is the content's own
length, unrounded. ncdu reads both from
the file's metadata (the same numbers
stat prints) and never opens the file.
The lab's smallest specimen makes the
rounding visible. tiny.txt holds a
single character:
$ stat -c '%n size=%s blocks=%b (%B-byte units)' homedir/tiny.txt
homedir/tiny.txt size=1 blocks=8 (512-byte units)
$ du -B1 homedir/tiny.txt
4096 homedir/tiny.txt
One byte of content, 8×512 = 4096 bytes
of disk. stat counts blocks in 512-byte
units whatever the real block size; du
and ncdu do the multiplication for you.
In the browser, tiny.txt shows 4.0 KiB
by disk usage and 1.0 B by apparent
size (the a key swaps; chapter 2). The
whole Documents folder is the same lesson
at scale: 3.2 KiB of text costing 12.0 KiB
of disk, three files and three whole
blocks, plus change.
Neither number lies. Disk usage is what
the disk feels; apparent size is what a
program reading the file sees. A directory
of ten thousand tiny files is small in
every editor and large on every disk —
which is why chapter 2's C sort, by item
count, exists.
Directories are entries too: ncdu shows a
directory's row as the total beneath it,
and its own entry takes space on some
filesystems (a block of its own on ext4,
none on the build machine's overlay —
which is why the lab's du and ncdu
totals agree to the byte).
The sparse file: 100 MiB of nothing
vm-images/old-vm.img was made with
truncate, which sets a length without
writing anything. The unwritten stretches
are holes — the filesystem records
"zeros here" and allocates no blocks:
$ stat -c '%n size=%s blocks=%b' homedir/vm-images/old-vm.img
homedir/vm-images/old-vm.img size=104857600 blocks=0
$ du -B1 homedir/vm-images
0 homedir/vm-images
The info popup (i, chapter 2) says it in
ncdu's own words:
│ Name: old-vm.img │
│ Disk usage: 0.0 B (0 B) │
│ Apparent size: 100.0 MiB (104.857.600 B) │
Virtual-machine images, scientific data files and half-written downloads are often sparse, deliberately or by accident. Two consequences worth keeping:
- A disk can be "full" of apparent size and empty of disk usage, or the reverse. The lab's footer — 68.9 MiB disk, 168.8 MiB apparent — differs by almost exactly this one file.
- Copying can change a file's weight. A
copy tool that reads content and writes
it plainly turns holes into real blocks
(the copy of old-vm.img would occupy a
real 100 MiB); tools with a sparse
option (
cp --sparse=auto,rsync --sparse) keep the holes holes. Stated, not demonstrated — seecriteria.md.
The file with two names: hard links
Pictures/holiday/ holds img-002.jpg
and img-002-copy.jpg. They are not a
file and its copy. They are two directory
entries for one inode — one body of
data with two names, a hard link:
$ stat -c '%n links=%h inode=%i' homedir/Pictures/holiday/img-002*
homedir/Pictures/holiday/img-002.jpg links=2 inode=8029818
homedir/Pictures/holiday/img-002-copy.jpg links=2 inode=8029818
(Inode numbers differ on your build; the point is that the two names share one.)
Watch how each tool accounts for it:
- The browser shows both rows at
3.0 MiB, each flagged
H— and the directory total is 5.0 MiB (with img-001.jpg's 2 MiB), counting the shared data once. The rows of a directory with hard links add up to more than its total, on purpose: theHis ncdu warning you the double-count is in the display, not in the total. duagrees:du -B1on the directory reports 5,242,880 bytes = 2 MiB + 3 MiB, the shared data once.- The info popup's Links tab (chapter 2) names the partner path, and 2.9's popup shows Link count and Inode outright.
- The export, chapter 6 will show, lists
both entries at full size, marked
hlnkc— sum an export naively and you overcount by exactly one copy.
The deletion consequence is important enough to preview here and prove in chapter 5: removing one name of a hardlinked file frees nothing — the data lives until the last name goes. People "delete the big file" and watch the disk stay full; this is one of the two classic reasons (chapter 9 has the other).
Reconciling the totals
Three honest ways to add up the same tree, from the build machine:
| Method | Bytes | Why |
|---|---|---|
du -B1 homedir |
72,208,384 | disk usage, hardlinks once |
| ncdu footer | 68.9 MiB (= the same) | disk usage, hardlinks once |
Sum of dsize over the export |
75,354,112 | both hardlink names at full size |
The difference between the last row and the first two is 3,145,728 bytes — one copy of the photograph, to the byte. When two tools disagree about a disk, do not ask which is broken; ask what each one counted. That question, plus "how old is this scan" (chapter 2) and "what was excluded" (chapter 4), resolves nearly every size mystery in chapter 9.
Check: reproduce the table's first row
yourself — du -B1 homedir | tail -1
against your own build of the lab. Your
byte count should match exactly; the
lab's sizes are fixed by the script, not
by the machine.