Limits

What this tool will not do, stated plainly — including the limits the lab ran into itself.

It cannot speed up a single job

One input takes as long as it takes. Parallelism divides a list; it does not accelerate the items. A command that runs one core at full stretch — most encoders in their default mode, most interpreters — gains nothing except company.

Shared bottlenecks stay shared

Cores multiply; disks, networks and services do not. Twelve jobs compressing files from one disk are bounded by that disk, and twelve jobs calling one API are bounded by its rate limit — and may earn you a ban where four polite jobs would not. When a parallel run is slower than the loop it replaced, this is almost always why, and the fix is a smaller -j, not a bigger one.

Starting jobs is not free

The lab's own timing table (chapter 4) is the honest exhibit: on a busy shared machine, eight one-second sleeps took 15 to 48 seconds of wall time, and the difference was job-start overhead, not sleeping. Every job costs a process start, a shell, and bookkeeping. Millisecond jobs — renaming, touching, echoing — can spend more time being started than being run. Batch tiny work into fewer, larger jobs (--pipe blocks, chapter 7) or keep the loop.

Output held is memory held

-k, --group and --results all buffer output. A job that prints gigabytes will occupy gigabytes until its turn to print comes. Long, chatty jobs want --line-buffer or a redirection into their own files instead.

It is not a scheduler

No dependencies between jobs, no priorities, no queues that survive the run, no restarting a failed job unless you kept the job log and ask (chapter 9). Cluster schedulers — Slurm, Kubernetes Jobs, a CI system — exist for the day the work outgrows a command line, and parallel makes no claim on their territory.

The remote chapter is a promise, not a memory

This guide verified local work only. SSH distribution (chapter 11) is described from the manual because the lab had no SSH server; treat its first run on your machines as the experiment it is.

Version drift is real

The tool is actively developed and the version is a date. Options arrive (--retry-failed is younger than the rest of chapter 9), and packaged builds differ from upstream ones — the citation behaviour in chapter 3 differs between the Debian package and upstream for exactly this reason. parallel --version and the local man parallel are the last word for the build in front of you.