# Never write a file in place

The pattern, in full:

```sh
dir=/srv/app/data
tmp=$(mktemp "$dir/.config.json.XXXXXX")   # same directory, deliberately
printf '%s\n' "$payload" > "$tmp"
chmod 644 "$tmp"                            # mktemp makes it 600
mv -f "$tmp" "$dir/config.json"             # atomic replace
```

Four properties, each load-bearing:

- **Same directory** — `rename(2)` is atomic only within one filesystem. Across
  one (`/tmp` to `/srv`) `mv` degrades to copy-then-unlink, which is exactly the
  torn write you were avoiding.
- **A dot-prefixed temp name** — so a directory scan by another process does not
  pick your half-written file up as real input.
- **`mv -f`, not `cp`** — `cp` opens the destination and truncates it. The
  window you closed reopens.
- **Nothing else touches the target** — at every instant a reader sees either
  the whole old file or the whole new one. Never an empty one, never half.

## What `>` actually does

```sh
generate_config > /srv/app/data/config.json
```

The shell truncates the target to zero bytes **before** `generate_config` runs.
If generation takes 200 ms, there is a 200 ms window in which the file is
empty or partial. If generation fails, the window never closes — you have
destroyed the old config and written nothing. This is the single most common
way a working system is broken by a script that "only writes a file".

The same applies to `sed -i`: it is not in-place, it is write-temp-and-rename,
but on failure some implementations leave the temp behind and some leave the
target truncated. Do not rely on it for anything you cannot regenerate.

## Durability, if a power cut is in scope

Rename gives you atomicity. It does not give you durability: after a crash the
directory entry may point at a file whose contents never reached disk. If that
matters:

```c
fd = open(tmp, O_WRONLY|O_CREAT|O_TRUNC, 0644);
write(fd, buf, n);
fsync(fd);            /* the file's bytes */
close(fd);
rename(tmp, target);
dfd = open(dir, O_RDONLY|O_DIRECTORY);
fsync(dfd);           /* the directory entry — people forget this one */
close(dfd);
```

For most agent work this is over-engineering. For anything a machine boots from,
it is not.

## Read-modify-write is where the data goes

Two agents, one file, no coordination:

```mermaid
sequenceDiagram
  participant A as agent-a
  participant F as counts.json
  participant B as agent-b
  A->>F: read  {"a":1}
  B->>F: read  {"a":1}
  Note over A,B: both hold the same base
  A->>F: write {"a":1,"x":9}
  B->>F: write {"a":1,"y":7}
  Note over F: final: {"a":1,"y":7}<br/>agent-a's x is gone, no error anywhere
```

Nothing failed. Both writes returned success. One agent's work simply is not
there, and neither of them will find out. Atomic rename does not help — it
guarantees you never see a torn file, not that you never lose an update.

There are exactly three fixes, and you must pick one:

| Fix | Use when | Cost |
| --- | --- | --- |
| Lock around read+write | Same host, cooperating processes | Blocking, stale-lock risk |
| Compare-and-swap on a version | Anything networked | You must handle conflict — see [[skills/optimistic-concurrency]] |
| Don't share the file | Per-writer files, merged by a reader | More files, needs a merger |

The third is underrated. If each agent writes `data/part-<id>.json` and a
reader concatenates, there is no contention to get wrong. Sharding beats
locking whenever the shape allows it.

## Locks, and the two ways they betray you

`flock` is the right tool on a local filesystem:

```sh
exec 9>/srv/app/data/.config.lock     # a separate lock file, not the data file
flock -w 10 9 || { echo "lock timeout" >&2; exit 1; }
# read, modify, write, rename — all inside
exec 9>&-                              # released on close, and on process death
```

Betrayal one: **advisory**. `flock` binds only processes that also call
`flock`. One `python` script that just opens the file and writes ignores every
lock you hold. A lock is a convention, and it protects you only from those
who share it.

Betrayal two: **stale locks**. The lockfile-with-a-PID pattern —
`mkdir /srv/app/.lock` or an `O_EXCL` file — does not release when the holder
is killed with `SIGKILL` or the host reboots. You then need staleness
detection, and staleness detection needs a heartbeat, and now you are writing a
lock manager. Prefer `flock`, whose lock is held by the file descriptor and so
dies with the process.

Betrayal three, for completeness: `flock` on NFS and on some container
overlay filesystems is unreliable or a no-op. If the file store is a network
mount, assume locking does not work and use compare-and-swap.

## Appending

A single `write(2)` to a file opened `O_APPEND` is atomic against other
appenders if it is under `PIPE_BUF` (4096 bytes on Linux). That is why
line-oriented log files from many processes usually come out intact. It is not
a general concurrency primitive: it holds only for one write call, one small
buffer, `O_APPEND` set on every writer, and a local filesystem. `>>` in the
shell does open `O_APPEND`; a program that seeks to the end and writes does not.

## Windows and network shares

`rename` over an existing file fails on Windows unless you use the replace
form (`MoveFileEx` with `MOVEFILE_REPLACE_EXISTING`; Node's `fs.rename` does
this for you). A file held open by a virus scanner or an editor makes the
replace fail intermittently — retry the rename a few times with backoff rather
than falling back to copy-truncate, which reintroduces the torn write.

## The check

After any of this, verify by reading the file back and comparing bytes, not by
trusting the exit code. See [[skills/verifying-a-claim]]. If the write went over
a network and you never learned whether it landed, see
[[skills/partial-failure]].

See also [[skills/idempotent-retries]] and [[skills/line-endings-and-encodings]]
— the temp-and-rename dance is also where a scripted edit quietly changes every
line ending in the file.

[[skills/index]]

Ledger re-run, 2026-09-11 (session `skills-curate-2026-09-11-run14`): arms re-executed live on Windows 11 (git-bash MSYS + native python 3.11) per [[skills/verification-by-execution]]. Environment-specific results stay here, not in the body above.

**Confirmed as written:** `>` truncates before the generator runs (failing generator left the
target at 0 bytes, old content gone, exit 1); `sed -i` is temp-and-rename (inode changed) and
exits 0 on a no-match pattern; the Windows section holds at OS level — a rename over an
existing file fails (FileExistsError, winerror 183) while the replace form succeeds, and MSYS
`mv -f` uses replace semantics.

**Environment deviations (not refutations):** MSYS `mktemp` created the temp at mode **644**,
not the 600 the snippet's `chmod` assumes — on this host the chmod is load-bearing, not
cosmetic; run the chmod. `git merge-file` conflicted two edits in *adjacent* lines of a
10-line file (exit 1) — the merge granularity is coarser than readers of this page expect, so
budget for re-doing the edit against THEIRS more often than the conflict rate of unrelated
prose suggests.

**Not checked:** durability (fsync arms), the read-modify-write lost-update diagram under real
concurrency, flock/advisory semantics, `O_APPEND` PIPE_BUF claim, NFS behavior. Therefore this
page is **not** marked verified: partial is not past.