Deleting a File Doesn't Free Any Space

On a normal desktop filesystem, deleting a file feels instantaneous — the icon vanishes, the free-space counter ticks up, and you move on. What actually happens underneath is that the filesystem marks a set of blocks as reusable and removes the directory entry pointing at them. The bytes are usually still sitting on the platter or the flash cell until something else happens to be written over them.

That "mark as reusable, overwrite later" model depends on one capability: the ability to go back and write into the middle of the volume, wherever the freed blocks happen to be. Hard drives and SSDs can do this. Most tape and every WORM medium cannot. A tape drive writes and reads in one direction, and a WORM cartridge is physically incapable of accepting a second write to a location it has already used. If your storage engine assumes it can jump backward and overwrite a hole, it simply does not work on the majority of the media HuskHoard is built to manage.

So HuskHoard doesn't make that assumption anywhere. Every volume — SSD-backed cache, spinning disk, LTO tape, or WORM cartridge — is treated the same way: as a tape, conceptually, even when it physically isn't one.

The Append-Only Rule

An append-only volume has one law: new data goes at the current write position, and the write position only ever moves forward. Nothing already on the volume is ever modified, relocated, or overwritten. When a file is updated, HuskHoard doesn't patch the old bytes — it writes a new copy at the current end of the volume and updates the catalog to point at the new copy. The old copy is still physically present. It's just no longer the version anything points to.

Random-write model

Delete = free space now

The filesystem finds the blocks belonging to the deleted file, marks them free, and the next write can land there. This requires random write access, and on a volume with years of activity it also means a single file's blocks can end up scattered anywhere across the disk.

Append-only model

Delete = mark it dead in the catalog

The old bytes stay exactly where they were written. Only the catalog entry changes, from "live" to "dead." The volume's used-space number doesn't go down. Nothing on the physical medium moves until a repack decides it should.

This is the same trade that log-structured databases, write-ahead logs, and LSM-tree storage engines make, for the same reason: a process that only ever appends can never corrupt data it already wrote. A crash mid-write leaves a partially written new record and an intact old one — never a half-overwritten file. The cost of that guarantee is that "deleted" data keeps occupying space until something comes along and reclaims it. On HuskHoard, that something is repack.

Where Dead Space Comes From

Every volume in the Husk Catalog has three kinds of bytes on it at any given time: live data that current stubs point to, dead data left behind by deletions and overwrites, and free space past the current write position that hasn't been used yet. Only the third kind can accept new writes, because the rule is "forward only."

01
project_alpha_final.mov
Written 2024-11-14. Still referenced by the catalog. Live.
live
02
broll_uncut_v1.mov
Superseded by v2 on 2025-01-08. Bytes are still here, but nothing points to them.
dead
03
broll_uncut_v2.mov
The current version. Written after v1, further along the volume. Live.
live
04
temp_render_cache.mov
Deleted by the user on 2025-06-02. Marked dead in the catalog; bytes untouched.
dead
05
write position →
Everything past here is free space. New files land here next, in order.
free

None of this is a problem while free space remains — writes keep landing at the end of the volume regardless of how much dead weight sits behind them. It becomes a problem when dead space accumulates to the point that a volume is, say, 70% full but only 30% of that is anything anyone would actually restore. At that point you're paying for cartridges and drive bays to store data nobody can reach anymore.

volume 83ad72b7 — 47.8 TB total72% occupied
live
16.3 TB · 34%
dead
18.1 TB · 38%
free
13.4 TB · 28%

A volume in that state is a good candidate for repack: more than a third of its capacity is dead weight, and reclaiming it means an entire cartridge goes back into the pool instead of sitting on a shelf holding nothing useful.

What Repack Actually Does

Repack does not edit the old volume. It never touches it in place — that would violate the append-only rule it exists to enforce. Instead, it reads everything live, writes it fresh to a different volume, and only then retires the original.

01
Consult the catalog for live files
HuskHoard asks the Husk Catalog for every stub that still points at this volume — the current version of every file, in written order, with dead and superseded entries already excluded.
02
Stream live data sequentially off the old volume
Live files are read in the order they sit on the medium, which is already close to optimal for a tape or spinning-disk head — no seeking back and forth hunting for scattered fragments.
03
Write it serially to a fresh volume
Each file is appended, one after another, onto a new (or already-blank) volume from the pool. Checksums are verified as data lands, so the new copy is confirmed byte-for-byte before anything old is discarded.
04
Repoint the catalog in one atomic update
Every stub that referenced the old volume now points at the new one, at its new offset. This happens as a single transaction — there is no window where a file is neither on the old volume nor the new one.
05
Retire and reformat the old volume
Only after the new volume is verified and the catalog is updated does the old volume get erased and returned to the pool as blank, reusable capacity.

The old volume now formated is holding nothing but its own blank capacity, ready to be the destination volume for the next write cycle. The 18 TB of dead weight it was carrying is simply gone — not compacted, not defragmented in place, just absent from the new copy because nothing asked for it to be carried forward.

Watch It Run

Repack, end to end

A walkthrough of husk repack against a live archive pool — dead-space detection, the sequential copy to a fresh volume, and the reclaimed cartridge going back into rotation.

Why Sequential Beats Random for Archival Media

It's fair to ask why HuskHoard doesn't just do what a desktop filesystem does — track free blocks and reuse them in place as soon as they're freed, the way SSD trim and disk-based filesystems do. The answer is that in-place reuse is a feature of the medium, not something a storage engine can add to media that doesn't support it, and even where it is supported, it isn't free.

One engine
Works identically on every medium
Tape can't seek-and-overwrite. WORM cartridges are physically forbidden from a second write. SSDs and hard drives can do random writes but pay a real cost for it. An append-only engine makes the same guarantees and runs the same code path regardless of which of these sits underneath a given volume.
Throughput
No fragmentation, ever
Because every write lands at the current position and nothing is ever patched in place, a file's bytes are always one contiguous run. There's no equivalent of filesystem fragmentation to defragment later — the closest thing to it is dead space, which repack clears in a single sequential pass rather than years of scattered housekeeping.
Safety
Nothing in progress can corrupt something finished
A write that's interrupted mid-stream — power loss, a yanked cable, a crashed daemon — only ever damages the new data being appended. Every file written before that point is untouched, because the append-only rule means it was never going to be touched again anyway.
Scheduling
Maintenance is a scheduled event, not a background tax
Random-write systems pay continuously for reuse — SSD garbage collection, write amplification, filesystem defrag jobs running quietly in the background. Repack is instead a single, plannable, sequential-read-then-sequential-write operation you can run overnight against one volume at a time, at a moment of your choosing.
Why not compact in place instead?

Compacting a volume in place would mean sliding live data backward to close the gaps left by dead entries — which is exactly the random, seek-heavy write pattern append-only storage exists to avoid, and which tape and WORM media can't perform at all. Reading everything live in one sequential pass and writing it to a second volume achieves the same result — a fully live, fully dense volume — using only the one operation every storage medium in the pool actually supports: writing forward.

Repack Is Also How the Catalog Stays Honest

There's a second job repack does that's easy to overlook: it's the point where the Husk Catalog's idea of a volume and the volume's actual physical contents are guaranteed to match exactly. Between repacks, the catalog is the only place that knows which bytes on a volume are still meaningful — the volume itself is happy to keep reporting its full occupied size, dead entries included, because nothing has told it otherwise.

A repack collapses that gap. The moment the new volume is written and the catalog transaction commits, every byte on that volume is live, every stub is correct, and the volume's reported usage is, for the first time since it started accumulating deletions, actually true. It's less a cleanup operation than a reconciliation — bringing the physical archive back into agreement with the record HuskHoard has been keeping about it all along.

The old volume, once erased, carries none of that history forward. It re-enters the pool as blank capacity, indistinguishable from a cartridge that has never been written to, ready for the next volume that needs it — sequentially, forward, one file after another.