Deleting files when your ZFS pool is 100% full

Author: Maarten Hoogendoorn
Date: May 2016

I've been running ZFS on my home server for a few years now.

Since my switch to NixOS late last year, I've been running ZFS on my laptop with an SSD as well. I've been quite happy so far! Creating (consistent) backups has never been easier.

However, somewhere in February, my disk filled up completely. I tried to fix this by simply removing some large files. This did not work; I was unable to delete any file! ZFS is a Copy-on-Write filesystem, which implies that it needs to allocate a new block for the new metadata of the directory, which is impossible if you're disk is full.

After some quick Google searches, the remedy appears to be to truncate files. So after running cat /dev/null > /path/to/some/big/file several times, I was able to recover enough free space to be able to delete more files.

To prevent this from happening again, I devised a trick to force some free blocks to be available: just create a new (unmounted) ZFS file system, and use the reseveration property to reserve some physical disk space to it.

[root@maarten-laptop:/home/maarten]# zfs create mlaptop/reserve
[root@maarten-laptop:/home/maarten]# zfs set reservation=1G mlaptop/reserve

If you're disk reaches 100% again, simply remove this file system with.

[root@maarten-laptop:/home/maarten]# zfs destroy mlaptop/reserve

Today, my disk filled up again, and the workflow of removing the extra file system, removing files and recreating a new reservation file system is a lot less stressfull than having to truncate files.