It’s only taken me 4+ weeks, but the filer is finally upgraded. Here’s hoping my random iops doubles (along with significant gains in streaming writes). The real reason I did this was because the first raidz stripe was ~completely full (30G free — down to 9GB free after several resilverings). I think this effectively reduced my io to the single new stripe. Once this gets baked in for a bit (and I’ve had time to make some more backups), I’ll throw my 5x400G drives back into the array for even more space. :)

Before:

[root@filer0 tgarner]# zfs list tank
NAME USED AVAIL REFER MOUNTPOINT
tank 3.96T 1.05T 52.7K /tank
[root@filer0 tgarner]# zpool list tank
NAME SIZE USED AVAIL CAP HEALTH ALTROOT
tank 6.37T 4.95T 1.41T 77% ONLINE -


After:

[root@filer0 ~]# zfs list tank
NAME USED AVAIL REFER MOUNTPOINT
tank 3.98T 3.17T 51.1K /tank
[root@filer0 ~]# zpool list tank
NAME SIZE USED AVAIL CAP HEALTH ALTROOT
tank 9.09T 4.97T 4.12T 54% ONLINE -