If you're going to upgrade your existing ZFS pool drives from 512/512e sector size (ashift=9) to newer AF/4k drives (ashift=12) you'd be better off creating a new pool with ashift=12 and using zfs send recv to populate the new pool with your data.
If you are researching prior to creating any pool, force ashift=12 during pool creation, even if you're using 512/e drives by using zpool create -o ashift=12 yourtank /dev/drive1 /dev/drive2 ...etc This will 'future-proof' your ZFS pool.
You may receive a similar error in FreeNAS:
"nas manage.py: [middleware.exceptions:38] [MiddlewareError: Disk replacement failed: "cannot replace (driveID) with gptid/(gptID): devices have different sector alignment, "
To check your current ashift value, use:
zdb -C | grep ashift
ashift: 9 = 2^9 = 512bytes (bad)
ashift: 12 = 2^12 = 4096bytes (good)
If you absolutely can't do that, these two sysctls allow you to use 4k drives as if they're 512/e. There is a significant performance hit if you do this.
Sysctls:
Set these sysctls via ssh or using the Sysctl UI underneath 'System' in the FreeNAS web interface.
vfs.zfs.vdev.larger_ashift_disable=1
vfs.zfs.vdev.larger_ashift_minimal=0
Then you should be able to offline/replace/resilver a 4k drive into your 512/e pool. Personally my throughput greatly increased despite ashift=9, but I moved my ZFS pool from 6x2TB 5400RPM to 6x2TB 7200RPM. Again, the right way to do this is to use ashift=12 with 4k drives.
crit?: 0