FreeNAS was configured and installed as a VM on ESX6 using pass PCI passthrough for the HBA. iSCSI performance seemed uneffected. From a VM on the same host as FreeNAS write speeds were in the 30 MB/s range and reads were under 5 MB/s. The solution was two fold.
Improving Write Speeds:
Freenas was configured to use synchronous writes. This essentially meant that the client waited for FreeNAS to confirm every block written. While this ensures data was not lost in the case of a power outage, it did make for poor performance. Sync can be disable on the FreeNAS using the command
zfs set sync=disabled XXX
where XXX is the name of the NFS share.
Improving Read Speeds:
Autofs was being used to manage the mounting of the NFS volumes. The parameters used in the
the auto.nfs files was “music -fstype=nfs,rw,async,vers=3,lookupcache=none,noac,tcp freenas:/mnt/vol2/music”
After the mount options were changed to “music -fstype=nfs,async freenas:/mnt/vol2/music” performance dramatically increased.
Speed Tests:
dd bs=1M count=256 if=/dev/zero of=test oflag=dsync
268435456 bytes (268 MB) copied, 1.17235 s, 229 M