6 minute read.Last Modified 2023-11-30 10:15 EST
There are several configuration recommendations and troubleshooting tips when using TrueNAS with a VMware hypervisor.
iSCSI IQN is an acronym that stands for “iSCSI Qualified Name”. It is comprised of the following naming schema with a preamble, node name and unique identifier:
VMware requires using an IQN in their software iSCSI implementation.
A VMware datastore backed by iSCSI-based storage will consist of at least three distinct pieces: the storage host, the switching infrastructure, and the VMware host itself. In order to maximize service availability, each of these elements needs to be able to tolerate some level of failure without significantly disrupting iSCSI traffic.
TrueNAS systems support high-availability (HA) through dual-controllers running in active/standby mode. A properly-configured HA TrueNAS system can offer up to 5x 9’s of system availability. TrueNAS also fully supports asymmetric logical unit access (ALUA) on iSCSI to significantly reduce failover time.
Network switching infrastructure can be made redundant and fault-tolerant through a number of methods, but multipathing is recommended as the best practice for iSCSI networks.
VMware’s official documentation details several ways the virtualization host(s) can be made redundant, so that is not covered here.
For a VMware ESXi host to communicate with an iSCSI capable storage array, the iSCSI protocol must be configured to provide: Discovery, Authentication, and Access Control (DAAC).
iSCSI offers two methods of target discovery: dynamic and static. Dynamic discovery lets the storage array respond automatically to the host initiator’s “SendTargets” request. Static discovery requires an administrator to manually add a list of the iSCSI targets to the initiator. Either method of discovery is fine, but dynamic discovery can make the iSCSI setup process easier.
iSCSI authentication is handled via the Challenge Handshake Authentication Protocol, or CHAP. CHAP uses a shared secret between targets and initiators to let them validate each other’s authenticity. By default, no CHAP-based authentication is performed by the VMware iSCSI initiator. If you do decide to use CHAP, authentication can either be unidirectional (where only the target authenticates the initiator) or bidirectional (where both the iSCSI initiator and the iSCSI target are required to authenticate to each other prior to transmitting iSCSI data).
VMware iSCSI initiators operating with unidirectional CHAP can be configured in two behavior modes. In “Required” mode, an iSCSI adapter will give precedence to non-CHAP connections, but if the iSCSI target requires it, the connection will use CHAP instead. Required mode is only supported by Software iSCSI and Dependent Hardware iSCSI adapters. Alternatively, initiators can run in “Prohibited” mode, where an iSCSI adapter will give precedence to CHAP connections, but if the iSCSI target does not support CHAP, the initiator can still connect.
Bidirectional CHAP (called “mutual CHAP” in TrueNAS) offers greater security by ensuring that both sides of the iSCSI connection authenticate against each other. Unidirectional CHAP does not let the iSCSI initiator authenticate the target, and running without CHAP obviously disables all authentication. For this reason, bidirectional CHAP is usually recommended but requires additional configuration and comes with greater administrative overhead when troubleshooting iSCSI connections.
Access control policies are set up within a storage array to ensure only certain initiators can connect to the target (even if they possess the correct CHAP password). Access control can be performed using the initiator’s name (IQN), its IP address, or its CHAP username.
The setup of vCenter iSCSI to TrueNAS requires that ESXi hosts be set up as initiators and TrueNAS storage arrays are set up as targets. To configure ESXi hosts with vCenter, see the VMware vCenter documentation.
To configure TrueNAS Enterprise storage arrays with vCenter, iXsystems has developed a vCenter plugin. The plugin uses TrueNAS REST APIs to automate LUN creation and assignment. When an VMFS (iSCSI) datastore is created using the plugin, the TrueNAS systems automatically activate their iSCSI system services.
When using TrueNAS as a VMware datastore:
Make sure guest VMs have the latest version of
vmware-toolsinstalled. VMware provides instructions to install VMware Tools on different guest operating systems.
Increase the VM disk timeouts to better survive long reboots or other delayed disk operations. Set the timeout to a minimum of 300 seconds. VMware provides instructions for setting disk timeouts on some specific guest operating systems:
- Windows guest operating system: https://docs.vmware.com/en/VMware-vSphere/6.7/com.vmware.vsphere.storage.doc/GUID-EA1E1AAD-7130-457F-8894-70A63BD0623A.html
- Linux guests running kernel version 2.6: https://kb.vmware.com/s/article/1009465
NOTE: Reboots or failovers will typically complete much faster than 300 seconds and Disk IO will resume automatically when finished.
When TrueNAS is used as a VMware datastore, you can coordinate creating and using ZFS and VMware snapshots. See VMware-Snapshots for details.
VMware’s VAAI allows storage tasks such as large data moves to be offloaded from the virtualization hardware to the storage array. These operations are performed locally on the NAS without transferring bulk data over the network.
VAAI for iSCSI supports these operations: