TrueNAS SCALETrueNAS SCALE Nightly Development Documentation
This content follows experimental early release software. Use the Product and Version selectors above to view content specific to a stable software release.

Installing SCALE Enterprise (HA)

TrueNAS Enterprise

TrueNAS SCALE Enterprise is generally available with the release of SCALE 22.12.2. Do not attempt to install Enterprise High Availability systems with TrueNAS SCALE until it becomes generally available, unless the deployment is experimental in nature.

Installing TrueNAS SCALE on High Availability (HA) systems is complicated and should be guided by Enterprise-level support. Contact iXsystems Support for assistance whenever attempting to install TrueNAS SCALE on Enterprise HA hardware.

Contacting Support

Customers who purchase iXsystems hardware or that want additional support must have a support contract to use iXsystems Support Services. The TrueNAS Community forums provides free support for users without an iXsystems Support contract.

iXsystems Customer Support
Support Portalhttps://support.ixsystems.com
Emailsupport@ixsystems.com
Telephone and Other Resourceshttps://www.ixsystems.com/support/

Do NOT use Linux or CLI commands to recover or clean-install the SCALE iso file or configure any initial configuration settings! Incorrect use of CLI commands can further disrupt your system access and can potentially do greater damage to your system. Proceed at your own risk.

Installing SCALE for an Enterprise (HA) System

This article outlines a procedure to do a clean install of a SCALE Enterprise High Availability (HA) systems using an iso file.

HA systems are dual controller systems with the primary controller referred to as controller 1 (sometimes also as controller A) and controller 2 (or controller B).

For best results, we recommend executing this procedure on both controllers at the same time. You can simultaneously install using two USB flash drives inserted into the USB port for each controller (1 and 2) or by establishing an IPMI connection with each controller in separate browser sessions.

Alternately, install and configure controller 1 while keeping controller 2 powered off. When controller 1 is completely configured, power on controller 2 to install TrueNAS and reboot the controller. When controller 2 boots after installing, sync the system configuration from controller 1 to controller 2.

SCALE includes features and functions to help with completing the configuration process after installing and getting access to the SCALE web interface.

Preparing for a Clean Install

For a list of SCALE Enterprise (HA) preparation information, see Preparing for SCALE UI Configuration (Enterprise).

Have this information handy to complete this procedure:

  • All the assigned network addresses and host names (VIP, controller 1 and 2 IP addresses).
  • Other network information including domain name(s), and DNS server, default gateway, alias, or other static IP addresses.
  • The IPMI access addresses for each controller and the administration credentials for IPMI access to these addresses.
  • SCALE license file provided by iXsystems.
  • SCALE Storage Controller 1 (A) and 2 (B) serial numbers (refer to contracts or documentation provided with the system or contact iXsystems Support and provide your contract number).
HA system controllers each have serial numbers, the lower number assigned is for controller 1 (e.g. of two controller serial numbers assigned A1-12345 and A1-12346, the A1-12345 is for controller 1 and A1-12346 is for controller 2).

When restoring after a clean install, also have ready:

  • Storage data backups to import into the Enterprise HA system.
  • System configuration file from the previous TrueNAS install.

Overview of the Installation Procedure

For best results, we recommend executing this procedure on both controllers at the same time. You can simultaneously install using two USB flash drives inserted into the USB port for each controller (1 and 2) or by establishing an IPMI connection with each controller in separate browser sessions.

Alternately, install and configure controller 1 while keeping controller 2 powered off. When controller 1 is completely configured, power on controller 2 to install TrueNAS and reboot the controller. When controller 2 boots after installing, sync the system configuration from controller 1 to controller 2.

There are two ways to install the HA dual controller system to ensure controller 1 comes online as the primary controller:

  • Install both controllers simultaneously beginning with controller 1, then immediately starting the install on controller 2.
  • Installing each controller individually to specific points in the installation process.

Simultaneous installation must start with controller 1, so it comes online first. Installing each controller individually follows a particular method to ensure controller 1 comes online as the primary controller.

The sections in this article cover the primary steps as a simultaneous installation:

  1. Download the iso file from the TrueNAS website and prepare the USB flash drives if not using IPMI for remote access.

  2. Log into your IPMI system using the network address assigned to controller 1, and then establish a second connection with controller 2 in a new browser session.

  3. Install SCALE using the iso file and select the Fresh Install option. Install on controller 1, then immediately begin installing on controller 2 in the other IPMI session to simultaneously install SCALE on both controllers.

  4. Disable DHCP, then enter the network settings to controller 1 using the Console Setup Menu. Enter the IP address and netmask assigned to controller 1, then enter the global network settings for host name, domain name, and nameservers.

    Use the SCALE UI for system configuration as it has safety mechanisms in place to prevent disrupting network access that could require you to repeat the clean install to access your system. However, if you are experienced with the Console Setup Menu and are using it to configure network settings you can configure the rest of the controller 1 network settings with the Console setup menu.

  5. Log into the SCALE UI for controller 1 to sign the EULA agreement and apply the system HA license.

  6. Disable failover to configure the rest of the network settings and edit the primary network interface on controller 1, and then enable failover.

  7. Complete the minimum storage requirement by adding or importing one pool on controller 1.

  8. Sign in using the Virtual IP (VIP) address.

  9. With controller 2 powered up, on controller 1 sync to peer to complete the install and make controller 2 the standby controller.

The sections that follow describe these steps in detail.

Overview of the Alternative Installation Process

This process of installing each controller sequentially has two methods:

  • Install and configure controller 1 up to the point where you are ready to sync to controller 2. When complete, install controller 2 and reboot. After the console setup menu displays, switch back to controller 1 and sync to peer. This synchronizes the completed configuration from controller 1 to controller 2 and keeps controller 1 designated as the primary controller. Or
  • Begin installing controller 2 immediately after installing controller 1. When controller 2 finishes installing, power it off and keep it powered down. When finished configuring controller 1, power up controller 2 and wait for it to finish booting up. Switch back to controller 1 and sync the configuration to controller 2.

This section provides an overview of the alternative method to clean install an HA system with controller 2 powered off while installing and configuring controller 1. These steps are nearly identical to the section above but controller 2 is either powered off or not installed while you install and configure controller 1.

  1. Use either the prepared USB flash drive inserted into a USB port for controller 1 or log into an IPMI session and install SCALE on controller 1. Finish the installation and allow controller 1 to complete its first boot.
  2. Use either the prepared USB flash drives inserted into a USB port for controller 2 or log into an IPMI session for controller 2 to install SCALE. When the installation finishes, power down controller 2.
  3. Configure the required network settings on controller 1 with the Console setup menu.
  4. Log into controller 1 using the IP address assigned to controller 1. Apply the HA license, sign the EULA, and complete the UI configuration to the point where you are ready to sync to peer on controller 1, but do not sync yet.
  5. Power up controller 2 and wait for it to complete the boot process.
  6. Log into controller 1, go to System Settings > Failover, and click Sync to Peer. This synchronizes controller 2 with controller 1 and reboots controller 2. Controller 2 becomes the standby controller when it finishes rebooting.

Downloading the SCALE Install File

Download the .iso file.

If you are remote to the system and are installing through an IPMI connection you do not need to save the .iso file to a USB flash drive.

If you are physically present with the TrueNAS SCALE system, burn the .iso file to a USB flash drive and use that as the install media.

Using IPMI to Install the ISO on a Controller

Use this process to install the iso file on both controller 1 and controller 2. Best practice is to begin the install on controller 1, then immediately begin the install on controller 2.

Installing ISO Steps
  1. Enter the IP address assigned to the controller 1 IPMI port into a web browser and log into your IPMI system with admin credentials.

  2. Select Remote Control > iKVM/HTML5 to open the Console Setup window.

    IPMI interfaces can vary but they generally have options for Remote Control and iKVM/HTML5 to open a console session on the platform.

  3. Install the .iso file. Select the Virtual Media > CD-ROM image option in your IPMI system.

    a. Enter the IP address of where you downloaded the .iso file into Share Host. You might need assistance from your Network or IT department to obtain this address.

    b. Enter the path to the .iso file. For example, if you stored the file in an iso folder enter /iso/TrueNAS-SCALE-22.12.1.iso in Path to Image.

    c. Click Save, then Mount. You should see the .iso file under Device 1 or the device name your IPMI configures.

  4. Return to the Remote Control > iKVM/HTML5 window opened in step 2. Either use your keyboard or open the keyboard in the window then:

    a. Type 8 to reboot controller 1 (also repeat for controller 2), and type y to confirm and reboot.

    b. As the system reboots, be prepared to hit the F11 key when you first see the TrueNAS Open Storage splash screen. Alternatively, you can start clicking on the F11 key on the online keyboard until you see the TrueNAS SCALE Installer screen.

    c. Select the UEFI: ATEN Virtual CDROM device from the boot list. The bootstrap loader begins. When it ends the SCALE installer opens.

Using the SCALE Installer

If you are doing a clean install from the SCALE .iso file to recover from an issue that requires you to re-install SCALE from the .iso, have your network configuration information ready to use for controller 1 after the installation completes. Do not configure network settings on controller 2. Also have your SCALE system configuration file and data backups handy, so you can recover your system settings and import your data into the recovered SCALE clean-install system.
SCALE Installer Steps

  1. Select Install/Upgrade.

  2. Select the desired install drive.

    Select Yes to proceed with a clean installation of TrueNAS SCALE from the .iso. This erases the contents of the selected drive!

  3. Select option 1 Administrative user (admin) then OK to install SCALE and create the admin user account and password. SCALE has implemented the administrator login as a replacement for the root user login as a security hardening measure. The system retains root as a fallback, but it is no longer the default. The admin account has full control over TrueNAS and is used to log in to the web interface. Set a strong password and protect it.

    Next, enter a password for the new admin user.

  4. (Optional) Select Yes and press Enter. When the operating system device has enough additional space, you can choose to allocate some space for a swap partition to improve performance.

  5. Select Yes at the Legacy Boot prompt to allow the system to boot via UEFI, or select No if your system hardware requires legacy BIOS boot. Press Enter to begin the installation.

  1. Select OK after the The TrueNAS installation on succeeded displays. The Console setup menu screen displays.

  2. Enter 3 to Reboot System and immediately return to the IPMI Virtual Media > CD-ROM image screen to click Unmount. Click Save. If you fail to unmount the iso image before the system completes the reboot, the bootstrap install continues in a boot loop.

SCALE is now installed on controller 1 and repeated for controller 2 starting with Using IPMI to Install the ISO on a Controller.

Configuring the Network with Console Setup Menu

After installing the SCALE .iso file on both controller 1 and 2 and finishing the TrueNAS SCALE Installer process, use the Console setup menu to configure the required network settings on controller 1, so it can access the SCALE UI. TrueNAS SCALE single controller systems use the DHCP-assigned IP address for the primary network interface to access the SCALE UI to complete the rest of the network and other configuration settings. However, HA systems with dual controllers must use static IP addresses.

To allow controller 1 to access the UI, you must disable DHCP and add the controller 1 static IP address and netmask as an alias on the primary network interface, and then enter the network settings for host name, domain name, default gateway, and the name servers (1 and 2). You can configure the rest of the HA global network settings in the SCALE web UI.

To use the Console setup menu to configure required network settings on controller 1:

TrueNAS SCALE Console Setup Menu
Figure 8: TrueNAS SCALE Console Setup Menu
  1. Type 1 and then press Enter to open the Network Interfaces screen.

  2. Use either Tab or the arrow keys to select the interface assigned as your primary network interface. If you have more than one interface installed and wired to your network, the primary interface is typically eno1. With the interface highlighted, press Enter to open the Update Network Interface screen.

  3. Tab or arrow down to ipv4_dhcp and change it to no.

  4. Tab or arrow down to the aliases setting and enter the static IP address for controller 1. Tab or arrow down to Save, and then press Enter. A pending network changes notice displays with additional options.

  5. Type a to apply the change, then p to make it persist. Type q to return to the main Console setup menu.

  6. Type 2 and then press Enter to open the Network Configuration screen.

  7. Use either Tab or the arrow keys to select each field. Type the value for each field listed below. Press Enter after each value.

    FieldDescription/Example
    hostnameThe host name you assign to controller 1. For example m50-123-1.
    domainThe domain name for the nework controller 1. For example my.companyname.net
    ipv4gatewayThe default gateway IP address for your network.
    nameserver1
    nameserver2
    The IP addresses for your network DNS servers.
  8. Use either Tab or the arrow keys to select Save, then type q to return to the main Console setup menu.

Configuring Settings in the SCALE UI

This section only applies to controller 1. Do not configure settings on controller 2.

Use the SCALE UI to:

  1. Apply the HA license.
  2. Complete the network settings.
  3. Add the first storage pool.
  4. Sync controller 1 with controller 2.

Applying the HA License

SCALE UI Enterprise customers see the End User License Agreement (EULA) screen the first time they log in. Sign the agreement to open the main SCALE Dashboard. Apply the system license next.

Go to System Settings > General and click Add License on the Support widget. Copy your license and paste it into the License field, then click Save License. The Reload dialog opens. Click Reload Now. Controller 1 restarts, and displays the EULA for controller 2. Sign the EULA agreement for controller 2, and add the license.

The controller 1 and 2 (or a and b) serial numbers display on the Support widget on the System Settings > General screen.

Configuring Network Settings

Both controllers must be powered on and ready before you configure network settings.

You must disable the failover service before you can configure network settings!

Only configure network settings on controller 1! When ready to sync to peer, SCALE applies settings to controller 2 at that time.

SCALE Enterprise (HA) systems use three static IP addresses for access to the UI:

  • VIP to provide UI access regardless of which controller is active. If your system fails over from controller 1 to 2, then fails over back to controller 1 later you might not know which controller is active.
  • IP for controller 1. If enabled, DHCP assigns an IP to the primary network interface on non-HA systems. Disable DHCP, and then manually enter the Controller 1 static IP address your network administrator assigned for this controller.
  • IP for Controller 2. Manually enter the second IP address assigned for this controller.

Have the list of network addresses, name sever and default gateway IP addresses, and host and domain names ready so you can complete the network configuration without disruption or system timeouts.

SCALE safeguards allow a default of 60 seconds to test and save changes to a network interface before reverting changes. This is to prevent users from breaking their network connection in SCALE.

To configure network settings on controller 1:

  1. Disable the failover service. Go to System Settings > Services locate the Failover service and click edit. Select Disable Failover and click Save.

  2. Edit the global network settings to add the controller and virtual hostnames and update any other network settings.

  3. Edit the primary network interface to add failover settings. Go to Network and click on the primary interface eno1 to open the Edit Interface screen for this interface.

    a. Turn DHCP off if it is on. Select DHCP to clear the checkbox.

    b. Add the failover settings. Select Critical, and then select 1 on the Failover Group dropdown list.

    c. Add the virtual IP (VIP) and controller 2 IP. Click Add for Aliases to display the additional IP address fields.

    First, enter the IP address for controller 1 into IP Address (This Controller) and select the netmask (CIDR) number from the dropdown list.

    Next, enter the controller 2 IP address into IP Address (TrueNAS Controller 2).

    Finally, enter the VIP address into Virtual IP Address (Failover Address).

  4. Click Save

  5. Click Test Changes after editing the interface settings. You have 60 seconds to test and then save changes before they revert. If this occurs, edit the interface again.

Adding the Storage Pool

Create or import a storage pool from a backup. You must have at least one storage pool on controller 1. After saving the storage pool, controller 2 automatically restarts. Wait until it comes back online before syncing controller 1 with controller 2.

For more information on how to create a new pool click here. For more information on how to import a pool click here.

Syncing Controller 1 and 2

  1. Turn the failover service back on. Go to System Settings > Services locate the Failover service and click edit.

  2. Select Disable Failover to clear the checkmark and turn failover back on, then click Save. The system might reboot. Use IPMI to monitor the status of controller 2 and wait until the controller is back up and running.

  3. Log out of the controller 1 UI, and log in using the VIP address.

  4. Sync controller 1 and 2. With controller 2 powered on, but not configured, from controller 1 click Sync To Peer. Select Reboot standby TrueNAS controller and Confirm, then click Proceed to start the sync operation. This sync controller 2 with controller 1 which adds the network settings and pool to controller 2.

When the system comes back up, log into SCALE using the virtual IP address. The main Dashboard displays two System Information widgets. In standard configurations by iXsystems, Controller 1 shows its serial number and a host name that includes the letter a. Controller 2 is labeled as Standby Controller and shows its serial number and a host name that includes the letter b. Take note of this information.

Main Dashboard for HA Systems
Figure 16: Main Dashboard for HA Systems

Troubleshooting HA Installation

If controller 2 comes online as the primary and controller 1 as the standby, you installed and configured the controllers incorrectly.

  1. Go to System Settings > Failover, clear the Default TrueNAS Controller option, and click Save. The system reboots and fails over to the current standby controller (in this case, to controller 1).

  2. Log back into the UI with the VIP address. Go to System Settings > Failover and select Default TrueNAS Controller to make controller 1 the primary controller.

  3. Select Sync to Peer. SCALE makes controller 2 the standby controller and syncs the configuration on controller 1 to controller

  4. Click Save.