Working with Multiple Instances of the File System
The Multi-file system feature allows multiple instances of the file server to run on a single node in a single process.
Multiple instances of the data-fabric file server can run on a single node in a single process with the installation of the HPE Ezmeral Data Fabric File Store, or the HPE Ezmeral Data Fabric Database software. On servers with SSDs with at least 2 storage pools (SP), two instances (per node) are configured by default. On servers without SSDs, a single instance is configured by default. Each instance runs as a separate library that is dynamically loaded into a single process. In this mode, each instance has a separate host ID; however, all the instances share the same hostname.
The maximum number of supported instances is 32. Instances should be configured based on the available CPU, memory, disks, and SPs. Each instance needs a minimum of 2GB, and instances should not exceed:
- Number of CPUs / 2
- Number of SPs (enforced)
File Server Instances
On File Store and HPE Ezmeral Data Fabric Database
installations, nodes with SSDs can run multiple file server instances. To determine whether
a node has SSDs, data-fabric uses
the value of mfs.disk.is.ssd
in the mfs.conf
file, which must be set to 1
. Add this parameter to
mfs.conf
, on a node that has SSDs.
For clusters with File Store or HPE Ezmeral Data Fabric Database license, if
you set the mfs.disk.is.ssd
in the mfs.conf file to 1,
CLDB configures nodes with SSDs to have 2 file server instances by default. On homogeneous
clusters, you can modify the number of instances by changing the value of the multimfs.numsps.perinstance
parameter.
Ports for Multiple Instances of the file system
Each instance listens on its own set of ports. Ensure that the appropriate ports are open for this feature. For example, instance 0 will use four ports from 5660, 5692 (5660+32), 5724 (5660+64), and 5756 (5660+96), instance 1 will use four ports from 5661, 5693, 5725, 5757, and so on for every additional instance. The topology of all instances is the same.
The total number of instances depends on the number of MFS threads, as indicated by the
parameter mfs.numrpcthreads
in mfs.conf.
To verify that these ports are open, run the following command from a remote machine:
mrconfig -i -h <ip> -p <port number> info threads
An error indicates that the port is not open. If a port (for example, port 5661) is blocked, this command prints something similar to the following:
------------------
|From Instance 5661::|
------------------
<...> rpc failed <...>
Host IDs for Multiple Instances of the file system
For multiple instances of the file system, do NOT add IDs manually, but let the system handle the host ID numbering.
Log Files
Each instance has its own set of log files in $MAPR_HOME/logs
. When
multiple instances are configured, the log files have the same name with a different
instance ID; for example, mfs.log.<N>-3
where N
is the
instance number.
The RPC and security trace information are in a separate file per instance,
mfs-<N>.err
, where N
is the instance number. For the
primary instance, the file name does not include the instance number.
For example, suppose there are 2 instances running on ports 5660 and 5661. There are 2 sets of log files, one for each instance:
mfs.log-3
for the primary instancemfs.log.1-3
for the second instance
The RPC and security trace information are present in the following files :
mfs.err
for the primary instancemfs-1.err
for the second instance