Example: Mounting a PersistentVolume for Static Provisioning

About this task

The information on this page is valid for both FUSE POSIX and Loopback NFS plugins. Examples or tables that mention the FUSE POSIX driver (com.mapr.csi-kdf) are equally valid for the Loopback NFS driver (com.mapr.csi-nfskdf).

For static provisioning, configuring a PersistentVolume has some advantages over annotating Kubernetes volume information in a pod spec:
  • The configuration file can be shared for use by multiple pod specs.
  • The configuration file enables the PersistentVolume to be mounted and available even when the pod spec that references it is removed.

For example, suppose a marketing volume exists in the secure file system under the path /Departments/Marketing. An administrator wants to statically provision this volume and make it available to multiple users. It is critical that data access is as fast as possible. To make this work, the administrator must do the following:

Procedure

  1. Create a PersistentVolume (PV) (if you have already not statically provisioned a volume as described in this example) and set the following volumeAttributes:
    • accessMode of the PV to ReadWriteOnce
    • securityType parameter to secure because the volume is on a secure cluster
    • volumePath in the CSI driver setting to the desired path, and fill in the cldbHosts and cluster information
    • platinum parameter to use the POSIX platinum client or the license parameter to select from three POSIX clients
    For example: 
    apiVersion: v1
kind: PersistentVolume
metadata:
  name: test-simplepv
  namespace: test-csi
  labels:
    name: pv-simplepv-test
spec:
  accessModes:
  - ReadWriteOnce
  persistentVolumeReclaimPolicy: Delete
  capacity:
    storage: 5Gi
  csi:
    nodePublishSecretRef:
      name: "mapr-ticket-secret"
      namespace: "test-csi"
    driver: com.mapr.csi-kdf
    volumeHandle: test-simplepv
    volumeAttributes:
      volumePath: "/"
      cluster: "clusterA"
      cldbHosts: "10.10.102.96"
      securityType: "secure"
      platinum: "true"

    The preceding example specifies the high-performance Platinum POSIX license by including platinum: "true" in the volumeAttributes.

    If you have a Platinum FUSE POSIX license, Release 1.0.2 and later provide another way to control the POSIX client. Instead of specifying platinum: "true", you can specify license: "<license-name>" and select one of three POSIX licenses (Basic, Container, or Platinum). Release 1.0.2 also adds support for a startupConfig line that lets you pass custom startup parameters to the FUSE process. The following example shows these options: 
    apiVersion: v1
kind: PersistentVolume
metadata:
  name: test-simplepv
  namespace: test-csi
  labels:
    name: pv-simplepv-test
spec:
  accessModes:
  - ReadWriteMany
  persistentVolumeReclaimPolicy: Delete
  capacity:
    storage: 5Gi
  csi:
    nodePublishSecretRef:
      name: "mapr-ticket-secret"
      namespace: "test-csi"
    driver: com.mapr.csi-kdf
    volumeHandle: test-simplepv
    volumeAttributes:
      volumePath: "/"
      cluster: "clusterB"
      cldbHosts: "10.10.10.210"
      securityType: "secure"
      license: "container"
      startupConfig: "-o allow_other -o big_writes -o auto_unmount -o async_dio -o max_background=24 -o auto_inval_data --disable_writeback"
    The following table shows the properties defined in the sample PersistentVolume:
    Parameter Notes
    metadata: name The PersistentVolume name.
    metadata: namespace The namespace in which the PersistentVolume is stored.
    accessModes How the PersistentVolume is mounted on the host.
    NOTE
    The accessMode is not used to set the access mode bit on the volume. The accessMode of the PV and PVC should be the same so that they can bind.
    For more information, see Kubernetes Access Modes and Access Modes.
    persistentVolumeReclaimPolicy Specifies what happens to the volume when it is released by its claim. The Retain value keeps the PVC around for manual cleanup. Delete deletes the PV from Kubernetes.
    NOTE
    If this volume was created using dynamic provisioning, Delete causes the underlying volume to be deleted.
    For more information, see Reclaiming.
    capacity Specifies how big the allocated storage should be. This value is not validated against the quota or advisory quota. It is up to the person creating the PV to specify this value accurately.
    csi: nodePublishSecretRef The Ticket Secret for the CSI driver.
    nodePublishSecretRef:name The name of the Ticket Secret that contains the ticket to use when mounting to the cluster. See Configuring a Secret.
    nodePublishSecretRef:namespace The namespace that contains the Ticket Secret. Use the same namespace as the namespace used by the PersistentVolume.
    csi: driver The CSI driver being used. Call it by specifying driver: mapr.com/maprfs.
    volumeHandle The existing volume name or unique volume name for static provisioning.
    volumePath The mount point within the file system. This parameter specifies an existing path. For example, you can specify the root volume as "/", providing access to the entire filesystem.
    cluster The data-fabric cluster name.
    cldbHosts The hostname or IP addresses of the CLDB hosts for the cluster. You must provide at least one CLDB host. For fault-tolerance, providing multiple CLDB hosts is recommended. To specify multiple hosts, separate each name or IP address by a space.
    securityType A parameter that indicates whether tickets are used or not used. If tickets are used, specify secure. Otherwise, specify unsecure.
    platinum If set to platinum: "true", the POSIX client uses the platinum driver for better performance. Note that the platinum driver consumes more host resources and Platinum licenses.
    license (FUSE POSIX) Release 1.0.2 and later support the license: "<license-name>" parameter in addition to the platinum: "<true | false>" parameter for the FUSE POSIX plugin. The license: "<license-name>" parameter can have one of three values that control the number of host resources that are consumed:
    • "container" for the Container driver (one binary, 8 threads)
    • "basic" for the Basic driver (one binary, 64 threads)
    • "platinum" for the Platinum driver (multiple binaries, each running 64 threads)
    Note the following considerations for using the license: "<license-name>" parameter:
    • To use the license: "<license-name>" parameter, you must have a Platinum license.
    • If you specify both the platinum: "<true | false>" parameter and the license: "<license-name>" parameter, the platinum: "<true | false>" parameter overrides the license: "<license-name>" parameter.
    • If neither the platinum: "<true | false>" nor the license: parameter is specified, the container driver is implemented.
    startupConfig (FUSE POSIX) Release 1.0.2 and later support specifying the startupConfig line. The startupConfig line allows you to specify FUSE configuration parameters that are passed to the fuse.conf file. For the parameters that can be passed, see Configuring the HPE Ezmeral Data Fabric FUSE-Based POSIX Client.
    If no startupConfig line is specified, these default startup settings are used:
    "-o allow_other -o big_writes -o auto_unmount"

    The default settings allow other users to access the mount point, enable writes larger than 4 KB, and automatically unmount the filesystem when the process is terminated.

    The following example includes the three default settings and adds some additional settings (shown in bold):
    startupConfig: "-o allow_other -o big_writes -o auto_unmount -o async_dio 
-o max_background=24 -o auto_inval_data --disable_writeback"
    The additional settings enable asynchronous direct I/O, set the maximum number of asynchronous requests to 24, automatically invalidate the kernel FUSE cache for any data change that causes a change in the files, and disable the writeback cache.
    startupConfig (Loopback NFS) The startupConfig line allows you to specify configuration parameters that are passed to the nfsserver.conf file. For the parameters that can be passed, see nfsserver.conf. The startupConfig values supported for Loopback NFS are all the configs supported in the nfsserver.conf file. Values must be separated by a space. If no startupConfig line is specified, these default startup settings are used:
    startupConfig: "NFS_HEAPSIZE=1024 DrCacheSize=1024000"
    numrpcthreads Sets the number of RPC threads for the data-fabric client. The default value is 1. The maximum value is 4. Use this option to increase throughput with FUSE basic or container licenses or with the Loopback NFS driver.
    trackMemory Enables memory profiling to debug memory leaks in the FUSE or Loopback NFS process. To be enabled after direction from the DF support team. The default value is false.
    logLevel Sets the log level to one of the following values: error, warn, info, or debug. For the FUSE POSIX driver (com.mapr.csi-kdf), the default value is error. For the Loopback NFS driver (com.mapr.csi-nfskdf), the default value is info.
    retainLogs Retains the logs for the pod on the host machine. The default value is false.
    startDelay
    Sets the wait time after launching the FUSE or Loopback NFS processes and before making the mount available to the application pod. The default value is 5.
    CAUTION
    Setting the value below 2 to 3 seconds may affect the availability of the mount.
    See Example: Statically Provisioning a Volume Using the Container Storage Interface (CSI) Storage Plugin for more information.
  2. Create a PersistentVolumeClaim (PVC) spec and set the accessMode of the PVC to ReadWriteOnce.
    For example: 
    kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: test-simplepvc
  namespace: test-csi
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 5G
    The following table shows the properties used in the sample PersistentVolumeClaim:
    Parameter Notes
    metadata: name The PersistentVolumeClaim name.
    metadata: namespace The namespace in which the PersistentVolumeClaim is configured.
    accessMode How the requested PersistentVolume is mounted on the host.
    NOTE
    The PV and PVC modes should be the same so that they can bind.
    For more information, see Kubernetes Access Modes and Access Modes.
  3. Generate a service ticket, and create and deploy a ticket secret on the pod (if you have already not done it as described in steps 1 and 2 of this example).
    See maprlogin for information on generating a ticket and Configuring a Secret for information on creating and deploying a ticket secret.
  4. Create the pod spec and set the runAsUser and the fsGroup parameters to the UID and GID of the user that created the ticket.
    For example: 
    apiVersion: v1
kind: Pod
metadata:
  name: test-pv
  namespace: test-csi
spec:
  securityContext:
    runAsUser: 1000
    fsGroup: 2000
  containers:
  - name: busybox
    image: busybox
    args:
    - sleep
    - "1000000"
    resources:
      requests:
        memory: "2Gi"
        cpu: "500m"
    volumeMounts:
    - mountPath: /mapr
      name: maprflex
  volumes:
    - name: maprflex
      persistentVolumeClaim:
        claimName: test-simplepvc
    The following table shows the properties defined in the sample pod spec:
    Parameter Notes
    apiVersion The Kubernetes API version for the pod spec.
    kind The kind of object being created. The example uses a naked pod for clarity. Generally, it is better to use a Deployment, DaemonSet, or StatefulSet for high availability and ease of upgrade.
    metadata: name The pod name.
    metadata: namespace The namespace in which the pod runs.
    securityContext: runAsUser The user ID to run the container under. This user ID must be the same as the user ID for which the ticket was generated.
    securityContext: fsGroup The group ID to run the container under. This group ID must be the same as the group ID of the user for which the ticket was generated.
    volumeMounts: mountPath A directory inside the container that is designated as the mount path.
    volumeMounts: name A name that you assign to the Kubernetes volumeMounts resource. Matches with Volumes: name.
    Volumes: name A string to identify the name of the Kubernetes volumes resource. The value should match volumeMounts: name.
    persistentVolumeClaim: claimName The name of the PersistentVolumeClaim (PVC). For more information, see PersistentVolumeClaims.
  5. Deploy the .yaml file on the pod by running the following command: 
    kubectl apply -f <filename>.yaml
    For each Pod mount request, the POSIX client starts with the pod's hostname and new generated hostid, which is tracked on the data-fabric cluster. You can run the node list command on the data-fabric cluster to determine the number of POSIX clients. For example:
    FUSE POSIX
    # maprcli node list -clientsonly true
clienttype       clienthealth hostname                              ip                                       lasthb  id
posixclientbasic Inactive     4f3d34fe-2007-11e9-8980-0cc47ab39644  10.10.102.94,172.17.0.1,192.168.28.0     11225   7407394893618656436
posixclientbasic Inactive     7906d011-200f-11e9-84c0-0cc47ab39644  10.10.102.94,172.17.0.1,192.168.28.0     8174    7544602061076655421
posixclientbasic Inactive     9ed61912-2004-11e9-8980-0cc47ab39644  10.10.102.92,172.17.0.1,192.168.184.128  11224   2540810767207593086
posixclientbasic Inactive     c35ab639-2010-11e9-84c0-0cc47ab39644  10.10.102.94,172.17.0.1,192.168.28.0     7568    7947067275504513691
posixclientbasic Active       e5dc10e8-2012-11e9-84c0-0cc47ab39644  10.10.102.94,172.17.0.1,192.168.28.0     18      5849529086453778130
    Loopback NFS
    # maprcli node list -clientsonly true
clienttype    clienthealth  hostname                                  ip                              lasthb  id
LOOPBACK_NFS  Active        3ae5bb79-0aa1-431d-a17b-2cf0ef692060      10.163.160.104,192.168.252.65   1       3740102597316282880
LOOPBACK_NFS  Active        8c096a3c-0424-466a-8eda-6a61999ac3e4      10.163.160.103,192.168.19.192   1       6892565781040807680
LOOPBACK_NFS  Active        ae92fe4b-a3c9-4cb3-8858-c688dd6e0bdc      10.163.160.103,192.168.19.192   1       1038944668644089888
LOOPBACK_NFS  Active        fe855a47-bf66-4b72-8f28-c713b5ec4004      10.163.160.105,192.168.153.128  1       5958455784535826944

Example

Full example, which includes PV, PVC, and pod configuration 
apiVersion: v1
kind: PersistentVolume
metadata:
  name: test-simplepv
  namespace: test-csi
  labels:
    name: pv-simplepv-test
spec:
  accessModes:
  - ReadWriteOnce
  persistentVolumeReclaimPolicy: Delete
  capacity:
    storage: 5Gi
  csi:
    nodePublishSecretRef:
      name: "mapr-ticket-secret"
      namespace: "test-csi"
    driver: com.mapr.csi-kdf
    volumeHandle: test-simplepv
    volumeAttributes:
      volumePath: "/"
      cluster: "clusterA"
      cldbHosts: "10.10.102.96"
      securityType: "secure"
      platinum: "true"
---
apiVersion: v1
kind: Pod
metadata:
  name: test-pv
  namespace: test-csi
spec:
  securityContext:
    runAsUser: 1000
    fsGroup: 2000
  containers:
  - name: busybox
    image: busybox
    args:
    - sleep
    - "1000000"
    resources:
      requests:
        memory: "2Gi"
        cpu: "500m"
    volumeMounts:
    - mountPath: /mapr
      name: maprflex
  volumes:
    - name: maprflex
      persistentVolumeClaim:
        claimName: test-simplepvc
---
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: test-simplepvc
  namespace: test-csi
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 5G