BDWB Shell Commands

This article describes bdwb, the command line tool for the App Workbench. This tool allows you to perform various functions, such as:

  • Add or modify images.
  • Create and modify App Store entries.
  • Manage roles in a multi-node, multi-service deployment.
  • Register services.
  • Configure clusters.
  • Add .conf files and init.d scripts to an application package.

Running BDWB

To run bdwb: switch to the directory where you want to begin creating the new application. You can run it one of two ways:

  • Batch Mode: You can run bdwb in a non-interactive mode directly from the command line, as described in Command Line Options, below.
  • Interactive: You can run bdwb using the interactive bdwb shell, as described in Interactive Commands, below.

Command Line Options

NOTE
This section describes the options available in version 3.6 of the App Workbench.

You can run bdwb from the command line by executing either of the following commands:

  • bdwb -i <instruction>, where <instruction> is one of the commands described in Interactive Commands, below. The specified instruction executes immediately.
  • bdwb <file_name>.wb, where <file_name>.wb is the name of a Workbench file that contains instructions for non-interactive processing.

Interactive Commands

To run bdwb interactively, execute the command bdwb from the command line. You can then run the following commands:

AppConfig

The appconfig command manages the packages that comprise an App Store bundle. This command has the following sub-commands:

  • list: List the configuration of the given package.
  • file: Add a local file path to the given AppConfig package. This sub-command uses the following arguments:
    • -f <path> or --filepath <path>: Full path to the AppConfig package on the local filesystem. Default is none.
    • --md5sum: The MD5 checksum of the AppConfig package. If not specified, the file checksum is calculated. Default is none.
    • --configapi <version>: Config API version used by the AppConfig package. Default is latest version, if not specified. See API Matrices.
  • autogen: Auto-generates a simple AppConfig bundle. This sub-command uses the following arguments:
    • --new <true|false>: Starts a new auto-generation package process and overrides any previous auto-generation package progress. Default is false.
    • --configapi <version>: Config API version used by the AppConfig package. Default is latest version, if not specified. See API Matrices.

    • --generate <true|false>: Auto-generate the AppConfig package that you previously created using the --new argument. Default is false.

      The following arguments are used to copy files to the deployed virtual node:

    • --pkgfile <file_name <file_name ... >>: Name(s) of the file(s) already in the AppConfig directory to be used as the source file(s). Default is none.
    • --destdir <directory>: Destination directory where the given file(s) is(are) to be copied on a deployed node. Default is none.

    • --dest <absolute_path>: Absolute path where the local file should be placed inside the container. Any directories necessary to put the file at the location are created. Default is none.

      The following argument appends files with cluster-specific properties:

    • --append <file_name>: Append files with cluster-specific properties. Absolute path of the configuration file inside the container. Default is none.

      The following arguments provide various options for handling custom scripts:

    • --execute <file_name>: Absolute path of the file to be executed inside the container. Default is none.
    • --sourcefile <file_path>: Path of the file to be sourced. Default is none.

    • --onroles <role_1> <role_2> ... <role_n>: Conditionally execute or source file on the specific virtual node role(s). By default, the scripts are executed or sourced on all the virtual node roles. Default is none.

      The following arguments are used to assign permissions to files or directories in the container image:

    • --abspath <absolute_path>: Absolute path of the file or directory whose permissions are being set. Default is none.
    • --perms <permissions>: RWX permissions to set for this file or directory. Default is none.
    • --uid <uid>: UID to set for the file or directory. Default is none.

    • --gid <gid>: GID to set for the file or directory. Default is none.

      The following arguments provide pattern replacement instructions for auto-generating config file customization.

    • --replace <file_name>: Absolute path of the configuration file inside the container. Default is none.
    • --pattern <pattern>: Pattern replacement instructions for auto-generating config file customization. Default is none. See Macros and Keys.
    • --macro ...: A command whose output is used to replace the pattern. This could be an SDK-defined function or a simple command invocation. Default is none. See Macros and Keys.
  • package: packages the AppConfig directory being developed. This sub-command uses the following arguments:
    • -d <package_directory> or --dir <package_directory>: A directory where the AppConfig scripts being developed are located. The directory name is used as the package name. If this argument is not specified, then the auto-generated AppConfig will be packaged. Default is none.
    • --configapi <version>: Config API version used by the AppConfig package. Default is latest version, if not specified. See API Matrices.
  • init: When you are manually developing an AppConfig script, this sub-command copies a few useful scripts that you can use as starter code. This sub-command uses the -d <destination_directory> or --dir <destination_directory> argument, which is the directory where the starter code is to be copied. This directory and all its parents will be created if they do not already exist. Default is none.
  • download: Downloads the AppConfig package from an HTTP url and adds it to the App Store entry. This sub-command uses the following arguments:
    • -u <setup_package_url> or --url <setup_package_url>: HTTP URL for downloading the AppConfig package. The file is downloaded to the staging directory. Default is none.
    • --md5sum <md5_sum>: The MD5 checksum of the AppConfig package, which is used to verify the checksum immediately after downloading. Default is none.
    • --configapi <api_version>: Config API version used by the AppConfig package. Default is latest version, if not specified. See API Matrices.

Attach

The attach command defines what other application can be attached. This command has the following sub-commands:

  • distro: Attach applications based on distro IDs. This sub-command uses the following argument:
    • -d <distro_id_1>...<distro_id_n> or --distroid <distro_id_1>...<distro_id_n>: A space-separated list of unique distro IDs that this application can attach to. Default is none.
  • service: Attach applications based on exported services and qualifiers. This sub-command uses the following arguments:
    • -s <service> or --service <service>: An application with the given service name is attachable to this cluster. Default is none.
    • -q <qualifier> or --qualifier <qualifier>: An optional service qualifier that further narrows down the attachable applications. Default is none.
  • category: Attach applications based on categories. This sub-command uses the following argument:
    • -c <category_1>...<category_n> or --category <category_1>...<category_n>: A space-separated list of categories that this entry will be available under during cluster creation. Any existing categories may be used, or new ones may also be defined here. Default is none.

Base Image

The baseimg command defines the base OS image to use for the application. This command uses the following sub-command:

  • init: When manually developing appconfig script, this copies a few useful scripts that you can use as starter code. This sub-command uses the following argument:
    • --os {centos7,rhel7,centos8,rhel8,ubuntu18}: Copies all the files related to building a Docker image that can be used as a base for apps on HPE Ezmeral Runtime Enterprise. These files are copied to the current directory. Default is none.

Builder

The builder command sets the organization name for the App Store entry. The organization name is used to disambiguate the distro ID of the entry as well as the name of the Docker image imported. This command uses the syntax builder -n <organization> or builder --name <organization>, where <organization> is the organization name to use for the entry. This must be a single word with no spaces. The input will be converted to all-lowercase if any mixed or uppercase characters are used. Default is none.

Catalog

The catalog command manages the App Store entry. This command has the following sub-commands:

  • new: Starts a session for creating a new App Store entry. Any previously-started sessions will be lost unless they were saved. This sub-command can use the following optional arguments:
    • --distroid <distro_id>: A distro ID that is unique across the entire HPE Ezmeral Runtime Enterprise Catalog. Default is none.
    • --name <name>: The name of the App Store entry. If the name includes spaces, then enclose it in double quotes, for example "Application Name". Default is none.
    • --depends_on <distro_id>: The distro ID of another App Store entry that this entry depends on. When used, this sub-command indicates that this is an add-on image. Default is none.
    • --desc <description>: The description for the App Store entry. Use double quotes to enclose the description. For example, "This is an application description.". Default is none.
    • -v <version> or --version <version>: Version of the App Store image formatted as x.y, where x is the major version and y is the minor version. Default is 1.0.
    • -c <categories <categories... > or --categories <categories <categories... >: Space-separated list of categories that this App Store entry will be available under during cluster creation. You may use any existing category or create a new one. Default is Hadoop.
    • --catalogapi <api_version>: Catalog API version used by the AppConfig package. Default is latest version, if not specified. See API Matrices.

    • --epic <target_epic_version>: The target HPE Ezmeral Container Platform version where the application package will be installed, such as

      --epic 5.2
      . See API Matrices.
  • save: Saves the current in-memory state of the App Store entry to a file. This sub-command can use the following optional arguments:
    • -f <path> or --filepath <path>: File path where to save the App Store entry JSON file. If not specified, the JSON file will be saved in the 'staging_dir' defined in bench.conf. Default is none.
    • --force <true|false>: Overwrites an existing catalog entry JSON file, if any. Default is false.
  • load: Loads an existing App Store entry. This sub-command uses the syntax load -f <:file_path> or load --filepath <:file_path>, which is the file path to an existing App Store entry JSON file.
  • modify: Allows selected fields in the App Store entry to be updated. This sub-command uses the following optional arguments:
    • --distroid <distro_id>: A distro ID that is unique across the entire HPE Ezmeral Runtime Enterprise Catalog. Default is none.
    • --name <name>: Catalog name for the end user. If the name includes spaces, then enclose it in double quotes, for example "Application Name". Default is none.
    • --depends_on <distro_id>: Distro ID of another App Store entry that this entry depends on. When used, this sub-command indicates that this is an add-on image. Default is none.
    • --desc <description>: Description for the end user. Use double quotes to enclose the description. For example, "This is an application description.". Default is none.
    • -v <version> or --version <version>: Version of the App Store formatted as x.y, where x is the major version and y is the minor version. Default is none.
    • -r or -recommend_distro: When set allows HPE Ezmeral Runtime Enterprise to recommend this distro when creating clusters in AI/ML tenants.
    • -c <categories <categories... > or --categories <categories <categories... >: Space-separated list of categories that this App Store entry will be available under during cluster creation. You may use any existing category or create a new one. Default is Hadoop.
    • -a <aiml_categories> or -aiml-categories <aiml_categories>: A space separated list of AIML categories. The available options are: AIML/Notebook, AIML/Training, and/or AIML/Deployment.
    • --catalogapi <api_version>: Catalog API version used by the AppConfig package. Default is latest version, if not specified. See API Matrices.
    • -epic <epic_version>: HPE Ezmeral Runtime Enterprise version the current app is being built for. The available options are: 3.1, 3.2, 3.3, 3.4, 3.5, 3.6, 3.7, 3.8, 3.9, 4.0, 5.0, 5.1, 5.2, and 5.3.
  • package: Package all components of the App Store entry into a bundle (.bin) file. This sub-command uses the following optional arguments:
    • -v <true|false> or --verbose <true|false>: Show details of the packing process. Default is false.
    • -o <operating_system> or --os <operating_system>: Determines which type of HPE Ezmeral Runtime Enterprise installation this App Store entry should be available for (any, centos, or rhel). For example, an RHEL entry will not show on an HPE Ezmeral Runtime Enterprise deployment that is installed on CentOS. Default is centos.

Cluster Config

The clusterconfig command manages the cluster configuration for an App Store entry. This command has the following sub-commands:

  • new: Creates a new configuration that maps various services and roles with each other. This command uses the syntax new -c <config_id> or new --configid <config_id> to specify a unique App Store configuration ID.
  • list: Lists details about the various cluster configurations for the current App Store. When one or more <config_id> are specified, only details for those configurations will be displayed. This command uses the syntax list <config_id <config_id ... >> to provide one or more space-separated config ID(s) for which to show details. If no IDs are provided, then the details of all currently-configured config IDs will be shown. Default is all.
  • assign: Defines a cluster configuration which essentially links various roles with the expected services. Multiple calls of this command with the same <config_id> can be used to associate different roles and service(s). This sub-command uses the following optional arguments:
    • -c <config_id> or --configid <config_id>: A unique App; Store configuration ID.
    • -r <role_id> or --roleid <role_id>: A unique App; Store-wide role identifier.
    • -s <service_id <service_id ...> or --srvcids <service_id <service_id ... >: Service ID(s) to be assigned to the role when this configuration is enabled. Default is none.

Define

The define command defines variables and/or constraints. This command has the following sub-command:

var <key_1=value_1> <key_2=value2> ... <key_n=value_n>: Defines one or more key(s) and assigns a value to each key. All occurrences of %KEY% in any subsequent App Workbench command will be replaced by the defined value. Neither the key nor the value may include a percent symbol (%), equal sign (=) or spaces.

Document

The document command provides container documentation management for the catalog entry. This command uses the following sub-commands:

  • file: Adds a local file path to the documentation for the catalog entry. This sub-command uses the following arguments:
    • -f <file_path> or --filepath <file_path>: File path to the documentation file on the local filesystem. Default is none.
    • -m <md5_sum> or --md5sum <md5_sum>: MD5 checksum of the documentation file. If this is not specified, then the checksum for the file is calculated. Default is none.
    • -t <mime_type> or --mimetype <mime_type>: Overrides the MIME type of the document file (such as text/markdown). If not specified, the MIME type is guessed from the file extension. Default is none.
  • list: Lists the configured document. This sub-command does not use any arguments.
  • download: Downloads the document file from an HTTP URL and adds it to the catalog feed entry. This sub-command uses the following arguments:
    • -l <document_url> or --url <document_url>: HTTP URL for downloading the document file. The file is downloaded to the staging directory. Default is none.
    • --md5sum <md5_checksum>: MD5 checksum of the document file that is used to verify the checksum immediately after downloading. Default is none.
  • init: Creates a template documentation markdown file. This sub-command uses the following argument:
    • --force {true,false}: Overwrite the documentation template file if it already exists. Default is false.

EOF

The EOF command exits the interactive bdwb shell.

Exit

The exit command exits the interactive bdwb shell.

Help

The help command provides help with the bdwb commands. It can be used as follows:

  • help: Lists the top-level bdwb commands.
  • help <command>: Describes the selected command. For example, help autogen provides help about the autogen command.
  • help <command> <option>: Describes the selected option. For example, help autogen destdir provides help about the destdir sub-command within the autogen command.

Image

The image command manages container images for the App Store entry. It has the following sub-commands:

  • build: Build an App Store image from a Dockerfile. Additional arguments can be passed to the docker build command by setting the environment variable AWB_DOCKER_BUILD_OPTS. This sub-command uses the following optional arguments:
    • -b <base_directory> or --basedir <base_directory>: Directory path where the Dockerfile and related files are located. Default is none.
    • -i <repotag> or --image-repotag <repotag>: Container name and tag for the newly-built image. This usually takes the form REGISTRY_HOST[:REGISTRY_PORT]/]REPOSITORY[:TAG]. Default is none.
    • -t <tags> or --additional-tags <tags>: Create additional tag(s) for the image that is built.
  • download: Download the image file from an HTTP URL and add it to the App Store entry.
    • --md5sum: The MD5 checksum of the image file. Used to verify the checksum immediately after downloading. Default is none.
    • --os <os>: OS distribution of the container image. Default is none.
    • --roles <role_1> <role_2> ... <role_n>: Assign the image to one or more specific virtual node role(s). If specified, the image is used for the role(s), such as master or worker, when that image is deployed on an HPE Ezmeral Runtime Enterprise cluster. This sub-command is only supported when the Catalog API version is 4 or higher. Also, the role must be previously defined in the metadata JSON file, such as by using the role add command. Default is all_roles.
    • -u <image_url> or --url <image_url>: HTTP URL for downloading the image. The file is downloaded to the staging directory. Default is none.
  • list: Lists the configured container image.
  • load: Loads an image. This sub-command uses the following optional arguments:
    • -f <file_path> or --filepath <file_path>: File path to the container image on the local filesystem. Default is none.
    • -i <repotag> or --image-repotag <repotag>: Container name and tag to save in the metadata. This usually takes the form REGISTRY_HOST[:REGISTRY_PORT]/]REPOSITORY[:TAG]. Default is none.
    • --md5sum <sum>: MD5 checksum of the appconfig package. If this is not specified, then the system calculates the checksum for the file. Default is none.
    • --os <os>: OS distribution of the container image. Default is none.
    • --roles <role_1> <role_2> ... <role_n>: Assign the image to one or more specific virtual node role(s). If specified, the image is used for the role(s), such as master or worker, when that image is deployed on an HPE Ezmeral Container Platform cluster. This sub-command is only supported when the Catalog API version is 4 or higher. Default is all_roles.
  • package: Pulls a Docker image and then packages it into the Catalog entry as a file. This sub-command uses the following optional arguments:
    • -i <repotag> or --image-repotag <repotag>: Container name and tag to save in the metadata. This usually takes the form REGISTRY_HOST[:REGISTRY_PORT]/]REPOSITORY[:TAG]. Default is none.
    • --os <os>: OS distribution of the container image. Default is none.

    • --roles <role_1> <role_2> ... <role_n>: Assign the image to one or more specific virtual node role(s). If specified, the image is used for the role(s), such as master or worker, when that image is deployed on an HPE Ezmeral Container Platform cluster. This sub-command is only supported when the Catalog API version is 4 or higher. Default is all_roles.

      The following arguments are available when the Docker registry requires authentication:

    • -u <username> or --username <username>: Username to be used when pushing the Docker image to a registry that requires authentication. You may also set the environment variable AWB_REGISTRY_USERNAME. Default is none.

    • -p <password> or --password <password>: Password to be used when pushing the Docker image to a registry that requires authentication. You may also set the environment variable AWB_REGISTRY_PASSWORD. Default is none.

      The following arguments are available when the Docker registry has content trust enabled:

    • -o <passphrase> or --ct-root-passphrase <passphrase>: Specifies the content trust root passphrase, if content trust is enabled for the Docker registry. You may also set the environment variable DOCKER_CONTENT_TRUST_ROOT_PASSPHRASE. Default is none.
    • -r <passphrase> or --ct-registry-passphrase <passphrase>: Specifies the content trust repository passphrase, if content trust is enabled for the Docker registry. You may also set the environment variable DOCKER_CONTENT_TRUST_REPOSITORY_PASSPHRASE. Default is none.
  • push: Pushes a Docker image to a registry and refers to it in the Catalog entry metadata. This sub-command uses the following optional arguments:
    • -i <repotag> or --image-repotag <repotag>: Container name and tag to save in the metadata. This usually takes the form REGISTRY_HOST[:REGISTRY_PORT]/]REPOSITORY[:TAG]. Default is none.
    • --os <os>: OS distribution of the container image. Default is none.

    • --roles <role_1> <role_2> ... <role_n>: Assign the image to one or more specific virtual node role(s). If specified, the image is used for the role(s), such as master or worker, when that image is deployed on an HPE Ezmeral Container Platform cluster. This sub-command is only supported when the Catalog API version is 4 or higher. Default is all_roles.

      The following arguments are available when the Docker registry requires authentication:

    • -u <username> or --username <username>: Username to be used when pushing the Docker image to a registry that requires authentication. You may also set the environment variable AWB_REGISTRY_USERNAME. Default is none.

    • -p <password> or --password <password>: Password to be used when pushing the Docker image to a registry that requires authentication. You may also set the environment variable AWB_REGISTRY_PASSWORD. Default is none.

      The following arguments are available when the Docker registry has content trust enabled:

    • -o <passphrase> or --ct-root-passphrase <passphrase>: Specifies the content trust root passphrase, if content trust is enabled for the Docker registry. You may also set the environment variable DOCKER_CONTENT_TRUST_ROOT_PASSPHRASE. Default is none.
    • -r <passphrase> or --ct-registry-passphrase <passphrase>: Specifies the content trust repository passphrase, if content trust is enabled for the Docker registry. You may also set the environment variable DOCKER_CONTENT_TRUST_REPOSITORY_PASSPHRASE. Default is none.
  • registry: Docker registry information. This sub-command uses the following arguments:
    • --auth-enabled: Specifies that the registry requires authentication. Please set the environment variables AWB_REGISTRY_USERNAME and AWB_REGISTRY_PASSWORD before invoking App Workbench. Default is False.
    • --trust: Specifies that content trust is enabled for the Docker images. Please set the environment variables DOCKER_CONTENT_TRUST_ROOT_PASSPHRASE and DOCKER_CONTENT_TRUST_REPOSITORY_PASSPHRASE before invoking App Workbench. Default is False.
    • --url <registry_host:port>: Registry URL and port specification. Default is none.
  • pull: Pulls an image from the repository. This sub-command uses the following arguments:
    • -i <image_reportag> or --image-repotag <image_repotag>: Container name and tag for the newly-built image. This usually takes the form REGISTRY_HOST[:REGISTRY_PORT]/]REPOSITORY[:TAG]. See Macros and Keys for more details. Default is none.
    • -t <new_repotag> or --retag <new_repotag>: Re-tag the image after pulling it from the remote registry. Default is none.
    • -u <username> or --username <username>: For an authentication-enabled registry, specifies the username for pulling the image from that registry. You may also set the environment variable AWB_REGISTRY_USERNAME. Default is none.
    • -p <password> or --password <password<>: For an authentication-enabled registry, specifies the password for pulling the image from that registry. You may also set the environment variable AWB_REGISTRY_PASSWORD. Default is none.
    • -o <passphrase> or --ct-root-passphrase <passphrase>: For a content-trust-enabled registry, specifies the content trust root passphrase. You may also set the environment variable DOCKER_CONTENT_TRUST_ROOT_PASSPHRASE. Default is none.
    • -r <passphrase> or --ct-registry-passphrase <passphrase>: For a content-trust-enabled registry, specifies the content trust repository passphrase. You may also set the environment variable DOCKER_CONTENT_TRUST_REPOSITORY_PASSPHRASE. Default is none.

Role

The role command manages the roles for the App Store entry. This command has the following sub-commands:

  • add: Add a new role to the current App Store entry. This sub-command uses the syntax add <role_id> <cardinality>, where <role_id> is a role ID that is unique to the App Store entry and <cardinality> is the cardinality for the role. Cardinality is a defined as a number followed by an optional plus sign (+). The number indicates the minimum number of nodes of that particular role that HPE Ezmeral Container Platform will force user to deploy, and the optional plus sign (+) indicates to HPE Ezmeral Runtime Enterprise that the user must be allowed to choose any number above the absolute minimum. For example:
    • 0+ means "zero nodes or more," which is generally used for Worker roles.
    • 1+ means "one node or more."
    • 2 means "two and only two nodes" of this role are allowed.
  • list: List details of role(s) defined in the current App Store entry. This sub-command uses the syntax list <role_id <role_id... >>, where you can add one or more space-separated role IDs. Default is all.
  • remove: Remove role(s) from the current App Store entry. This sub-command uses the syntax remove <role_id> <role_id... >>, where you can remove one or more specific space-separated role IDs. Default is all.

Service

The service command manages the services that are part of App Store entry. This command has the syntax service <option>>, where <option> is one of the following:

  • add: Adds a service to the App Store entry. This sub-command has the following arguments:
    • --srvcid <service_id> -n <service_name>, where:
      • <service_id> is the service ID, which is unique to the entire App Store entry. Default is none.
      • <service_name> is the name of the service to be displayed in the HPE Ezmeral Container Platform interface. Default is none.
    • --export_as <export_name> -s <scheme> --port <port> --path <path> --display <true|false>, where:
      • <export_name> is the name this service is exported as in the HPE Ezmeral Runtime Enterprise interface. Default is none.
      • <scheme> is the URI scheme for the service, if any. Default is none.
      • <port> is the URI port number, if any. Default is none.
      • <path> is the URI path for the service, if any. Default is none.
      • --display Displays the service to the user in the Cluster Details page. Adding this option sets it to true; default is false.
    • --sysv <service_name>: SystemV service name for managing the life cycle of the service. Default is none.
    • --sysctl <unit_name>: SystemD unit name for managing the life cycle of the service. Default is none.
    • --onroles <role_1> <role_2> ... <role_n>: Virtual node role(s) to which the service should be assigned by default. Default is none.

Sources

The sources command delivers source files as part of App Store bundles. This command has the syntax sources <package>, where <package> lists the package sources in the application bundle. The following files and directories are automatically packaged if they exist:

  • /appconfig
  • /images
  • logo file (if specified using the logo command, as described in Logo, above). For example, app_logo.png
  • instruction (if being used). For example, app_package.wb.
  • metadata JSON file

Workbench

The workbench command manages a workspace for developing an App Store entry. This command has the syntax workbench <option>>, where <option> is one of the following:

  • initapp: Initializes a workspace for developing a new App Store entry. This sub-command has the following arguments:
    • -apptype <type_of_app>: Type of application (either EPIC or KubeDirector).
    • -f or --force: Forces App Workbench initialization. Default is false.
  • clean: Cleans up all temporary artifacts and log files that may have been generated during the package process.
  • version: Displays the App Workbench version.