Macros and Keys

Application configuration scripts that need to be populated with dynamic values (such as an IP address or FQDN) can do so by specifying a pattern in that config file by including the following command in the .wb file:

appconfig autogen --replace <filename> --pattern <pattern> --macro <macro>

This command replaces the <pattern> with the output of the <macro> in the <filename>.

There are three types of macro:

Inline: Simple commands like echo $VARIABLE may be defined directly when specifying the pattern replace instruction in the .wb file. Any valid bash statement can be used as an inline macro.
Macros defined by App Workbench: This article describes those in detail.
User-defined macros: Users can define their own macros in a file and use the command appconfig autogen --sourcefile <file> to make those macros available to the pattern replace command. The sourcefile command must be specified before any pattern replace command(s) that may use the custom macros.

The following macros are available:

NOTE This list is presented in the same order in which it appears in the macro definition file macros.sh.

Node Details: Gets details about a virtual node. See Node Details.
Get Cluster Configuration Choice: Returns the value of the specified key. See Get Cluster Configuration Choice.
Get Cluster Configuration Metadata: Returns the value of the specified metadata key. See Get Cluster Configuration.
Get Specific Configuration Choice Key: Returns the value of a specific configuration choice key. See Get Specific Configuration Choice Key.
Get Specific Metadata Choice Key: Returns the value of a specific metadata choice key. See Get Specific Metadata Choice Key.
Get a Unique Integer for This Host: Returns an integer for the current host. See Get This Host Number.
Get a Unique Integer for Another Host: Returns an integer for another host. See Get Other Host Number.
Get Unique Integer by Service: Returns a unique integer for a service. See Get Unique Integer.
Get Total Available VRAM: Returns the amount of available virtual RAM in MB. See Get Total Available VRAM.
Get Total Available VCPU Cores: Returns the number of virtual CPU cores available to Spark. See Get Total Available VCPU Cores.
Get FQDN List: Returns the FQDNs of the virtual nodes in the cluster. See Get FQDN List.
Get IP Address List: Returns the IP addresses of the virtual nodes in the cluster. See Get IP Address List.
Get Virtual Node FQDN: Returns the FQDN of a specific virtual node. See Get Virtual Node FQDN.
Get Virtual Node IP Address: Returns the IP address of a specific virtual node. See Get Virtual Node IP Address.
Generate Application-Specific URL: Generates a URL for the specified application. See Generate Application-Specific URL.
Get Tenant Information: Gets the value of the specified tenant namespace key. See Get Tenant Information.

This article presents the following information for each macro:

Description: Function of the macro.
Input(s): Information that must be supplied to the macro.
Output: Output of the macro for both success and failure outcomes.
Usage Example: Generic usage example for the macro.

Node Details

The following node details can be obtained for a Docker container (virtual node) in a cluster:

FQDN
Role
Domain
Distribution
Hostname
Nodegroup ID
Dependencies
DataTap.jar
Cluster Name
Total Virtual CPU Cores
Total Virtual RAM

FQDN

Name

FQDN

Description

The fully qualified domain name of the Controller.

Usage Example

appconfig autogen --pattern @@@@MYPATTERN@@@@ --macro FQDN

Role

Name

ROLE

Description

Role of the Container (such as Master, Worker, or Edge).

Usage Example

appconfig autogen --pattern @@@@MYPATTERN@@@@ --macro ROLE

Domain

Name

DOMAIN

Description

Domain to which the virtual node belongs.

Usage Example

appconfig autogen --pattern @@@@MYPATTERN@@@@ --macro DOMAIN

Distribution

Name

DISTRO

Description

Distribution (App Store application) being run on the virtual node.

Usage Example

appconfig autogen --pattern @@@@MYPATTERN@@@@ --macro DISTRO

Hostname

Name

HOSTNAME

Description

Hostname of the virtual node.

Usage Example

appconfig autogen --pattern @@@@MYPATTERN@@@@ --macro HOSTNAME

Nodegroup ID

Name

NODEGROUP

Description

ID number of the nodegroup to which the virtual node belongs.

Usage Example

appconfig autogen --pattern @@@@MYPATTERN@@@@ --macro NODEGROUP

Dependencies

Name

DEPENDS_ON

Description

The distro_id of the primary nodegroup that this node depends on.

Usage Example

appconfig autogen --pattern @@@@MYPATTERN@@@@ --macro DEPENDS_ON

DataTap.jar

Name

DTAP_JAR

Description

The published location of the DataTap jar, which should be copied appropriately by the application configuration scripts.

Usage Example

appconfig autogen --pattern @@@@MYPATTERN@@@@ --macro DTAP_JAR

Cluster Name

Name

CLUSTER_NAME

Description

Name of the cluster to which the virtual node belongs.

Usage Example

appconfig autogen --pattern @@@@MYPATTERN@@@@ --macro CLUSTER_NAME

Total Virtual CPU Cores

Name

TOTAL_VCPU

Description

Total number of virtual CPU cores assigned to the virtual node.

Usage Example

appconfig autogen --pattern @@@@MYPATTERN@@@@ --macro TOTAL_VCPU

Total Virtual RAM

Name

TOTAL_VMEM

Description

Total amount of virtual RAM assigned to the virtual node, in MB.

Usage Example

appconfig autogen --pattern @@@@MYPATTERN@@@@ --macro TOTAL_VMEM

Get Cluster Configuration Choice

Name

CLUSTER_CONFIG_CHOICE

Description

Returns the value of a specific configuration choice key for any nodegroup in the virtual cluster. Each cluster can have multiple nodegroups.

Inputs

NGID: Nodegroup ID. This is usually the nodegroup to which the current virtual node belongs, so \$NODEGROUP will be used.
KEY: Configuration choice key for which to return the value. This is application-specific.

Outputs

This macro returns the following information:

Success: Returns the value for the requested key.
Failure: Nothing; exits the script with a non-zero status.

Usage Example

appconfig autogen --pattern @@@@MYPATTERN@@@@ --macro CLUSTER_CONFIG_CHOICE \$NODEGROUP <key>

Get Cluster Configuration Metadata

Name

CLUSTER_CONFIG_METADATA

Description

Returns the value of a specific configuration metadata key for any nodegroup within the virtual cluster.

Inputs

NGID: Nodegroup ID. This is usually the nodegroup to which the current virtual node belongs, so \$NODEGROUP will be used
KEY: Configuration choice key for which to return the value. This is application-specific.

Outputs

Success: Returns the value for the requested key.
Failure: Nothing; exits the script with a non-zero status.

Usage Example

appconfig autogen --pattern @@@@MYPATTERN@@@@ --macro CLUSTER_CONFIG_METADATA \$NODEGROUP <key>

Get Specific Configuration Choice Key

Name

NODEGROUP_CONFIG_CHOICE

Description

Returns the value of a specific configuration choice key for the nodegroup that the current virtual node belongs to.

Input

KEY: Configuration choice key for which to return the value. This is application-specific.

Output

Same as CLUSTER_CONFIG_CHOICE(). See Get Cluster Configuration Choice.

Usage Example

appconfig autogen --pattern @@@@MYPATTERN@@@@ --macro NODEGROUP_CONFIG_CHOICE <key>

Get Specific Metadata Choice Key

Name

NODEGROUP_CONFIG_METADATA

Description

Returns the value of a specific configuration metadata key for the nodegroup that the current virtual node belongs to.

Input

KEY: Configuration choice key for which to return the value. This is application-specific

Output

Same as CLUSTER_CONFIG_METADATA(). See Get Cluster Configuration Metadata.

Usage Example

appconfig autogen --pattern @@@@MYPATTERN@@@@ --macro NODEGROUP_CONFIG_METADATA <key>

Get a Unique Integer for This Host

Name

UNIQUE_SELF_NODE_INT

Description

Returns an integer for the current host. This number is unique across the nodegroup and is guaranteed to be between 1 and the number of hosts in the nodegroup. This function returns the same unique integer on a given virtual node across multiple invocations.

Input

nothing

Output

Success: An integer is echoed as an output.
Failure: Nothing is echoed from the function.

Usage Example

appconfig autogen --pattern @@@@MYPATTERN@@@@ --macro UNIQUE_SELF_NODE_INT

Get a Unique Integer for Another Host

Name

UNIQUE_ANOTHER_NODE_INT

Description

Returns an integer for a specified remote host in the current nodegroup. This number is unique across the nodegroup and is guaranteed to be between 1 and the number of hosts in the nodegroup. This function returns the same unique integer on a given virtual node across multiple invocations.

Input

Hostname of the remote host.

Output

Success: An integer is echoed as an output.
Failure: Nothing is echoed from the function.

Usage Example

appconfig autogen --pattern @@@@MYPATTERN@@@@ --macro UNIQUE_ANOTHER_NODE_INT

Get Unique Integer by Service

Name

UNIQUE_INT_ID_BY_SRVC

Description

Returns a unique integer for a Catalog service based on an unspecified criterion. The integer generated is guaranteed to be between 1 and the number of hosts in the nodegroup that run the specified service. This function returns the same unique integer on a given virtual node across multiple invocations.

Input

SRVCID: ID number of the Catalog service.

Output

Success: An integer is echoed as an output.
Failure: Nothing is echoed from the function.

Usage Example

appconfig autogen --pattern @@@@MYPATTERN@@@@ --macro UNIQUE_INT_ID_BY_SRVC

Get Total Available VRAM

Name

GET_TOTAL_VMEMORY_MB

Description

Get the total amount of available virtual memory, in MB

Input

nothing

Output

Success: Total amount of available VRAM, in MB.
Failure: Nothing; exits when the return status of bd_vcli is non-zero.

Usage Example

appconfig autogen --pattern @@@@MYPATTERN@@@@ --macro GET_TOTAL_VMEMORY_MB

Get Total Available VCPU Cores

Name

GET_TOTAL_VCORES

Description

Get the total number of virtual CPU cores available for Spark.

Input

nothing

Output

Success: Total number of virtual CPU cores available to Spark.
Failure: Nothing; exits when the return status of bd_vcli is non-zero.

Usage Example

appconfig autogen --pattern @@@@MYPATTERN@@@@ --macro GET_TOTAL_VCORES

Get FQDN List

Name

GET_FQDN_LIST

Description

Get a list of Fully Qualified Domain Name(s) (FQDN) for the virtual node(s) with the specified role (e.g. Master, Worker, Edge).

Input

Role (such as master, worker, etc.)

Output

Success: FQDNs of the virtual node(s) in the cluster that have the specified role.
Failure: Nothing; exits when the return status of bd_vcli is non-zero.

Usage Example

appconfig autogen --pattern @@@@MYPATTERN@@@@ --macro GET_FQDN_LIST \$ROLE <role>

Get IP Address List

Name

GET_IPADDR_LIST

Description

Get a list of IP addresses for the virtual node(s) with the specified role (e.g. Master, Worker, Edge).

Input

Role (such as master, worker, etc.)

Output

Success: IP addresses of the virtual node(s) in the cluster that have the specified role.
Failure: Nothing; exits when the return status of bd_vcli is non-zero.

Usage Example

appconfig autogen --pattern @@@@MYPATTERN@@@@ --macro GET_IPADDR_LIST \$ROLE <role>

Get Virtual Node FQDN

Name

GET_NODE_FQDN

Description

Get the Fully Qualified Domain Name (FQDN) for the current virtual node.

Input

nothing

Output

Success: FQDN of the current node.
Failure: Exits when the return status of bd_vcli is non-zero.

Usage Example

appconfig autogen --pattern @@@@MYPATTERN@@@@ --macro GET_NODE_FQDN

Get Virtual Node IP Address

Name

GET_NODE_IPADDR

Description

Get the IP address for the current virtual node.

Input

nothing

Output

Success: IP address of the current virtual node.
Failure: Returns a non-zero status.

Usage Example

appconfig autogen --pattern @@@@MYPATTERN@@@@ --macro GET_NODE_IPADDR

Generate Application-Specific URL

Name

GET_SERVICE_URL

Description

Generate a URL for a specific application.

Inputs

SRVC_ID: ID number of the service for which to generate the URL.
ROLE: Virtual node role on which the specified service runs (such as master or worker).
NODEGRP: ID number of the nodegroup to which this URL applies. This disambiguates situations when the specified service is running in multiple nodegroups. If this argument is not specified and the same service name exists in multiple nodegroups, then the first nodegroup ID will be automatically used for this disambiguation.

Output

Success: The application URL.
Failure: Nothing; exits when the return status of bd_vcli is non-zero

Usage Example

appconfig autogen --pattern @@@@MYPATTERN@@@@ --macro GET_SERVICE_URL \$SRV_ID <service_id> \$ROLE <role> \$NODEGRP <nodegroup_id>

Get Tenant Information

Name

TENANT_INFO

Description

Returns the value of the specified tenant namespace key.

Input

KEY: Such as aws_access_key or aws_secret_key. See the list of valid tenant namespace keys in Application Configuration API.

Output

Success: Value of the specified tenant namespace key.
Failure: Nothing; exits when the return status of bd_vcli is non-zero.

Usage Example

appconfig autogen --pattern @@@MYPATTERN@@@ --macro TENANT_INFO <key>