mapr-support-collect.sh

Collects information about a cluster's recent activity, to help Data Fabric Support diagnose problems.

The "mini-dump" option limits the size of the support output. When the -m or --mini-dump option is specified along with a size, mapr-support-collect.sh collects only a head and tail, each limited to the specified size, from any log file that is larger than twice the specified size. The total size of the output is therefore limited to approximately 2 * size * number of logs. The size can be specified in bytes, or using the following suffixes:

  • b - bytes
  • k - kilobytes (1024 bytes)
  • m - megabytes (1024 kilobytes)

Syntax

 /opt/mapr/support/tools/mapr-support-collect.sh
       -h, --hosts HOST_FILE
          hosts file, each line has entries [user@]host[:port]
       -H, --host HOST_ENTRY
          additional host entry of the form [user@]host[:port], multiple can be specified
       -Q, --no-cldb
          do not query CLDB for list of nodes
       -n, --name NAME
          name of output file
       -d, --output-dir DIR_PATH
          absolute path of output directory
       -l, --no-logs
          do not include log files
       --no-hadoop-logs
          do not include hadoop log files
       --hbase-logs
          include hbase log files
       --sqoop-logs
          include sqoop log files
       --eco-logs
          include all ecosystem log files
       --oozie-logs
          include oozie log files
       --spark-logs
          include spark log files
       --pig-logs
          include pig log files
       --impala-logs
          include impala log files
       --hue-logs
          include hue log files
       --hive-logs
          include hive log files
       --flume-logs
          include flume log files
       --drill-logs
          include drill log files
       --no-kibana-logs
          do not include kibana log files
       --no-grafana-logs
          do not include grafana log files
       --no-elasticsearch-logs
          do not include elasticsearch log files
       --no-opentsdb-logs
          do not include opentsdb log files
       --no-collectd-logs
          do not include collectd log files
       --no-fluentd-logs
          do not include fluentd log files
       --no-vol-info
          do not collect volume information
       -L, --libraries
          include libraries
       -s, --no-statistics
          do not include statistics
       -c, --no-conf
          do not include configurations
       -i, --no-sysinfo
          do not include system information
       -x, --exclude-cluster
          do not collect cluster diagnostics
       -u, --user USER
          username for ssh connections
       -K, --strict-hostkey
          check for strict host key in ssh connection
       -p, --par PAR
          maximum number of nodes from which support dumps will be gathered concurrently (default: 10)
       -t, --dump-timeout DUMPTIMEOUT
          timeout for execution of mapr-support-dump command on a node (default: 3600 seconds, 0 = no limit)
       -T, --scp-timeout SCPTIMEOUT
          timeout for copy of support dump output from a remote node to local filesystem (default: no limit)
       -y, --yes
          do not require acknowledgement of the number of nodes that will be affected
       -O, --online
          Leverage MapR APIs to gather support dumps. When not specified, SSH and SCP will be used.
       -S, --scp-port SCPPORT
          the local port to which remote nodes will establish an SCP session
       --collect-cores
          Collect cores of running mfs processes from all nodes (off by default)
       --move-cores
          Move mfs and nfs cores from coresDir from all nodes (off by default)
       --use-hostname
          Use hostname to ssh instead of IP addresses (off by default)
       --cldb CLDBNODE
          Use this option when the CLDB Service is down to point to a CLDB node
       --port PORT
          port number used by FileServer (default: 5660)
       --nfsport NFS_MGMT_PORT
          port number used by NFSServer (default: 9998)
       -m, --mini-dump SIZE
          Collects only first and last number of bytes of each log file if file is greater then 2*SIZE. 
          SIZE may have a multiplier suffix: b 512, k 1024, m 1024*1024
       -A, --logs-age DAYS
          Use this option to collect logs newer than specified DAYS
          (default: 7, nolimit: 0)
       -f, --filter FILTER_STRING
          Use this option to filter nodes for which support dump should be collected
       -?, --help
          display usage

Parameters

Parameter

Description

-h or --hosts

A file containing a list of hosts. Each line contains one host entry, in the format [user@]host[:port].

-H or --host

One or more hosts in the format [user@]host[:port].

-Q or --no-cldb

If specified, the command does not query the CLDB for list of nodes.

-n or --name

Specifies the name of the output file. If not specified, the default is a date-named file in the format YYYY-MM-DD-hh-mm-ss.tar.

-d or --output-dir

The absolute path to the output directory. The default path is /opt/mapr/support/collect/.

-l or --no-logs

If specified, the command output does not include any log files.

--no-hadoop-logs

If specified, the command output does not include Hadoop log files.

--hbase-logs

If specified, the command output includes HBase log files.
--sqoop-logs If specified, the command output includes Sqoop log files.
--eco-logs If specified, the command output includes the log files for all MapR ecosystem components.
--oozie-logs If specified, the command output includes Oozie log files.
--spark-logs If specified, the command output includes Spark log files.
--pig-logs If specified, the command output includes Apache Pig log files.
--impala-logs If specified, the command output includes Apache Impala log files.
--hue-logs If specified, the command output includes Apache Hue log files.
--hive-logs If specified, the command output includes Apache Hive log files.
--flume-logs If specified, the command output includes Apache Flume log files.
--drill-logs If specified, the command output includes Apache Drill log files.
--no-kibana-logs If specified, the command output does not include Kibana log files.
--no-grafana-logs If specified, the command output does not include Grafana log files.
--no-elasticsearch-logs If specified, the command output does not include Elasticsearch log files.
--no-opentsdb-logs If specified, the command output does not include OpenTSDB log files.
--no-collectd-logs If specified, the command output does not include Collectd log files.
--no-fluentd-logs If specified, the command output does not include Fluentd log files.
--no-vol-info If specified, the command output does not include any volume information.

-L or --no-libraries

If specified, the command output does not include libraries.

-c or --no-conf

If specified, the command output does not include configurations.

-i or --no-sysinfo

If specified, the command output does not include system information.

-x or --exclude-cluster

If specified, the command does not collect cluster diagnostics.

Even if cluster diagnostics are excluded, the script still collects local logs and local system diagnostic information.

-u or --user

The username for ssh connections.

-K or --strict-hostkey

If specified, checks for strict host key in the SSH connection. When specified, ssh never automatically adds host keys to the ~/.ssh/known_hosts file, and refuses to connect to a host whose host key has changed. This provides maximum protection against trojan horse attacks, but can be troublesome when the /etc/ssh/ssh_known_hosts file is poorly maintained or connections to new hosts are frequently made. This option forces the user to manually add all new hosts.

-p or --par

The maximum number of nodes from which support dumps are gathered concurrently (default: 10).

-t or --dump-timeout

The timeout in seconds for execution of the mapr-support-dump command on a node (default: 3600 seconds or 0 = no limit).

-T or --scp-timeout

The timeout in seconds for copy of support dump output from a remote node to the local filesystem (default: no limit).

-y or --yes

If specified, the command does not require acknowledgement of the number of nodes that are affected.

-O or --online

Specifies a space-separated list of nodes from which to gather support output, and uses the MapR APIs instead of ssh for transmitting the support data.

-S or --scp-port

The local port to which remote nodes establish a SCP session. The default port is 22.

--collect-cores

If specified, the command collects cores of running MFS processes from all nodes (default: off).

--move-cores

If specified, the command moves MFS and NFS cores from /opt/cores to /opt/mapr/logs/cores/ on all nodes (default: off).

--use-hostname

If specified, uses hostnames instead of IP address for SSH (default: off).

--cldb <cldbnode>

Use this option when the CLDB Service is down to point to a CLDB node.

--port

The port number used by FileServer (default: 5660).
--nfsport The port number used by NFSServer (default: 9998).

-m or --mini-dump <size>

For any log file greater than 2 * <size>, collects only a head and tail each of the specified size. The <size> may have a suffix specifying units:

  • b - blocks (512 bytes)
  • k - kilobytes (1024 bytes)
  • m - megabytes (1024 kilobytes)
-A or --logs-ag

Use this option to collect logs newer than specified days (default: 7, nolimit: 0)

-f or --filter <filter string>

Use this option to specify a filter string. Support information is only collected for nodes with names that match the filter string.

-? or --help

Displays usage help text

Examples

Collect support information and dump it to the file /opt/mapr/support/collect/mysupport-output.tar:
/opt/mapr/support/tools/mapr-support-collect.sh -n mysupport-output
2019-09-16 21:37:28.884 INFO Creating nodes file
2019-09-16 21:37:28.907 INFO Querying CLDB for nodes in the cluster
2019-09-16 21:37:31.883 INFO Created nodes file
Diagnostics will be collected from 1 nodes. Press enter to continue: 
2019-09-16 21:37:35.650 INFO Collecting cluster information
2019-09-16 21:38:35.282 INFO Collecting dump on <ip>
Password: 
2019-09-16 21:41:12.917 INFO Copying dump from <ip>
Password: 
2019-09-16 21:44:25.137 INFO Making a tarball of all the dumps

------------------------------------------------------------------------------------
------------------ Finished collecting diagnostics information ---------------------
------------------------------------------------------------------------------------
Successfully collected support information on cluster from CLDB.

Total no. of nodes from which dump collection was attempted: 1
Nodes from which support information gathering succeeded:    1
Number of nodes from which dump collection failed:           0
Number of nodes from which dump file could not be copied:    0

The tar ball of the dumps is available at: /opt/mapr/support/collect/mysupport-output.tar