Installing HPE Data Fabric Streams Python Client

The HPE Data Fabric Streams Python Client is a binding for librdkafka that is dependent on the HPE Data Fabric Streams C client (HPE Data Fabric Streams C Client is a distribution of librdkafka that works with HPE Data Fabric Streams).

Prerequisites

NOTE
As of release 5.2.1, you can create Python client applications for HPE Data Fabric Streams.
Verify that the following components are installed on the node:
  • HPE Data Fabric Streams C Client (mapr-librdkafka)
  • GNU Compiler Collection (GCC) is installed on the node.
  • Valid Python version
    • For a cluster running Release 7.9 or a lower version core, Python version 2.7.1 and above, up to Python version 3.6.x.
    • For a cluster running Release 7.10 or higher, Python version 3.7 up to Python version 3.12.
  • Python pip
  • python-devel (This is required for nodes with the Linux operating system.)
NOTE
For instructions on setting up the EEP repository, see Step 11: Install Ecosystem Components Manually.
IMPORTANT
Because the HPE Data Fabric Streams Python Client is dependent on the HPE Data Fabric Streams C Client, you must configure the HPE Data Fabric Streams C Client before using the HPE Data Fabric Streams Python Client. See Configuring the HPE Data Fabric Streams C Client.

Installation

NOTE
The Python client is available for Linux or Mac operating systems.
Set the following environment variables:
export LIBRARY_PATH=/opt/mapr/lib/
export C_INCLUDE_PATH=/opt/mapr/include/
export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/opt/mapr/lib
To install the HPE Data Fabric Streams Python Client using the Python Software Foundation, run the following command as root or using sudo:
  • On Linux: 
    pip install --global-option=build_ext --global-option="--library-dirs=/opt/mapr/lib" --global-option="--include-dirs=/opt/mapr/include/" mapr-streams-python
  • On Mac:
    pip install --user --global-option=build_ext --global-option="--library-dirs=/opt/mapr/lib" --global-option="--include-dirs=/opt/mapr/include/" mapr-streams-python
NOTE
The referenced package works on nodes with the Linux or the Mac operating system. The Python Client for HPE Data Fabric Streams is not supported on Windows.
Alternatively, you can install the HPE Data Fabric Streams Python Client via the HPE Data Fabric package repository: 
https://package.ezmeral.hpe.com/releases/MEP/<MEP version>/mac/mapr-streams-python-<version>.tar.gz

Troubleshooting Mac OS Installation

If you install the HPE Data Fabric Streams Python Client on a Mac without the --user flag, you may encounter a "Not Permitted" error, signifying that you don't have permission to create a LICENSE file. The error will look similar to the following:
Copying LICENSE -> /System/Library/Frameworks/Python.framework/Versions/2.7/
                error: [Error 1] Operation not permitted:
                '/System/Library/Frameworks/Python.framework/Versions/2.7/LICENSE'
To fix this issue, execute the following steps. These steps apply to users with a new Mac OS Sierra version 10.12.5.
  1. Reboot your Mac while simultaneously holding the Command and R keys to go into Mac OS X Recovery mode.
  2. Select Utilities, and then Terminal.
  3. Type csrutil disable. A message will pop up, informing you that your System Integrity Protection (SIP) has been sucessfully disabled.
  4. Reboot your Mac again.
  5. Go into the Terminal and execute the following command as the root user:
    pip install --global-option=build_ext  --global-option="--library-dirs=/opt/mapr/lib" --global-option="--include-dirs=/opt/mapr/include/" mapr-streams-python
  6. Once the script runs successfully, reboot your Mac while holding the Command and R keys. You will go into Mac OS X Recovery mode.
  7. Select Utilities, and then Terminal.
  8. Type csruntil enable to enable your SIP.
NOTE
To avoid this error, consider using a virtual Python environment or the --user flag.