Configuring the HPE Ezmeral Data Fabric Database OJAI Connector for Apache Spark
Before using the HPE Ezmeral Data Fabric Database OJAI Connector for Apache
Spark, you must edit the pom.xml
file for your project.
Add the Spark core dependency into the
pom.xml
file:NOTE
If all
dependent JAR files are already present on the node, consider setting the
scope
parameter to provided
. For example:
<scope>provided</scope>
Setting the scope this way reduces the
size of the JAR file.<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-core_<scala_version></artifactId>
<version><spark_artifact_version></version>
</dependency>
Add the Spark Maven dependency to the pom.xml
file:
<dependency>
<groupId>com.mapr.db</groupId>
<artifactId>maprdb-spark</artifactId>
<version><spark_artifact_version></version>
</dependency>
For example, see the dependencies for Spark 2.4.4.0 (EEP 6.3.0 release):
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-core_2.11</artifactId>
<version>2.4.4.0-mapr-630</version>
</dependency>
<dependency>
<groupId>com.mapr.db</groupId>
<artifactId>maprdb-spark</artifactId>
<version>2.4.4.0-mapr-630</version>
</dependency>
To enable Maven to download dependencies, add the following repository information to the
pom.xml
file:
<repository>
<id>mapr-releases</id>
<url>https://repository.mapr.com/maven/</url>
<snapshots>
<enabled>false</enabled>
</snapshots>
<releases>
<enabled>true</enabled>
</releases>
</repository>