Type Mapping Between HPE Ezmeral Data Fabric Database JSON and DataFrames
This table maps data types between HPE Ezmeral Data Fabric Database JSON OJAI and Apache Spark DataFrame.
NOTE
Not all DataFrame data types are supported by HPE Ezmeral Data Fabric Database, for a list of supported data types, see JSON Documents.OJAI Data Type | DataFrame Data Type |
---|---|
Boolean | BooleanType |
String | StringType |
Byte | ByteType |
Short | ShortType |
Int | IntegerType |
Long | LongType |
Float | FloatType |
Double | DoubleType |
Decimal | DecimalType |
Date | DateType |
Time | TimestampType |
TimeStamp | TimeStampType |
Interval | CalendarIntervalType |
Binary | BinaryType |
Map | StructType |
Array | ArrayType |
NOTE
The OJAI Time
data type is converted to a Spark
TimestampType
with the date set to the epoch date. Spark SQL does not
support a TIME
type.