Serialization Formats in Neptune Streams - Amazon Neptune
Services or capabilities described in Amazon Web Services documentation might vary by Region. To see the differences applicable to the China Regions, see Getting Started with Amazon Web Services in China.

Serialization Formats in Neptune Streams

Amazon Neptune uses two different formats for serializing graph-changes data to log streams, depending on whether the graph was created using Gremlin or SPARQL.

Both formats share a common record serialization format, as described in Neptune Streams API Response Format, that contains the following fields:

  • commitTimestamp   –   The time at which the commit for the transaction was requested, in milliseconds from the Unix epoch.

  • eventId   –   The sequence identifier of the stream change record.

  • data   –   The serialized Gremlin, SPARQL, or OpenCypher change record. The serialization formats of each record are described in more detail in the next sections.

  • op   –   The operation that created the change.

PG_JSON Change Serialization Format

Note

As of engine release 1.1.0.0, the Gremlin stream output format (GREMLIN_JSON) output by the Gremlin stream endpoint (https://Neptune-DNS:8182/gremlin/stream) is being deprecated. It is replaced by PG_JSON, which is currently identical to GREMLIN_JSON.

A Gremlin or openCypher change record, contained in the data field of a log stream response, has the following fields:

  • id – String, required.

    The ID of the Gremlin or openCypher element.

  • type – String, required.

    The type of this Gremlin or openCypher element. Must be one of the following:

    • vl – Vertex label for Gremlin; node label for openCypher.

    • vp – Vertex properties for Gremlin; node properties for openCypher.

    • e – Edge and edge label for Gremlin; relationship and relationship type for openCypher.

    • ep – Edge properties for Gremlin; relationship properties for openCypher.

  • key – String, required.

    The property name. For element labels, this is "label".

  • value – value object, required.

    This is a JSON object that contains a value field for the value itself, and a datatype field for the JSON data type of that value.

    "value": { "value": "the new value", "dataType": "the JSON datatype of the new value" }
  • from – String, optional.

    If this is an edge (type="e"), the ID of the corresponding from vertex or source node.

  • to – String, optional.

    If this is an edge (type="e"), the ID of the corresponding to vertex or target node.

Gremlin Examples

  • The following is an example of a Gremlin vertex label.

    { "id": "an ID string", "type": "vl", "key": "label", "value": { "value": "the new value of the vertex label", "dataType": "String" } }
  • The following is an example of a Gremlin vertex property.

    { "id": "an ID string", "type": "vp", "key": "the property name", "value": { "value": "the new value of the vertex property", "dataType": "the datatype of the vertex property" } }
  • The following is an example of a Gremlin edge.

    { "id": "an ID string", "type": "e", "key": "label", "value": { "value": "the new value of the edge", "dataType": "String" }, "from": "the ID of the corresponding "from" vertex", "to": "the ID of the corresponding "to" vertex" }

openCypher Examples

  • The following is an example of an openCypher node label.

    { "id": "an ID string", "type": "vl", "key": "label", "value": { "value": "the new value of the node label", "dataType": "String" } }
  • The following is an example of an openCypher node property.

    { "id": "an ID string", "type": "vp", "key": "the property name", "value": { "value": "the new value of the node property", "dataType": "the datatype of the node property" } }
  • The following is an example of an openCypher relationship.

    { "id": "an ID string", "type": "e", "key": "label", "value": { "value": "the new value of the relationship", "dataType": "String" }, "from": "the ID of the corresponding source node", "to": "the ID of the corresponding target node" }

SPARQL NQUADS Change Serialization Format

Neptune logs changes to SPARQL quads in the graph using the Resource Description Framework (RDF) N-QUADS language defined in the W3C RDF 1.1 N-Quads specification.

The data field in the change record simply contains a stmt field that holds an N-QUADS statement expressing the changed quad, as in the following example.

"stmt" : "<https://test.com/s> <https://test.com/p> <https://test.com/o> .\n"