Using Neptune Streams

• For a SPARQL graph DB:   https://Neptune-DNS:8182/sparql/stream.

• For a Gremlin or openCypher graph DB:   https://Neptune-DNS:8182/propertygraph/stream or https://Neptune-DNS:8182/pg/stream.

Parameters

• limit   –   long, optional. Range: 1–100,000. Default: 10.

  Specifies the maximum number of records to return. There is also a size limit of 10 MB on the response that can't be modified and that takes precedence over the number of records specified in the limit parameter. The response does include a threshold-breaching record if the 10 MB limit was reached.

• iteratorType   –   String, optional.

  This parameter can take one of the following values:

  • AT_SEQUENCE_NUMBER(default)   –   Indicates that reading should start from the event sequence number specified jointly by the commitNum and opNum parameters.

  • AFTER_SEQUENCE_NUMBER   –   Indicates that reading should start right after the event sequence number specified jointly by the commitNum and opNum parameters.

  • TRIM_HORIZON   –   Indicates that reading should start at the last untrimmed record in the system, which is the oldest unexpired (not yet deleted) record in the change-log stream. This mode is useful during application startup, when you don't have a specific starting event sequence number.

  • LATEST   –   Indicates that reading should start at the most recent record in the system, which is the latest unexpired (not yet deleted) record in the change-log stream. This is useful when there is a need to read records from current top of the streams so as not to process older records, such as during disaster recovery or a zero-downtime upgrade. Note that in this mode, there is at most only one record returned.

• commitNum   –   long, required when iteratorType is AT_SEQUENCE_NUMBER or AFTER_SEQUENCE_NUMBER.

  The commit number of the starting record to read from the change-log stream.

  This parameter is ignored when iteratorType is TRIM_HORIZON or LATEST.

• opNum   –   long, optional (the default is 1).

  The operation sequence number within the specified commit to start reading from in the change-log stream data.

• INSERT   –   A Gremlin vertex can have multiple labels, and a Gremlin element can have multiple properties. A separate change record is generated for each label and property when an element is inserted.

• UPDATE   –   When a Gremlin element property is changed, two change records are generated: the first for removing the previous value, and the second for inserting the new value.

• DELETE   –   A separate change record is generated for each element property that is deleted. For example, when a Gremlin edge with properties is deleted, one change record is generated for each of the properties, and after that, one is generated for deletion of the edge label.

  When a Gremlin vertex is deleted, all the incoming and outgoing edge properties are deleted first, then the edge labels, then the vertex properties, and finally the vertex labels. Each of these deletions generates a change record.

• lastEventId   –   Sequence identifier of the last change in the stream response. An event ID is composed of two fields: A commitNum identifies a transaction that changed the graph, and an opNum identifies a specific operation within that transaction. This is shown in the following example.

    "eventId": {
      "commitNum": 12,
      "opNum": 1
    }

• lastTrxTimestamp   –   The time at which the commit for the transaction was requested, in milliseconds from the Unix epoch.

• format   –   Serialization format for the change records being returned. The possible values are PG_JSON for Gremlin or openCypher change records, and NQUADS for SPARQL change records.

• records   –   An array of serialized change-log stream records included in the response. Each record in the records array contains these fields:

  • commitTimestamp   –   The time at which the commit for the transaction was requested, in milliseconds from the Unix epoch.

  • eventId   –   The sequence identifier of the stream change record.

  • data   –   The serialized Gremlin, SPARQL, or OpenCypher change record. The serialization formats of each record are described in more detail in the next section, Serialization Formats in Neptune Streams.

  • op   –   The operation that created the change.

  • isLastOp   –   Only present if this operation is the last one in its transaction. When present, it is set to true. Useful for ensuring that an entire transaction is consumed.

• totalRecords   –   The total number of records in the response.