Services or capabilities described in Amazon Web Services documentation might vary by Region. To see the differences applicable to the China Regions, see Getting Started with Amazon Web Services in China (PDF).

Amazon Managed Service for Apache Flink was previously known as Amazon Kinesis Data Analytics for Apache Flink.

Managing application backups using snapshots

A snapshot is the Managed Service for Apache Flink implementation of an Apache Flink Savepoint. A snapshot is a user- or service-triggered, created, and managed backup of the application state. For information about Apache Flink Savepoints, see Savepoints in the Apache Flink Documentation. Using snapshots, you can restart an application from a particular snapshot of the application state.

In Managed Service for Apache Flink, you manage snapshots using the following API actions:

For the per-application limit on the number of snapshots, see Quota. If your application reaches the limit on snapshots, then manually creating a snapshot fails with a LimitExceededException.

Managed Service for Apache Flink never deletes snapshots. You must manually delete your snapshots using the DeleteApplicationSnapshot action.

To load a saved snapshot of application state when starting an application, use the ApplicationRestoreConfiguration parameter of the StartApplication or UpdateApplication action.

Automatic snapshot creation

If SnapshotsEnabled is set to true in the ApplicationSnapshotConfiguration for the application, Managed Service for Apache Flink automatically creates and uses snapshots when the application is updated, scaled, or stopped to provide exactly-once processing semantics.

Automatically created snapshots have the following qualities:

Restoring from a snapshot that contains incompatible state data

Because snapshots contain information about operators, restoring state data from a snapshot for an operator that has changed since the previous application version may have unexpected results. An application will fault if it attempts to restore state data from a snapshot that does not correspond to the current operator. The faulted application will be stuck in either the STOPPING or UPDATING state.

To allow an application to restore from a snapshot that contains incompatible state data, set the AllowNonRestoredState parameter of the FlinkRunConfiguration to true using the UpdateApplication action.

You will see the following behavior when an application is restored from an obsolete snapshot:

If you need to resume an application that is incompatible with existing savepoint data, we recommend that you skip restoring from the snapshot by setting the ApplicationRestoreType parameter of the StartApplication action to SKIP_RESTORE_FROM_SNAPSHOT.

For more information about how Apache Flink deals with incompatible state data, see State Schema Evolution in the Apache Flink Documentation.

Snapshot API examples

This section includes example requests for API actions for using snapshots with an application. For information about how to use a JSON file for input for an API action, see Managed Service for Apache Flink API example code.

Enable snapshots for an application

The following example request for the UpdateApplication action enables snapshots for an application:

{
   "ApplicationName": "MyApplication",
   "CurrentApplicationVersionId": 1,
   "ApplicationConfigurationUpdate": { 
      "ApplicationSnapshotConfigurationUpdate": { 
         "SnapshotsEnabledUpdate": "true"
       }
    }
}

Create a snapshot

The following example request for the CreateApplicationSnapshot action creates a snapshot of the current application state:

{
   "ApplicationName": "MyApplication",
   "SnapshotName": "MyCustomSnapshot"
}

List snapshots for an application

The following example request for the ListApplicationSnapshots action lists the first 50 snapshots for the current application state:

{
   "ApplicationName": "MyApplication",
   "Limit": 50
}

List details for an application snapshot

The following example request for the DescribeApplicationSnapshot action lists details for a specific application snapshot:

{
   "ApplicationName": "MyApplication",
   "SnapshotName": "MyCustomSnapshot"
}

Delete a snapshot

The following example request for the DeleteApplicationSnapshot action deletes a previously saved snapshot. You can get the SnapshotCreationTimestamp value using either ListApplicationSnapshots or DeleteApplicationSnapshot:

{
   "ApplicationName": "MyApplication",
   "SnapshotName": "MyCustomSnapshot",
   "SnapshotCreationTimestamp": 12345678901.0,
}

Restart an application using a named snapshot

The following example request for the StartApplication action starts the application using the saved state from a specific snapshot:

{
   "ApplicationName": "MyApplication",
   "RunConfiguration": { 
      "ApplicationRestoreConfiguration": { 
         "ApplicationRestoreType": "RESTORE_FROM_CUSTOM_SNAPSHOT",
         "SnapshotName": "MyCustomSnapshot"
      }
   }
}

Restart an application using the most recent snapshot

The following example request for the StartApplication action starts the application using the most recent snapshot:

{
   "ApplicationName": "MyApplication",
   "RunConfiguration": { 
      "ApplicationRestoreConfiguration": { 
         "ApplicationRestoreType": "RESTORE_FROM_LATEST_SNAPSHOT"
      }
   }
}

Restart an application using no snapshot

The following example request for the StartApplication action starts the application without loading application state, even if a snapshot is present:

{
   "ApplicationName": "MyApplication",
   "RunConfiguration": { 
      "ApplicationRestoreConfiguration": { 
         "ApplicationRestoreType": "SKIP_RESTORE_FROM_SNAPSHOT"
      }
   }
}