Getting started with schema registry
The following sections provide an overview and walk you through setting up and using Schema Registry. For information about schema registry concepts and components, see Amazon Glue Schema registry.
Topics
- Installing SerDe Libraries
- Creating a registry
- Creating a schema
- Updating a schema or registry
- Deleting a schema or registry
- IAM examples for serializers
- IAM examples for deserializers
- Private connectivity using Amazon PrivateLink
- Accessing Amazon CloudWatch metrics
- Sample Amazon CloudFormation template for schema registry
IAM examples for serializers
Note
Amazon managed policies grant necessary permissions for common use cases. For information on using managed policies to manage the schema registry, see Amazon managed (predefined) policies for Amazon Glue.
For serializers, you should create a minimal policy similar to that below to give you the ability to find the schemaVersionId
for a given schema definition. Note, you should have read permissions on the registry in order to read the schemas in the registry. You can limit the registries that can be read by using the Resource
clause.
Code example 13:
{ "Sid" : "GetSchemaByDefinition", "Effect" : "Allow", "Action" : [ "glue:GetSchemaByDefinition" ], "Resource" : ["arn:aws:glue:us-east-2:012345678:registry/registryname-1", "arn:aws:glue:us-east-2:012345678:schema/registryname-1/schemaname-1", "arn:aws:glue:us-east-2:012345678:schema/registryname-1/schemaname-2" ] }
Further, you can also allow producers to create new schemas and versions by including the following extra methods. Note, you should be able to inspect the registry in order to add/remove/evolve the schemas inside it. You can limit the registries that can be inspected by using the Resource
clause.
Code example 14:
{ "Sid" : "RegisterSchemaWithMetadata", "Effect" : "Allow", "Action" : [ "glue:GetSchemaByDefinition", "glue:CreateSchema", "glue:RegisterSchemaVersion", "glue:PutSchemaVersionMetadata", ], "Resource" : ["arn:aws-cn:glue:
aws-region
:123456789012:registry/registryname-1", "arn:aws-cn:glue:aws-region
:123456789012:schema/registryname-1/schemaname-1", "arn:aws-cn:glue:aws-region
:123456789012:schema/registryname-1/schemaname-2" ] }
IAM examples for deserializers
For deserializers (consumer side), you should create a policy similar to that below to allow the deserializer to fetch the schema from the Schema Registry for deserialization. Note, you should be able to inspect the registry in order to fetch the schemas inside it.
Code example 15:
{ "Sid" : "GetSchemaVersion", "Effect" : "Allow", "Action" : [ "glue:GetSchemaVersion" ], "Resource" : ["*"] }
Private connectivity using Amazon PrivateLink
You can use Amazon PrivateLink to connect your data producer’s VPC to Amazon Glue by defining an interface VPC endpoint for Amazon Glue. When you use a VPC interface endpoint, communication between your VPC and Amazon Glue is conducted entirely within the Amazon network. For more information, see Using Amazon Glue with VPC Endpoints.