Using pull through cache rules - Amazon ECR
Services or capabilities described in Amazon Web Services documentation might vary by Region. To see the differences applicable to the China Regions, see Getting Started with Amazon Web Services in China (PDF).

Using pull through cache rules

With pull through cache rules, you can sync the contents of an upstream registry with your Amazon ECR private registry. Amazon ECR currently supports creating pull through cache rules for the following upstream registries.

  • Docker Hub, Microsoft Azure Container Registry, Google Artifact Registry, and GitHub Container Registry (Requires authentication)

  • Amazon ECR Public, the Kubernetes container image registry, and Quay (Doesn't require authentication)

For the upstream registries that require authentication, you must store your credentials in an Amazon Secrets Manager secret. The Amazon ECR console makes it easy for you to create the Secrets Manager secret for each of the authenticated upstream registries. For more information on creating a Secrets Manager secret using the Secrets Manager console, see Storing your upstream repository credentials in an Amazon Secrets Manager secret.

After you've created a pull through cache rule for the upstream registry, simply pull an image from that upstream registry using your Amazon ECR private registry URI. Amazon ECR then creates a repository and caches that image in your private registry. On your subsequent pull requests of the cached image with a given tag, Amazon ECR checks the upstream registry to see if there is a new version of the image with that specific tag and attempts to update the image in your private registry at least once every 24 hours.

Amazon ECR has added support for repository creation templates, currently in preview, which gives you the control to specify initial configurations for new repositories created by Amazon ECR on your behalf using pull through cache rules. Each template contains a repository namespace prefix which is used to match new repositories to a specific template. Templates can specify the configuration for all repository settings including resource-based access policies, tag immutability, encryption, and lifecycle policies. The settings in a repository creation template are only applied during repository creation and don't have any effect on existing repositories or repositories created using any other method. For more information, see Manage your repository creation templates.

Considerations for using pull through cache

The following should be considered when using Amazon ECR pull through cache rules.

  • Creating pull through cache rules isn't supported in the following Regions.

    • China (Beijing) (cn-north-1)

    • China (Ningxia) (cn-northwest-1)

    • Amazon GovCloud (US-East) (us-gov-east-1)

    • Amazon GovCloud (US-West) (us-gov-west-1)

  • Amazon Lambda doesn't support pulling container images from Amazon ECR using a pull through cache rule.

  • When pulling images using pull through cache, the Amazon ECR FIPS service endpoints aren't supported the first time an image is pulled. Using the Amazon ECR FIPS service endpoints work on subsequent pulls though.

  • When a cached image is pulled through the Amazon ECR private registry URI, the image pulls are initiated by Amazon IP addresses. This ensures that the image pull doesn't count against any pull rate quotas implemented by the upstream registry.

  • When a cached image is pulled through the Amazon ECR private registry URI, Amazon ECR checks the upstream repository at least once every 24 hours to verify whether the cached image is the latest version. If there is a newer image in the upstream registry, Amazon ECR attempts to update the cached image. This timer is based off the last pull of the cached image.

  • If Amazon ECR is unable to update the image from the upstream registry for any reason and the image is pulled, the last cached image will still be pulled.

  • When creating the Secrets Manager secret that contains the upstream registry credentials, the secret name must use the ecr-pullthroughcache/ prefix. The secret must also be in the same account and Region that the pull through cache rule is created in.

  • When a multi-architecture image is pulled using a pull through cache rule, the manifest list and each image referenced in the manifest list are pulled to the Amazon ECR repository. If you only want to pull a specific architecture, you can pull the image using the image digest or tag associated with the architecture rather than the tag associated with the manifest list.

  • Amazon ECR uses a service-linked IAM role, which provides the permissions needed for Amazon ECR to create the repository, retrieve the Secrets Manager secret value for authentication, and push the cached image on your behalf. The service-linked IAM role is created automatically when a pull through cache rule is created. For more information, see Amazon ECR service-linked role for pull through cache.

  • By default, the IAM principal pulling the cached image has the permissions granted to them through their IAM policy. You may use the Amazon ECR private registry permissions policy to further scope the permissions of an IAM entity. For more information, see Using registry permissions.

  • Amazon ECR repositories created using the pull through cache workflow are treated like any other Amazon ECR repository. All repository features, such as replication and image scanning are supported.

  • When Amazon ECR creates a new repository on your behalf using a pull through cache action, the following default settings are applied to the repository unless there is a matching repository creation template. You can use a repository creation template to define the settings applied to repositories created by Amazon ECR on your behalf. For more information, see Manage your repository creation templates.

    • Tag immutability — Turned off, tags are mutable and can be overwritten.

    • Encryption — The default AES256 encryption is used.

    • Repository permissions — Omitted, no repository permissions policy is applied.

    • Lifecycle policy — Omitted, no lifecycle policy is applied.

    • Resource tags — Omitted, no resource tags are applied.

  • Turning on image tag immutability for repositories using a pull through cache rule will prevent Amazon ECR from updating images using the same tag.

  • When an image is pulled using the pull through cache rule for the first time a route to the internet may be required. There are certain circumstances in which a route to the internet is required so it's best to set up a route to avoid any failures. Thus, if you've configured Amazon ECR to use an interface VPC endpoint using Amazon PrivateLink then you need to ensure the first pull has a route to the internet. One way to do this is to create a public subnet in the same VPC, with an internet gateway, and then route all outbound traffic to the internet from their private subnet to the public subnet. Subsequent image pulls using the pull through cache rule don't require this. For more information, see Example routing options in the Amazon Virtual Private Cloud User Guide.

Required IAM permissions

In addition to the Amazon ECR API permissions needed to authenticate to a private registry and to push and pull images, the following additional permissions are needed to use pull through cache rules effectively.

  • ecr:CreatePullThroughCacheRule – Grants permission to create a pull through cache rule. This permission must be granted via an identity-based IAM policy.

  • ecr:BatchImportUpstreamImage – Grants permission to retrieve the external image and import it to your private registry. This permission can be granted by using the private registry permissions policy, an identity-based IAM policy, or by using the resource-based repository permissions policy. For more information about using repository permissions, see Private repository policies.

  • ecr:CreateRepository – Grants permission to create a repository in a private registry. This permission is required if the repository storing the cached images doesn't already exist. This permission can be granted by either an identity-based IAM policy or the private registry permissions policy.

  • ecr:TagResource – Grants permission to add metadata tags to an Amazon ECR resource. This permission is only required if you are pulling an image that uses a pull through cache rule that has an associated repository creation template that is configured to add resource tags to the repository. This permission must be granted via an identity-based IAM policy.

Using registry permissions

Amazon ECR private registry permissions may be used to scope the permissions of individual IAM entities to use pull through cache. If an IAM entity has more permissions granted by an IAM policy than the registry permissions policy is granting, the IAM policy takes precedence. For example, if user has ecr:* permissions granted, no additional permissions are needed at the registry level.

  1. Open the Amazon ECR console at https://console.amazonaws.cn/ecr/.

  2. From the navigation bar, choose the Region to configure your private registry permissions statement in.

  3. In the navigation pane, choose Private registry, Registry permissions.

  4. On the Registry permissions page, choose Generate statement.

  5. For each pull through cache permissions policy statement you want to create, do the following.

    1. For Policy type, choose Pull through cache policy.

    2. For Statement id, provide a name for the pull through cache statement policy.

    3. For IAM entities, specify the users, groups, or roles to include in the policy.

    4. For Repository namespace, select the pull through cache rule to associate the policy with.

    5. For Repository names, specify the repository base name to apply the rule for. For example, if you want to specify the Amazon Linux repository on Amazon ECR Public, the repository name would be amazonlinux.

Use the following Amazon CLI command to specify the private registry permissions using the Amazon CLI.

  1. Create a local file named ptc-registry-policy.json with the contents of your registry policy. The following example grants the ecr-pull-through-cache-user permission to create a repository and pull an image from Amazon ECR Public, which is the upstream source associated with the previously created pull through cache rule.

    { "Sid": "PullThroughCacheFromReadOnlyRole", "Effect": "Allow", "Principal": { "AWS": "arn:aws:iam::111122223333:user/ecr-pull-through-cache-user" }, "Action": [ "ecr:CreateRepository", "ecr:BatchImportUpstreamImage" ], "Resource": "arn:aws:ecr:us-east-1:111122223333:repository/ecr-public/*" }
    Important

    The ecr-CreateRepository permission is only required if the repository storing the cached images doesn't already exist. For example, if the repository creation action and the image pull actions are being done by separate IAM principals such as an administrator and a developer.

  2. Use the put-registry-policy command to set the registry policy.

    aws ecr put-registry-policy \ --policy-text file://ptc-registry.policy.json

Next steps

Once you are ready to start using pull through cache rules, the following are the next steps.