Getting started with Amazon App Mesh and Kubernetes
Important
End of support notice: On September 30, 2026, Amazon will discontinue support for Amazon App Mesh. After September 30, 2026, you will no longer be able to access the Amazon App Mesh console or Amazon App Mesh resources. For more information, visit this blog post Migrating from Amazon App Mesh to Amazon ECS Service Connect
When you integrate Amazon App Mesh with Kubernetes using the App Mesh controller for Kubernetes, you manage App Mesh resources, such as meshes, virtual services, virtual nodes, virtual routers, and routes through Kubernetes. You also automatically add the App Mesh sidecar container images to Kubernetes pod specifications. This tutorial guides you through the installation of the App Mesh controller for Kubernetes to enable this integration.
The controller is accompanied by the deployment of the following Kubernetes custom resource definitions:
meshes
, virtual services
, virtual nodes
,
and virtual routers
. The controller watches for
creation, modification, and deletion of the custom resources and makes changes to the corresponding App Mesh
Service Meshes, Virtual services, Virtual nodes, Virtual gateways, Gateway routes, Virtual routers
(including Routes) resources through the App Mesh API. To learn more or contribute to the
controller, see the GitHub
project
The controller also installs a webhook that injects the following containers into Kubernetes pods that are labeled with a name that you specify.
-
App Mesh Envoy proxy – Envoy uses the configuration defined in the App Mesh control plane to determine where to send your application traffic.
-
App Mesh proxy route manager – Updates
iptables
rules in a pod's network namespace that route inbound and outbound traffic through Envoy. This container runs as a Kubernetes init container inside of the pod.
Prerequisites
-
An existing understanding of App Mesh concepts. For more information, see What Is Amazon App Mesh?.
-
An existing understanding of Kubernetes concepts. For more information, see What is Kubernetes
in the Kubernetes documentation. -
An existing Kubernetes cluster. If you don't have an existing cluster, see Getting Started with Amazon EKS in the Amazon EKS User Guide. If you're running your own Kubernetes cluster on Amazon EC2, then ensure that Docker is authenticated to the Amazon ECR repository that the Envoy image is in. For more information, see Envoy image, Registry authentication in the Amazon Elastic Container Registry User Guide, and Pull an Image from a Private Registry
in the Kubernetes documentation. -
App Mesh supports Linux services that are registered with DNS, Amazon Cloud Map, or both. To use this getting started guide, we recommend that you have three existing services that are registered with DNS. The procedures in this topic assume that the existing services are named
serviceA
,serviceB
, andserviceBv2
and that all services are discoverable through a namespace namedapps.local
.You can create a service mesh and its resources even if the services don't exist, but you cannot use the mesh until you have deployed actual services.
-
The Amazon CLI version 1.18.116 or later or 2.0.38 or later installed. To install or upgrade the Amazon CLI, see Installing the Amazon CLI.
-
A
kubectl
client that is configured to communicate with your Kubernetes cluster. If you're using Amazon Elastic Kubernetes Service, you can use the instructions for installingkubectl
and configuring akubeconfig
file. -
Helm version 3.0 or later installed. If you don't have Helm installed, see Using Helm with Amazon EKS in the Amazon EKS User Guide.
-
Amazon EKS currently only supports
IPv4_ONLY
andIPv6_ONLY
only IP preferences, because Amazon EKS currently only supports pods that are capable of serving either onlyIPv4
traffic or onlyIPv6
traffic.
The remaining steps assume that the actual services are named serviceA
,
serviceB
, and serviceBv2
and that all services are discoverable through a
namespace named apps.local
.
Step 1: Install the integration components
Install the integration components one time to each cluster that hosts pods that you want to use with App Mesh.
To install the integration components
-
The remaining steps of this procedure require a cluster without a pre-release version of the controller installed. If you have installed a pre-release version, or are not sure whether you have, you can download and run a script that checks to see whether a pre-release version is installed on your cluster.
curl -o pre_upgrade_check.sh https://raw.githubusercontent.com/aws/eks-charts/master/stable/appmesh-controller/upgrade/pre_upgrade_check.sh sh ./pre_upgrade_check.sh
If the script returns
Your cluster is ready for upgrade. Please proceed to the installation instructions
then you can proceed to the next step. If a different message is returned, then you'll need to complete the upgrade steps before continuing. For more information about upgrading a pre-release version, see Upgradeon GitHub. -
Add the
eks-charts
repository to Helm.helm repo add eks https://aws.github.io/eks-charts
-
Install the App Mesh Kubernetes custom resource definitions (CRD).
kubectl apply -k "https://github.com/aws/eks-charts/stable/appmesh-controller/crds?ref=master"
-
Create a Kubernetes namespace for the controller.
kubectl create ns appmesh-system
-
Set the following variables for use in later steps. Replace
andcluster-name
with the values for your existing cluster.Region-code
export CLUSTER_NAME=
cluster-name
export AWS_REGION=Region-code
-
(Optional) If you want to run the controller on Fargate, then you need to create a Fargate profile. If you don't have
eksctl
installed, see Installing or Upgradingeksctl
in the Amazon EKS User Guide. If you'd prefer to create the profile using the console, see Creating a Fargate profile in the Amazon EKS User Guide.eksctl create fargateprofile --cluster $CLUSTER_NAME --name
appmesh-system
--namespaceappmesh-system
-
Create an OpenID Connect (OIDC) identity provider for your cluster. If you don't have
eksctl
installed, you can install it with the instructions in Installing or upgradingeksctl
in the Amazon EKS User Guide. If you'd prefer to create the provider using the console, see Enabling IAM roles for service accounts on your cluster in the Amazon EKS User Guide.eksctl utils associate-iam-oidc-provider \ --region=$AWS_REGION \ --cluster $CLUSTER_NAME \ --approve
-
Create an IAM role, attach the AWSAppMeshFullAccess
and AWSCloudMapFullAccess Amazon managed policies to it, and bind it to the appmesh-controller
Kubernetes service account. The role enables the controller to add, remove, and change App Mesh resources.Note
The command creates an Amazon IAM role with an auto-generated name. You are not able to specify the IAM role name that is created.
eksctl create iamserviceaccount \ --cluster $CLUSTER_NAME \ --namespace appmesh-system \ --name appmesh-controller \ --attach-policy-arn arn:aws:iam::aws:policy/AWSCloudMapFullAccess,arn:aws:iam::aws:policy/AWSAppMeshFullAccess \ --override-existing-serviceaccounts \ --approve
If you prefer to create the service account using the Amazon Web Services Management Console or Amazon CLI, see Creating an IAM role and policy for your service account in the Amazon EKS User Guide. If you use the Amazon Web Services Management Console or Amazon CLI to create the account, you also need to map the role to a Kubernetes service account. For more information, see Specifying an IAM role for your service account in the Amazon EKS User Guide.
-
Deploy the App Mesh controller. For a list of all configuration options, see Configuration
on GitHub. -
To deploy the App Mesh controller for a private cluster, you have to enable App Mesh and service discovery Amazon VPC endpoints to the linked private subnet first. You're also required to set the
accountId
.--set accountId=$AWS_ACCOUNT_ID
To enable X-Ray tracing in a private cluster, enable the X-Ray and Amazon ECR Amazon VPC endpoints. The controller uses
public.ecr.aws/xray/aws-xray-daemon:latest
by default, so pull this image to local and push it into your personal ECR repository.Note
Amazon VPC endpoints currently don't support Amazon ECR public repositories.
The following example shows deploying the controller with configurations for X-Ray.
helm upgrade -i appmesh-controller eks/appmesh-controller \ --namespace appmesh-system \ --set region=
$AWS_REGION
\ --set serviceAccount.create=false \ --set serviceAccount.name=appmesh-controller \ --set accountId=$AWS_ACCOUNT_ID
\ --set log.level=debug \ --set tracing.enabled=true \ --set tracing.provider=x-ray \ --set xray.image.repository=your-account-id
.dkr.ecr.your-region
.amazonaws.com/your-repository
\ --set xray.image.tag=your-xray-daemon-image-tag
Verify if the X-Ray daemon is injected successfully when binding the application deployment with your virtual node or gateway.
For more information, see Private Clusters in the Amazon EKS User Guide.
-
Deploy the App Mesh controller for other clusters. For a list of all configuration options, see Configuration
on GitHub. helm upgrade -i appmesh-controller eks/appmesh-controller \ --namespace appmesh-system \ --set region=$AWS_REGION \ --set serviceAccount.create=false \ --set serviceAccount.name=appmesh-controller
Note
If your Amazon EKS cluster family is
IPv6
, please set the cluster name when deploying the App Mesh controller by adding the following option to the previous command--set clusterName=$CLUSTER_NAME
.Important
If your cluster is in the
cn-north-1
orcn-northwest-1
, then you need to add the following options to the previous command.Replace
account-id
andRegion-code
with one of the appropriate sets of values.--set sidecar.image.repository=
account-id
.dkr.ecr.Region-code
.amazonaws.com.cn/aws-appmesh-envoy-
919366029133.dkr.ecr.cn-north-1.amazonaws.com.cn/aws-appmesh-envoy:v1.29.6.1-prod
-
919830735681.dkr.ecr.cn-northwest-1.amazonaws.com.cn/aws-appmesh-envoy:v1.29.6.1-prod
--set init.image.repository=
account-id
.dkr.ecr.Region-code
.amazonaws.com.cn/aws-appmesh-proxy-route-manager-
919366029133.dkr.ecr.cn-north-1.amazonaws.com.cn/aws-appmesh-proxy-route-manager:v7-prod
-
919830735681.dkr.ecr.cn-northwest-1.amazonaws.com.cn/aws-appmesh-proxy-route-manager:v7-prod
--set image.repository=
account-id
.dkr.ecr.Region-code
.amazonaws.com.cn/appmesh-controller-
918309763551.dkr.ecr.cn-north-1.amazonaws.com.cn/amazon/appmesh-controller
-
961992271922.dkr.ecr.cn-northwest-1.amazonaws.com.cn/amazon/appmesh-controller
Important
If your cluster is in the
me-south-1
,ap-east-1
,ap-southeast-3
,eu-south-1
,il-central-1
, oraf-south-1
Regions, then you need to add the following option to the previous command:Replace
account-id
andRegion-code
with one of the appropriate sets of values.-
For the sidecar image:
-
--set image.repository=
account-id
.dkr.ecr.Region-code
.amazonaws.com/amazon/appmesh-controller -
772975370895.dkr.ecr.me-south-1.amazonaws.com/aws-appmesh-envoy:v1.29.6.1-prod
-
856666278305.dkr.ecr.ap-east-1.amazonaws.com/aws-appmesh-envoy:v1.29.6.1-prod
-
909464085924.dkr.ecr.ap-southeast-3.amazonaws.com/aws-appmesh-envoy:v1.29.6.1-prod
-
422531588944.dkr.ecr.eu-south-1.amazonaws.com/aws-appmesh-envoy:v1.29.6.1-prod
-
564877687649.dkr.ecr.il-central-1.amazonaws.com/aws-appmesh-envoy:v1.29.6.1-prod
-
924023996002.dkr.ecr.af-south-1.amazonaws.com/aws-appmesh-envoy:v1.29.6.1-prod
-
-
The older image URIs can be found in the change log
on GitHub. The Amazon accounts on which the images are present have changed in version v1.5.0
. Older version of the images are hosted on Amazon accounts found on the Amazon Elastic Kubernetes Service Amazon container image registries. -
For the controller image:
-
--set sidecar.image.repository=
account-id
.dkr.ecr.Region-code
.amazonaws.com/aws-appmesh-envoy -
772975370895.dkr.ecr.me-south-1.amazonaws.com/amazon/appmesh-controller:v1.13.1
-
856666278305.dkr.ecr.ap-east-1.amazonaws.com/amazon/appmesh-controller:v1.13.1
-
909464085924.dkr.ecr.ap-southeast-3.amazonaws.com/amazon/appmesh-controller:v1.13.1
-
422531588944.dkr.ecr.eu-south-1.amazonaws.com/amazon/appmesh-controller:v1.13.1
-
564877687649.dkr.ecr.il-central-1.amazonaws.com/amazon/appmesh-controller:v1.13.1
-
924023996002.dkr.ecr.af-south-1.amazonaws.com/amazon/appmesh-controller:v1.13.1
-
-
For the sidecar init image:
-
--set sidecar.image.repository=
account-id
.dkr.ecr.Region-code
.amazonaws.com/aws-appmesh-envoy -
772975370895.dkr.ecr.me-south-1.amazonaws.com/aws-appmesh-proxy-route-manager:v7-prod
-
856666278305.dkr.ecr.ap-east-1.amazonaws.com/aws-appmesh-proxy-route-manager:v7-prod
-
909464085924.dkr.ecr.ap-southeast-3.amazonaws.com/aws-appmesh-proxy-route-manager:v7-prod
-
422531588944.dkr.ecr.eu-south-1.amazonaws.com/aws-appmesh-proxy-route-manager:v7-prod
-
564877687649.dkr.ecr.il-central-1.amazonaws.com/aws-appmesh-proxy-route-manager:v7-prod
-
924023996002.dkr.ecr.af-south-1.amazonaws.com/aws-appmesh-proxy-route-manager:v7-prod
-
Important
Only version v1.9.0.0-prod or later is supported for use with App Mesh.
-
-
Confirm that the controller version is
v1.4.0
or later. You can review the change logon GitHub. kubectl get deployment appmesh-controller \ -n appmesh-system \ -o json | jq -r ".spec.template.spec.containers[].image" | cut -f2 -d ':'
Note
If you view the log for the running container, you may see a line that includes the following text, which can be safely ignored.
Neither -kubeconfig nor -master was specified. Using the inClusterConfig. This might not work.
Step 2: Deploy App Mesh resources
When you deploy an application in Kubernetes, you also create the Kubernetes custom resources so that
the controller can create the corresponding App Mesh resources. The following procedure helps you deploy
App Mesh resources with some of their features. You can find example manifests for deploying other App Mesh
resource features in the v1beta2
sub-folders of many of the feature folders listed at
App Mesh
walkthroughs
Important
Once the controller has created an App Mesh resource, we recommend that you only make changes to or
delete the App Mesh resource using the controller. If you make changes to or delete the resource
using App Mesh, the controller won't change or recreate the changed or deleted App Mesh resource for
ten hours, by default. You can configure this duration to be less. For more information, see Configuration
To deploy App Mesh resources
-
Create a Kubernetes namespace to deploy App Mesh resources to.
-
Save the following contents to a file named
namespace.yaml
on your computer.apiVersion: v1 kind: Namespace metadata: name:
my-apps
labels: mesh:my-mesh
appmesh.k8s.aws/sidecarInjectorWebhook: enabled -
Create the namespace.
kubectl apply -f namespace.yaml
-
-
Create an App Mesh service mesh.
-
Save the following contents to a file named
mesh.yaml
on your computer. The file is used to create a mesh resource named
. A service mesh is a logical boundary for network traffic between the services that reside within it.my-mesh
apiVersion: appmesh.k8s.aws/v1beta2 kind: Mesh metadata: name:
my-mesh
spec: namespaceSelector: matchLabels: mesh:my-mesh
-
Create the mesh.
kubectl apply -f mesh.yaml
-
View the details of the Kubernetes mesh resource that was created.
kubectl describe mesh
my-mesh
Output
Name: my-mesh Namespace: Labels: <none> Annotations: kubectl.kubernetes.io/last-applied-configuration: {"apiVersion":"appmesh.k8s.aws/v1beta2","kind":"Mesh","metadata":{"annotations":{},"name":"my-mesh"},"spec":{"namespaceSelector":{"matchLa... API Version: appmesh.k8s.aws/v1beta2 Kind: Mesh Metadata: Creation Timestamp: 2020-06-17T14:51:37Z Finalizers: finalizers.appmesh.k8s.aws/mesh-members finalizers.appmesh.k8s.aws/aws-appmesh-resources Generation: 1 Resource Version: 6295 Self Link: /apis/appmesh.k8s.aws/v1beta2/meshes/my-mesh UID: 111a11b1-c11d-1e1f-gh1i-j11k1l111m711 Spec: Aws Name: my-mesh Namespace Selector: Match Labels: Mesh: my-mesh Status: Conditions: Last Transition Time: 2020-06-17T14:51:37Z Status: True Type: MeshActive Mesh ARN: arn:aws:appmesh:us-west-2:111122223333:mesh/my-mesh Observed Generation: 1 Events: <none>
-
View the details about the App Mesh service mesh that the controller created.
aws appmesh describe-mesh --mesh-name
my-mesh
Output
{ "mesh": { "meshName": "my-mesh", "metadata": { "arn": "arn:aws:appmesh:us-west-2:111122223333:mesh/my-mesh", "createdAt": "2020-06-17T09:51:37.920000-05:00", "lastUpdatedAt": "2020-06-17T09:51:37.920000-05:00", "meshOwner": "111122223333", "resourceOwner": "111122223333", "uid": "111a11b1-c11d-1e1f-gh1i-j11k1l111m711", "version": 1 }, "spec": {}, "status": { "status": "ACTIVE" } } }
-
-
Create an App Mesh virtual node. A virtual node acts as a logical pointer to a Kubernetes deployment.
-
Save the following contents to a file named
virtual-node.yaml
on your computer. The file is used to create an App Mesh virtual node named
in themy-service-a
namespace. The virtual node represents a Kubernetes service that is created in a later step. The value formy-apps
hostname
is the fully qualified DNS hostname of the actual service that this virtual node represents.apiVersion: appmesh.k8s.aws/v1beta2 kind: VirtualNode metadata: name:
my-service-a
namespace:my-apps
spec: podSelector: matchLabels: app:my-app-1
listeners: - portMapping: port:80
protocol:http
serviceDiscovery: dns: hostname:my-service-a.my-apps.svc.cluster.local
Virtual nodes have capabilities, such as end-to-end encryption and health checks, that aren't covered in this tutorial. For more information, see Virtual nodes. To see all available settings for a virtual node that you can set in the preceding spec, run the following command.
aws appmesh create-virtual-node --generate-cli-skeleton yaml-input
-
Deploy the virtual node.
kubectl apply -f virtual-node.yaml
-
View the details of the Kubernetes virtual node resource that was created.
kubectl describe virtualnode
my-service-a
-nmy-apps
Output
Name: my-service-a Namespace: my-apps Labels: <none> Annotations: kubectl.kubernetes.io/last-applied-configuration: {"apiVersion":"appmesh.k8s.aws/v1beta2","kind":"VirtualNode","metadata":{"annotations":{},"name":"my-service-a","namespace":"my-apps"},"s... API Version: appmesh.k8s.aws/v1beta2 Kind: VirtualNode Metadata: Creation Timestamp: 2020-06-17T14:57:29Z Finalizers: finalizers.appmesh.k8s.aws/aws-appmesh-resources Generation: 2 Resource Version: 22545 Self Link: /apis/appmesh.k8s.aws/v1beta2/namespaces/my-apps/virtualnodes/my-service-a UID: 111a11b1-c11d-1e1f-gh1i-j11k1l111m711 Spec: Aws Name: my-service-a_my-apps Listeners: Port Mapping: Port: 80 Protocol: http Mesh Ref: Name: my-mesh UID: 111a11b1-c11d-1e1f-gh1i-j11k1l111m711 Pod Selector: Match Labels: App: nginx Service Discovery: Dns: Hostname: my-service-a.my-apps.svc.cluster.local Status: Conditions: Last Transition Time: 2020-06-17T14:57:29Z Status: True Type: VirtualNodeActive Observed Generation: 2 Virtual Node ARN: arn:aws:appmesh:us-west-2:111122223333:mesh/my-mesh/virtualNode/my-service-a_my-apps Events: <none>
-
View the details of the virtual node that the controller created in App Mesh.
Note
Even though the name of the virtual node created in Kubernetes is
, the name of the virtual node created in App Mesh ismy-service-a
. The controller appends the Kubernetes namespace name to the App Mesh virtual node name when it creates the App Mesh resource. The namespace name is added because in Kubernetes you can create virtual nodes with the same name in different namespaces, but in App Mesh a virtual node name must be unique within a mesh.my-service-a_my-apps
aws appmesh describe-virtual-node --mesh-name
my-mesh
--virtual-node-namemy-service-a_my-apps
Output
{ "virtualNode": { "meshName": "my-mesh", "metadata": { "arn": "arn:aws:appmesh:us-west-2:111122223333:mesh/my-mesh/virtualNode/my-service-a_my-apps", "createdAt": "2020-06-17T09:57:29.840000-05:00", "lastUpdatedAt": "2020-06-17T09:57:29.840000-05:00", "meshOwner": "111122223333", "resourceOwner": "111122223333", "uid": "111a11b1-c11d-1e1f-gh1i-j11k1l111m711", "version": 1 }, "spec": { "backends": [], "listeners": [ { "portMapping": { "port": 80, "protocol": "http" } } ], "serviceDiscovery": { "dns": { "hostname": "my-service-a.my-apps.svc.cluster.local" } } }, "status": { "status": "ACTIVE" }, "virtualNodeName": "my-service-a_my-apps" } }
-
-
Create an App Mesh virtual router. Virtual routers handle traffic for one or more virtual services within your mesh.
-
Save the following contents to a file named
virtual-router.yaml
on your computer. The file is used to create a virtual router to route traffic to the virtual node namedmy-service-a
that was created in the previous step. The controller creates the App Mesh virtual router and route resources. You can specify many more capabilities for your routes and use protocols other thanhttp
. For more information, see Virtual routers and Routes. Notice that the virtual node name referenced is the Kubernetes virtual node name, not the App Mesh virtual node name that was created in App Mesh by the controller.apiVersion: appmesh.k8s.aws/v1beta2 kind: VirtualRouter metadata: namespace:
my-apps
name:my-service-a-virtual-router
spec: listeners: - portMapping: port:80
protocol:http
routes: - name:my-service-a-route
httpRoute: match: prefix:/
action: weightedTargets: - virtualNodeRef: name:my-service-a
weight:1
(Optional) To see all available settings for a virtual router that you can set in the preceding spec, run the following command.
aws appmesh create-virtual-router --generate-cli-skeleton yaml-input
To see all available settings for a route that you can set in the preceding spec, run the following command.
aws appmesh create-route --generate-cli-skeleton yaml-input
-
Deploy the virtual router.
kubectl apply -f virtual-router.yaml
-
View the Kubernetes virtual router resource that was created.
kubectl describe virtualrouter
my-service-a-virtual-router
-nmy-apps
Abbreviated output
Name: my-service-a-virtual-router Namespace: my-apps Labels: <none> Annotations: kubectl.kubernetes.io/last-applied-configuration: {"apiVersion":"appmesh.k8s.aws/v1beta2","kind":"VirtualRouter","metadata":{"annotations":{},"name":"my-service-a-virtual-router","namespac... API Version: appmesh.k8s.aws/v1beta2 Kind: VirtualRouter ... Spec: Aws Name: my-service-a-virtual-router_my-apps Listeners: Port Mapping: Port: 80 Protocol: http Mesh Ref: Name: my-mesh UID: 111a11b1-c11d-1e1f-gh1i-j11k1l111m711 Routes: Http Route: Action: Weighted Targets: Virtual Node Ref: Name: my-service-a Weight: 1 Match: Prefix: / Name: my-service-a-route Status: Conditions: Last Transition Time: 2020-06-17T15:14:01Z Status: True Type: VirtualRouterActive Observed Generation: 1 Route AR Ns: My - Service - A - Route: arn:aws:appmesh:us-west-2:111122223333:mesh/my-mesh/virtualRouter/my-service-a-virtual-router_my-apps/route/my-service-a-route Virtual Router ARN: arn:aws:appmesh:us-west-2:111122223333:mesh/my-mesh/virtualRouter/my-service-a-virtual-router_my-apps Events: <none>
-
View the virtual router resource that the controller created in App Mesh. You specify
my-service-a-virtual-router_my-apps
forname
, because when the controller created the virtual router in App Mesh, it appended the Kubernetes namespace name to the name of the virtual router.aws appmesh describe-virtual-router --virtual-router-name
my-service-a-virtual-router_my-apps
--mesh-namemy-mesh
Output
{ "virtualRouter": { "meshName": "my-mesh", "metadata": { "arn": "arn:aws:appmesh:us-west-2:111122223333:mesh/my-mesh/virtualRouter/my-service-a-virtual-router_my-apps", "createdAt": "2020-06-17T10:14:01.547000-05:00", "lastUpdatedAt": "2020-06-17T10:14:01.547000-05:00", "meshOwner": "111122223333", "resourceOwner": "111122223333", "uid": "111a11b1-c11d-1e1f-gh1i-j11k1l111m711", "version": 1 }, "spec": { "listeners": [ { "portMapping": { "port": 80, "protocol": "http" } } ] }, "status": { "status": "ACTIVE" }, "virtualRouterName": "my-service-a-virtual-router_my-apps" } }
-
View the route resource that the controller created in App Mesh. A route resource was not created in Kubernetes because the route is part of the virtual router configuration in Kubernetes. The route information was shown in the Kubernetes resource detail in sub-step
c
. The controller did not append the Kubernetes namespace name to the App Mesh route name when it created the route in App Mesh because route names are unique to a virtual router.aws appmesh
describe-route
\ --route-namemy-service-a-route
\ --virtual-router-namemy-service-a-virtual-router_my-apps
\ --mesh-namemy-mesh
Output
{ "route": { "meshName": "my-mesh", "metadata": { "arn": "arn:aws:appmesh:us-west-2:111122223333:mesh/my-mesh/virtualRouter/my-service-a-virtual-router_my-apps/route/my-service-a-route", "createdAt": "2020-06-17T10:14:01.577000-05:00", "lastUpdatedAt": "2020-06-17T10:14:01.577000-05:00", "meshOwner": "111122223333", "resourceOwner": "111122223333", "uid": "111a11b1-c11d-1e1f-gh1i-j11k1l111m711", "version": 1 }, "routeName": "my-service-a-route", "spec": { "httpRoute": { "action": { "weightedTargets": [ { "virtualNode": "my-service-a_my-apps", "weight": 1 } ] }, "match": { "prefix": "/" } } }, "status": { "status": "ACTIVE" }, "virtualRouterName": "my-service-a-virtual-router_my-apps" } }
-
-
Create an App Mesh virtual service. A virtual service is an abstraction of a real service that is provided by a virtual node directly or indirectly by means of a virtual router. Dependent services call your virtual service by its name. Though the name doesn't matter to App Mesh, we recommend naming the virtual service the fully qualified domain name of the actual service that the virtual service represents. By naming your virtual services this way, you don't need to change your application code to reference a different name. The requests are routed to the virtual node or virtual router that is specified as the provider for the virtual service.
-
Save the following contents to a file named
virtual-service.yaml
on your computer. The file is used to create a virtual service that uses a virtual router provider to route traffic to the virtual node namedmy-service-a
that was created in a previous step. The value forawsName
in thespec
is the fully qualified domain name (FQDN) of the actual Kubernetes service that this virtual service abstracts. The Kubernetes service is created in Step 3: Create or update services. For more information, see Virtual services.apiVersion: appmesh.k8s.aws/v1beta2 kind: VirtualService metadata: name:
my-service-a
namespace:my-apps
spec: awsName:my-service-a.my-apps.svc.cluster.local
provider: virtualRouter: virtualRouterRef: name:my-service-a-virtual-router
To see all available settings for a virtual service that you can set in the preceding spec, run the following command.
aws appmesh create-virtual-service --generate-cli-skeleton yaml-input
-
Create the virtual service.
kubectl apply -f virtual-service.yaml
-
View the details of the Kubernetes virtual service resource that was created.
kubectl describe virtualservice
my-service-a
-nmy-apps
Output
Name: my-service-a Namespace: my-apps Labels: <none> Annotations: kubectl.kubernetes.io/last-applied-configuration: {"apiVersion":"appmesh.k8s.aws/v1beta2","kind":"VirtualService","metadata":{"annotations":{},"name":"my-service-a","namespace":"my-apps"}... API Version: appmesh.k8s.aws/v1beta2 Kind: VirtualService Metadata: Creation Timestamp: 2020-06-17T15:48:40Z Finalizers: finalizers.appmesh.k8s.aws/aws-appmesh-resources Generation: 1 Resource Version: 13598 Self Link: /apis/appmesh.k8s.aws/v1beta2/namespaces/my-apps/virtualservices/my-service-a UID: 111a11b1-c11d-1e1f-gh1i-j11k1l111m711 Spec: Aws Name: my-service-a.my-apps.svc.cluster.local Mesh Ref: Name: my-mesh UID: 111a11b1-c11d-1e1f-gh1i-j11k1l111m711 Provider: Virtual Router: Virtual Router Ref: Name: my-service-a-virtual-router Status: Conditions: Last Transition Time: 2020-06-17T15:48:40Z Status: True Type: VirtualServiceActive Observed Generation: 1 Virtual Service ARN: arn:aws:appmesh:us-west-2:111122223333:mesh/my-mesh/virtualService/my-service-a.my-apps.svc.cluster.local Events: <none>
-
View the details of the virtual service resource that the controller created in App Mesh. The Kubernetes controller did not append the Kubernetes namespace name to the App Mesh virtual service name when it created the virtual service in App Mesh because the virtual service's name is a unique FQDN.
aws appmesh describe-virtual-service --virtual-service-name
my-service-a.my-apps.svc.cluster.local
--mesh-namemy-mesh
Output
{ "virtualService": { "meshName": "my-mesh", "metadata": { "arn": "arn:aws:appmesh:us-west-2:111122223333:mesh/my-mesh/virtualService/my-service-a.my-apps.svc.cluster.local", "createdAt": "2020-06-17T10:48:40.182000-05:00", "lastUpdatedAt": "2020-06-17T10:48:40.182000-05:00", "meshOwner": "111122223333", "resourceOwner": "111122223333", "uid": "111a11b1-c11d-1e1f-gh1i-j11k1l111m711", "version": 1 }, "spec": { "provider": { "virtualRouter": { "virtualRouterName": "my-service-a-virtual-router_my-apps" } } }, "status": { "status": "ACTIVE" }, "virtualServiceName": "my-service-a.my-apps.svc.cluster.local" } }
-
Though not covered in this tutorial, the controller can also deploy App Mesh Virtual gateways and Gateway routes. For a walkthrough of deploying
these resources with the controller, see Configuring Inbound Gateway
Step 3: Create or update services
Any pods that you want to use with App Mesh must have the App Mesh sidecar containers added to them. The injector automatically adds the sidecar containers to any pod deployed with a label that you specify.
-
Enable proxy authorization. We recommend that you enable each Kubernetes deployment to stream only the configuration for its own App Mesh virtual node.
-
Save the following contents to a file named
proxy-auth.json
on your computer. Make sure to replace thealternate-colored values
with your own.{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": "appmesh:StreamAggregatedResources", "Resource": [ "arn:aws:appmesh:
Region-code
:111122223333
:mesh/my-mesh/virtualNode/my-service-a_my-apps" ] } ] } -
Create the policy.
aws iam create-policy --policy-name
my-policy
--policy-document file://proxy-auth.json -
Create an IAM role, attach the policy you created in the previous step to it, create a Kubernetes service account, and bind the policy to the Kubernetes service account. The role enables the controller to add, remove, and change App Mesh resources.
eksctl create iamserviceaccount \ --cluster $CLUSTER_NAME \ --namespace
my-apps
\ --namemy-service-a
\ --attach-policy-arn arn:aws:iam::111122223333
:policy/my-policy
\ --override-existing-serviceaccounts \ --approveIf you prefer to create the service account using the Amazon Web Services Management Console or Amazon CLI, see Creating an IAM Role and policy for your service account in the Amazon EKS User Guide. If you use the Amazon Web Services Management Console or Amazon CLI to create the account, you also need to map the role to a Kubernetes service account. For more information, see Specifying an IAM role for your service account in the Amazon EKS User Guide.
-
-
(Optional) If you want to deploy your deployment to Fargate pods, then you need to create a Fargate profile. If you don't have
eksctl
installed, you can install it with the instructions in Installing or Upgradingeksctl
in the Amazon EKS User Guide. If you'd prefer to create the profile using the console, see Creating a Fargate profile in the Amazon EKS User Guide.eksctl create fargateprofile --cluster
my-cluster
--regionRegion-code
--namemy-service-a
--namespacemy-apps
-
Create a Kubernetes service and deployment. If you have an existing deployment that you want to use with App Mesh, then you need to deploy a virtual node, as you did in sub-step
3
of Step 2: Deploy App Mesh resources. Update your deployment to make sure that its label matches the label that you set on the virtual node, so that the sidecar containers are automatically added to the pods and the pods are redeployed.-
Save the following contents to a file named
example-service.yaml
on your computer. If you change the namespace name and are using Fargate pods, make sure that the namespace name matches the namespace name that you defined in your Fargate profile.apiVersion: v1 kind: Service metadata: name:
my-service-a
namespace:my-apps
labels: app:my-app-1
spec: selector: app:my-app-1
ports: - protocol:TCP
port:80
targetPort:80
--- apiVersion: apps/v1 kind: Deployment metadata: name:my-service-a
namespace:my-apps
labels: app:my-app-1
spec: replicas:3
selector: matchLabels: app:my-app-1
template: metadata: labels: app:my-app-1
spec: serviceAccountName:my-service-a
containers: - name:nginx
image:nginx:1.19.0
ports: - containerPort:80
Important
The value for the
app
matchLabels
selector
in the spec must match the value that you specified when you created the virtual node in sub-step3
of Step 2: Deploy App Mesh resources, or the sidecar containers won't be injected into the pod. In the previous example, the value for the label ismy-app-1
. If you deploy a virtual gateway, rather than a virtual node, then theDeployment
manifest should include only the Envoy container. For more information about the image to use, see Envoy image. For a sample manfest, see the deployment exampleon GitHub. -
Deploy the service.
kubectl apply -f
example-service.yaml
-
View the service and deployment.
kubectl -n
my-apps
get podsOutput
NAME READY STATUS RESTARTS AGE my-service-a-54776556f6-2cxd9 2/2 Running 0 10s my-service-a-54776556f6-w26kf 2/2 Running 0 18s my-service-a-54776556f6-zw5kt 2/2 Running 0 26s
-
View the details for one of the pods that was deployed.
kubectl -n
my-apps
describe podmy-service-a-54776556f6-2cxd9
Abbreviated output
Name: my-service-a-54776556f6-2cxd9 Namespace: my-app-1 Priority: 0 Node: ip-192-168-44-157.us-west-2.compute.internal/192.168.44.157 Start Time: Wed, 17 Jun 2020 11:08:59 -0500 Labels: app=nginx pod-template-hash=54776556f6 Annotations: kubernetes.io/psp: eks.privileged Status: Running IP: 192.168.57.134 IPs: IP: 192.168.57.134 Controlled By: ReplicaSet/my-service-a-54776556f6 Init Containers: proxyinit: Container ID: docker://e0c4810d584c21ae0cb6e40f6119d2508f029094d0e01c9411c6cf2a32d77a59 Image: 111345817488.dkr.ecr.us-west-2.amazonaws.com/aws-appmesh-proxy-route-manager:v2 Image ID: docker-pullable://111345817488.dkr.ecr.us-west-2.amazonaws.com/aws-appmesh-proxy-route-manager Port: <none> Host Port: <none> State: Terminated Reason: Completed Exit Code: 0 Started: Fri, 26 Jun 2020 08:36:22 -0500 Finished: Fri, 26 Jun 2020 08:36:22 -0500 Ready: True Restart Count: 0 Requests: cpu: 10m memory: 32Mi Environment: APPMESH_START_ENABLED: 1 APPMESH_IGNORE_UID: 1337 APPMESH_ENVOY_INGRESS_PORT: 15000 APPMESH_ENVOY_EGRESS_PORT: 15001 APPMESH_APP_PORTS: 80 APPMESH_EGRESS_IGNORED_IP: 169.254.169.254 APPMESH_EGRESS_IGNORED_PORTS: 22 AWS_ROLE_ARN: arn:aws:iam::111122223333:role/eksctl-app-mesh-addon-iamserviceaccount-my-a-Role1-NMNCVWB6PL0N AWS_WEB_IDENTITY_TOKEN_FILE: /var/run/secrets/eks.amazonaws.com/serviceaccount/token ... Containers: nginx: Container ID: docker://be6359dc6ecd3f18a1c87df7b57c2093e1f9db17d5b3a77f22585ce3bcab137a Image: nginx:1.19.0 Image ID: docker-pullable://nginx Port: 80/TCP Host Port: 0/TCP State: Running Started: Fri, 26 Jun 2020 08:36:28 -0500 Ready: True Restart Count: 0 Environment: AWS_ROLE_ARN: arn:aws:iam::111122223333:role/eksctl-app-mesh-addon-iamserviceaccount-my-a-Role1-NMNCVWB6PL0N AWS_WEB_IDENTITY_TOKEN_FILE: /var/run/secrets/eks.amazonaws.com/serviceaccount/token ... envoy: Container ID: docker://905b55cbf33ef3b3debc51cb448401d24e2e7c2dbfc6a9754a2c49dd55a216b6 Image: 840364872350.dkr.ecr.us-west-2.amazonaws.com/aws-appmesh-envoy:v1.12.4.0-prod Image ID: docker-pullable://840364872350.dkr.ecr.us-west-2.amazonaws.com/aws-appmesh-envoy Port: 9901/TCP Host Port: 0/TCP State: Running Started: Fri, 26 Jun 2020 08:36:36 -0500 Ready: True Restart Count: 0 Requests: cpu: 10m memory: 32Mi Environment: APPMESH_RESOURCE_ARN: arn:aws:iam::111122223333:mesh/my-mesh/virtualNode/my-service-a_my-apps APPMESH_PREVIEW: 0 ENVOY_LOG_LEVEL: info AWS_REGION: us-west-2 AWS_ROLE_ARN: arn:aws:iam::111122223333:role/eksctl-app-mesh-addon-iamserviceaccount-my-a-Role1-NMNCVWB6PL0N AWS_WEB_IDENTITY_TOKEN_FILE: /var/run/secrets/eks.amazonaws.com/serviceaccount/token ... Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal Pulling 30s kubelet, ip-192-168-44-157.us-west-2.compute.internal Pulling image "111345817488.dkr.ecr.us-west-2.amazonaws.com/aws-appmesh-proxy-route-manager:v2" Normal Pulled 23s kubelet, ip-192-168-44-157.us-west-2.compute.internal Successfully pulled image "111345817488.dkr.ecr.us-west-2.amazonaws.com/aws-appmesh-proxy-route-manager:v2" Normal Created 21s kubelet, ip-192-168-44-157.us-west-2.compute.internal Created container proxyinit Normal Started 21s kubelet, ip-192-168-44-157.us-west-2.compute.internal Started container proxyinit Normal Pulling 20s kubelet, ip-192-168-44-157.us-west-2.compute.internal Pulling image "nginx:1.19.0" Normal Pulled 16s kubelet, ip-192-168-44-157.us-west-2.compute.internal Successfully pulled image "nginx:1.19.0" Normal Created 15s kubelet, ip-192-168-44-157.us-west-2.compute.internal Created container nginx Normal Started 15s kubelet, ip-192-168-44-157.us-west-2.compute.internal Started container nginx Normal Pulling 15s kubelet, ip-192-168-44-157.us-west-2.compute.internal Pulling image "840364872350.dkr.ecr.us-west-2.amazonaws.com/aws-appmesh-envoy:v1.12.4.0-prod" Normal Pulled 8s kubelet, ip-192-168-44-157.us-west-2.compute.internal Successfully pulled image "840364872350.dkr.ecr.us-west-2.amazonaws.com/aws-appmesh-envoy:v1.12.4.0-prod" Normal Created 7s kubelet, ip-192-168-44-157.us-west-2.compute.internal Created container envoy Normal Started 7s kubelet, ip-192-168-44-157.us-west-2.compute.internal Started container envoy
In the preceding output, you can see that the
proxyinit
andenvoy
containers were added to the pod by the controller. If you deployed the example service to Fargate, then theenvoy
container was added to the pod by the controller, but theproxyinit
container was not.
-
-
(Optional) Install add-ons such as Prometheus, Grafana, Amazon X-Ray, Jaeger, and Datadog. For more information, see App Mesh add-ons
on GitHub and the Observability section of the App Mesh User Guide.
Note
For more examples and walkthroughs for App Mesh, see the App Mesh examples repository
Step 4: Clean up
Remove all of the example resources created in this tutorial. The controller also removes the
resources that were created in the my-mesh
App Mesh service mesh.
kubectl delete namespace
my-apps
If you created a Fargate profile for the example service, then remove it.
eksctl delete fargateprofile --name
my-service-a
--clustermy-cluster
--regionRegion-code
Delete the mesh.
kubectl delete mesh
my-mesh
(Optional) You can remove the Kubernetes integration components.
helm delete appmesh-controller -n appmesh-system
(Optional) If you deployed the Kubernetes integration components to Fargate, then delete the Fargate profile.
eksctl delete fargateprofile --name
appmesh-system
--clustermy-cluster
--regionRegion-code