Amazon managed pipelines
An Amazon managed pipeline is a pipeline that Amazon creates and maintains for a common ML
workflow. You can run it from your account without writing a pipeline definition. Amazon
owns the pipeline, so its ARN uses aws in place of an account ID:
arn:aws:sagemaker:region:aws:pipeline/pipeline-name
When you run a managed pipeline, the execution runs in your account and uses an IAM role that you provide. The jobs that the execution creates, such as training jobs, and the data they write are in your account.
Find managed pipelines
To list managed pipelines, call ListPipelines with
ManagedBy set to AWS. To list only the pipelines that you
created, set it to CUSTOMER.
import boto3 sm = boto3.client("sagemaker") response = sm.list_pipelines(ManagedBy="AWS") for summary in response["PipelineSummaries"]: print(summary["PipelineArn"], summary.get("PipelineDescription"))
To see a managed pipeline's definition and parameters, call
DescribePipeline with its ARN. To see its versions and their aliases, call
ListPipelineVersions.
pipeline_arn = "arn:aws:sagemaker:us-west-2:aws:pipeline/pipeline-name" pipeline = sm.describe_pipeline(PipelineName=pipeline_arn) versions = sm.list_pipeline_versions(PipelineName=pipeline_arn)
Managed pipelines aren't available in every Amazon Web Services Region. If a managed pipeline isn't
available to your account, requests for it fail with a ResourceNotFound
error.
Run a managed pipeline
Call StartPipelineExecution with the pipeline ARN and a
RoleArn. The execution uses this role to create and access resources in
your account, so the role needs the permissions that the pipeline's steps require.
RoleArn is required for a managed pipeline. You can't specify it for a
pipeline that you created, because those executions use the pipeline's role.
response = sm.start_pipeline_execution( PipelineName=pipeline_arn, RoleArn="arn:aws:iam::111122223333:role/MySageMakerExecutionRole", PipelineParameters=[ {"Name": "ParameterName", "Value": "value"}, ], ) execution_arn = response["PipelineExecutionArn"]
The execution ARN contains your account ID. You monitor the execution the same way as
an execution of your own pipeline, with DescribePipelineExecution and
ListPipelineExecutionSteps. DescribePipelineExecution also
returns the RoleArn that the execution uses.
Amazon can release new versions of a managed pipeline. If you don't specify a version,
the execution uses the latest version that's available to you. To get the same behavior on
every run, specify PipelineVersionId or PipelineVersionAlias. For
more information about aliases, see Pipeline version aliases.
Customize a managed pipeline
You can't update or delete a managed pipeline, or change the aliases of its versions.
These requests fail with an AccessDenied error. To change a managed pipeline,
copy its definition into a pipeline in your account and edit the copy. Your copy doesn't
receive the updates that Amazon makes to the managed pipeline.
definition = sm.describe_pipeline(PipelineName=pipeline_arn)["PipelineDefinition"] sm.create_pipeline( PipelineName="my-pipeline", PipelineDefinition=definition, RoleArn="arn:aws:iam::111122223333:role/MySageMakerExecutionRole", )