笔记本执行 CLI 命令示例 - Amazon EMR
Amazon Web Services 文档中描述的 Amazon Web Services 服务或功能可能因区域而异。要查看适用于中国区域的差异,请参阅中国的 Amazon Web Services 服务入门

笔记本执行 CLI 命令示例

以下是从 EMR Notebooks 控制台看到的 EMR Notebooks 演示示例。要运行的笔记本使用相对于主目录的文件路径来定位。在此示例中,有两个可以运行的笔记本文件:demo_pyspark.ipynb 和 my_folder/python3.ipynb。

文件 demo_pyspark.ipynb 的相对路径是 demo_pyspark.ipynb,如下所示。

python3.ipynb 的相对路径是 my_folder/python3.ipynb,如下所示。

有关 EMR API NotebookExecution 操作的更多信息,请参阅 EMR API 操作。

运行笔记本

您可以使用 Amazon CLI 通过 start-notebook-execution 操作来运行您的笔记本,如以下示例所示。

aws emr --region us-east-1 \ start-notebook-execution \ --editor-id e-BKTM2DIHXBEDRU44ANWRKIU8N \ --notebook-params '{"input_param":"my-value", "good_superhero":["superman", "batman"]}' \ --relative-path test.ipynb \ --notebook-execution-name my-execution \ --execution-engine '{"Id" : "j-2QMOV6JAX1TS2"}' \ --service-role EMR_Notebooks_DefaultRole { "NotebookExecutionId": "ex-IZWZZVR9DKQ9WQ7VZWXJZR29UGHTE" }

笔记本输出

下面是示例笔记本的输出。单元格 [3] 显示新注入的参数值。

描述笔记本

您可以使用 describe-notebook-execution 操作以访问关于特定笔记本执行的信息。

aws emr --region us-east-1 \ describe-notebook-execution --notebook-execution-id ex-IZWZZVR9DKQ9WQ7VZWXJZR29UGHTE { "NotebookExecution": { "NotebookExecutionId": "ex-IZWZZVR9DKQ9WQ7VZWXJZR29UGHTE", "EditorId": "e-BKTM2DIHXBEDRU44ANWRKIU8N", "ExecutionEngine": { "Id": "j-2QMOV6JAX1TS2", "Type": "EMR", "MasterInstanceSecurityGroupId": "sg-05ce12e58cd4f715e" }, "NotebookExecutionName": "my-execution", "NotebookParams": "{\"input_param\":\"my-value\", \"good_superhero\":[\"superman\", \"batman\"]}", "Status": "FINISHED", "StartTime": 1593490857.009, "Arn": "arn:aws:elasticmapreduce:us-east-1:123456789012:notebook-execution/ex-IZWZZVR9DKQ9WQ7VZWXJZR29UGHTE", "LastStateChangeReason": "Execution is finished for cluster j-2QMOV6JAX1TS2.", "NotebookInstanceSecurityGroupId": "sg-0683b0a39966d4a6a", "Tags": [] } }

停止运行笔记本

如果您想要停止笔记本正在运行的执行,您可以借助 stop-notebook-execution 命令。

# stop a running execution aws emr --region us-east-1 \ stop-notebook-execution --notebook-execution-id ex-IZWZX78UVPAATC8LHJR129B1RBN4T # describe it aws emr --region us-east-1 \ describe-notebook-execution --notebook-execution-id ex-IZWZX78UVPAATC8LHJR129B1RBN4T { "NotebookExecution": { "NotebookExecutionId": "ex-IZWZX78UVPAATC8LHJR129B1RBN4T", "EditorId": "e-BKTM2DIHXBEDRU44ANWRKIU8N", "ExecutionEngine": { "Id": "j-2QMOV6JAX1TS2", "Type": "EMR" }, "NotebookExecutionName": "my-execution", "NotebookParams": "{\"input_param\":\"my-value\", \"good_superhero\":[\"superman\", \"batman\"]}", "Status": "STOPPED", "StartTime": 1593490876.241, "Arn": "arn:aws:elasticmapreduce:us-east-1:123456789012:editor-execution/ex-IZWZX78UVPAATC8LHJR129B1RBN4T", "LastStateChangeReason": "Execution is stopped for cluster j-2QMOV6JAX1TS2. Internal error", "Tags": [] } }

按开启时间列出笔记本的执行情况

您可以将 --from 参数输入至 list-notebook-executions,按开启时间列出笔记本的执行情况。

# filter by start time aws emr --region us-east-1 \ list-notebook-executions --from 1593400000.000 { "NotebookExecutions": [ { "NotebookExecutionId": "ex-IZWZX78UVPAATC8LHJR129B1RBN4T", "EditorId": "e-BKTM2DIHXBEDRU44ANWRKIU8N", "NotebookExecutionName": "my-execution", "Status": "STOPPED", "StartTime": 1593490876.241 }, { "NotebookExecutionId": "ex-IZWZZVR9DKQ9WQ7VZWXJZR29UGHTE", "EditorId": "e-BKTM2DIHXBEDRU44ANWRKIU8N", "NotebookExecutionName": "my-execution", "Status": "RUNNING", "StartTime": 1593490857.009 }, { "NotebookExecutionId": "ex-IZWZYRS0M14L5V95WZ9OQ399SKMNW", "EditorId": "e-BKTM2DIHXBEDRU44ANWRKIU8N", "NotebookExecutionName": "my-execution", "Status": "STOPPED", "StartTime": 1593490292.995 }, { "NotebookExecutionId": "ex-IZX009ZK83IVY5E33VH8MDMELVK8K", "EditorId": "e-BKTM2DIHXBEDRU44ANWRKIU8N", "NotebookExecutionName": "my-execution", "Status": "FINISHED", "StartTime": 1593489834.765 }, { "NotebookExecutionId": "ex-IZWZXOZF88JWDF9J09GJ91R57VI0N", "EditorId": "e-BKTM2DIHXBEDRU44ANWRKIU8N", "NotebookExecutionName": "my-execution", "Status": "FAILED", "StartTime": 1593488934.688 } ] }

按开启时间和状态列出笔记本的执行情况

list-notebook-executions 命令也可以采用 --status 参数来筛选结果。

# filter by start time and status aws emr --region us-east-1 \ list-notebook-executions --from 1593400000.000 --status FINISHED { "NotebookExecutions": [ { "NotebookExecutionId": "ex-IZWZZVR9DKQ9WQ7VZWXJZR29UGHTE", "EditorId": "e-BKTM2DIHXBEDRU44ANWRKIU8N", "NotebookExecutionName": "my-execution", "Status": "FINISHED", "StartTime": 1593490857.009 }, { "NotebookExecutionId": "ex-IZX009ZK83IVY5E33VH8MDMELVK8K", "EditorId": "e-BKTM2DIHXBEDRU44ANWRKIU8N", "NotebookExecutionName": "my-execution", "Status": "FINISHED", "StartTime": 1593489834.765 } ] }