监控数据加载
Aurora PostgreSQL Limitless Database 提供了多种监控数据加载任务的方式:
列出数据加载任务
您可以连接到集群端点,然后使用 rds_aurora.limitless_data_load_jobs
视图列出数据加载任务。
postgres_limitless=> SELECT * FROM rds_aurora.limitless_data_load_jobs LIMIT 6; job_id | status | message | source_db_identifier | source_db_name | full_load_complete_time | progress_details | start_time | last_updated_time | streaming_mode | source_engine_type | ignore_primary_key_conflict | is_dryrun ---------------+-----------+---------+-------------------------------+----------------+-------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------+------------------------+------------------------+-------------------+--------------------+-----------------------------+----------- 1725697520693 | COMPLETED | | persistent-kdm-auto-source-01 | postgres | 2024-09-07 08:48:15+00 | {"FULL_LOAD": {"STATUS": "COMPLETED", "DETAILS": "9 of 9 tables loaded", "COMPLETED_AT": "2024/09/07 08:48:15+00", "RECORDS_MIGRATED": 600003}} | 2024-09-07 08:47:13+00 | 2024-09-07 08:48:15+00 | full_load | aurora_postgresql | t | f 1725696114225 | COMPLETED | | persistent-kdm-auto-source-01 | postgres | 2024-09-07 08:24:20+00 | {"FULL_LOAD": {"STATUS": "COMPLETED", "DETAILS": "3 of 3 tables loaded", "COMPLETED_AT": "2024/09/07 08:24:20+00", "RECORDS_MIGRATED": 200001}} | 2024-09-07 08:23:56+00 | 2024-09-07 08:24:20+00 | full_load | aurora_postgresql | t | f 1725696067630 | COMPLETED | | persistent-kdm-auto-source-01 | postgres | 2024-09-07 08:23:45+00 | {"FULL_LOAD": {"STATUS": "COMPLETED", "DETAILS": "6 of 6 tables loaded", "COMPLETED_AT": "2024/09/07 08:23:45+00", "RECORDS_MIGRATED": 400002}} | 2024-09-07 08:23:10+00 | 2024-09-07 08:23:45+00 | full_load | aurora_postgresql | t | f 1725694221753 | CANCELED | | persistent-kdm-auto-source-01 | postgres | | {} | 2024-09-07 07:31:18+00 | 2024-09-07 07:51:49+00 | full_load_and_cdc | aurora_postgresql | t | f 1725691698210 | COMPLETED | | persistent-kdm-auto-source-01 | postgres | 2024-09-07 07:10:51+00 | {"FULL_LOAD": {"STATUS": "COMPLETED", "DETAILS": "1 of 1 tables loaded", "COMPLETED_AT": "2024/09/07 07:10:51+00", "RECORDS_MIGRATED": 100000}} | 2024-09-07 07:10:42+00 | 2024-09-07 07:10:52+00 | full_load | aurora_postgresql | t | f 1725691695049 | COMPLETED | | persistent-kdm-auto-source-01 | postgres | 2024-09-07 07:10:48+00 | {"FULL_LOAD": {"STATUS": "COMPLETED", "DETAILS": "1 of 1 tables loaded", "COMPLETED_AT": "2024/09/07 07:10:48+00", "RECORDS_MIGRATED": 100000}} | 2024-09-07 07:10:41+00 | 2024-09-07 07:10:48+00 | full_load | aurora_postgresql | t | f (6 rows)
任务记录将在 90 天后删除。
使用任务 ID 查看数据加载任务的详细信息
如果您知道任务 ID,则可以连接到集群端点,然后使用 rds_aurora.limitless_data_load_job_details
视图查看该数据加载任务的详细信息,包括表名称、任务状态和已加载的行数。您可以在对数据加载启动函数的响应中获取任务 ID,也可以从 rds_aurora.limitless_data_load_jobs
视图中获取。
postgres_limitless=> SELECT * FROM rds_aurora.limitless_data_load_job_details WHERE job_id='1725696114225'; job_id | destination_table_name | destination_schema_name | start_time | status | full_load_rows | full_load_total_rows | full_load_complete_time | cdc_insert | cdc_update | cdc_delete --------------+------------------------+-------------------------+------------------------+-----------+----------------+----------------------+-------------------------+------------+------------+------------ 1725696114225 | standard_1 | public | 2024-09-07 08:23:57+00 | COMPLETED | 100000 | 100000 | 2024-09-07 08:24:08+00 | 0 | 0 | 0 1725696114225 | standard_2 | public | 2024-09-07 08:24:08+00 | COMPLETED | 100000 | 100000 | 2024-09-07 08:24:17+00 | 0 | 0 | 0 1725696114225 | standard_3 | public | 2024-09-07 08:24:18+00 | COMPLETED | 1 | 1 | 2024-09-07 08:24:20+00 | 0 | 0 | 0 1725696114225 | standard_4 | public | 2024-09-07 08:23:58+00 | PENDING | 0 | 0 | | 0 | 0 | 0 (4 rows)
任务记录将在 90 天后删除。
监控 Amazon CloudWatch 日志组
数据加载任务状态更改为 RUNNING
后,您可以使用 Amazon CloudWatch Logs 查看运行进度。
监控 CloudWatch 日志流
登录 Amazon Web Services Management Console并打开 CloudWatch 控制台(https://console.aws.amazon.com/cloudwatch/
-
导航到日志,然后导航到日志组。
-
选择 /aws/rds/aurora-limitless-database 日志组。
-
通过 job_id 搜索您的数据加载任务的日志流。
日志流的模式为 Data-Load-Job-
job_id
。 -
选择日志流以查看日志事件。
每个日志流都显示包含任务状态和已加载到 Aurora PostgreSQL Limitless Database 目标表中的行数的事件。如果数据加载任务失败,还会创建错误日志,显示失败状态和原因。
任务记录将在 90 天后删除。
监控 RDS 事件
数据加载任务还会发布 RDS 事件,例如当任务成功、失败或取消时。您可以查看目标数据库中的事件。
有关更多信息,请参阅 数据库分片组事件。