Using an OpenSearch Ingestion pipeline with OpenTelemetry Collector - Amazon OpenSearch Service
Services or capabilities described in Amazon Web Services documentation might vary by Region. To see the differences applicable to the China Regions, see Getting Started with Amazon Web Services in China (PDF).

Using an OpenSearch Ingestion pipeline with OpenTelemetry Collector

This sample OpenTelemetry configuration file exports trace data from the OpenTelemetry Collector and sends it to an OpenSearch Ingestion pipeline. For more information about ingesting trace data, see Trace Analytics in the Data Prepper documentation.

Note the following:

  • The endpoint value must include your pipeline endpoint. For example, https://pipeline-endpoint.us-east-1.osis.amazonaws.com.

  • The service value must be osis.

  • The compression option for the OTLP/HTTP Exporter must match the compression option on the pipeline's OpenTelemetry source. 

extensions:
  sigv4auth:
    region: "us-east-1"
    service: "osis"
 
receivers:
  jaeger:
    protocols:
      grpc:
 
exporters:
  otlphttp:
    traces_endpoint: "https://pipeline-endpoint.us-east-1.osis.amazonaws.com/v1/traces"
    auth:
      authenticator: sigv4auth
    compression: none
 
service:
  extensions: [sigv4auth]
  pipelines:
    traces:
      receivers: [jaeger]
      exporters: [otlphttp]

You can then configure an OpenSearch Ingestion pipeline like the following, which specifies the OTel trace plugin as the source:

version: "2"
otel-trace-pipeline:
  source:
    otel_trace_source:
      path: "/v1/traces"
  processor:
    - trace_peer_forwarder:
  sink:
    - pipeline:
        name: "trace-pipeline"
    - pipeline:
        name: "service-map-pipeline"
trace-pipeline:
  source:
    pipeline:
      name: "otel-trace-pipeline"
  processor:
    - otel_traces:
  sink:
    - opensearch:
        hosts: ["https://search-domain-endpoint.us-east-1.es.amazonaws.com"]
        index_type: trace-analytics-raw
        aws:
          # IAM role that OpenSearch Ingestion assumes to access the domain sink   
          sts_role_arn: "arn:aws:iam::{account-id}:role/pipeline-role"
          region: "us-east-1"
        
service-map-pipeline:
  source:
    pipeline:
      name: "otel-trace-pipeline"
  processor:
    - service_map:
  sink:
    - opensearch:
        hosts: ["https://search-domain-endpoint.us-east-1.es.amazonaws.com"]
        index_type: trace-analytics-service-map
        aws:
          # IAM role that the pipeline assumes to access the domain sink   
          sts_role_arn: "arn:aws:iam::{account-id}:role/pipeline-role"
          region: "us-east-1"

For another example pipeline, see the preconfigured trace analytics blueprint. For more information, see Using blueprints to create a pipeline.