Migrate to OpenTelemetry Node.js

When the X-Ray SDK for Node.js was used, the aws-xray-sdk package was required to configure the X-Ray SDK with service plug-ins or local sampling rules before using the SDK to instrument your code.

var AWSXRay = require('aws-xray-sdk');

AWSXRay.config([AWSXRay.plugins.EC2Plugin,AWSXRay.plugins.ElasticBeanstalkPlugin]);
AWSXRay.middleware.setSamplingRules(<path to file>);
        

For the code example below, you will need the following dependencies:

npm install --save \
  @opentelemetry/api \
  @opentelemetry/sdk-node \
  @opentelemetry/exporter-trace-otlp-proto \
  @opentelemetry/propagator-aws-xray \
  @opentelemetry/resource-detector-aws
        

You must set up and configure the OpenTelemetry SDK before running your application code. This can be done by using the –-require flag. Create a file named instrumentation.js, which will contain your OpenTelemetry instrumentation configuration and setup.

It is recommend that you configure the following components:

/*instrumentation.js*/
// Require dependencies
const { NodeSDK } = require('@opentelemetry/sdk-node');
const { OTLPTraceExporter } = require('@opentelemetry/exporter-trace-otlp-proto');
const { AWSXRayPropagator } = require("@opentelemetry/propagator-aws-xray");
const { detectResources } = require('@opentelemetry/resources');
const { awsEc2Detector } = require('@opentelemetry/resource-detector-aws');

const resource = detectResources({
    detectors: [awsEc2Detector],
});
 
const _traceExporter = new OTLPTraceExporter({
    url: 'http://localhost:4318/v1/traces'
});

const sdk = new NodeSDK({
    resource: resource,
    textMapPropagator: new AWSXRayPropagator(),
    traceExporter: _traceExporter
});

sdk.start();
        

Then, you can run your application with your OpenTelemetry setup like:

node --require ./instrumentation.js app.js
        

You can use OpenTelemetry SDK library instrumentations to automatically create spans for libraries such as the Amazon SDK. Enabling these will automatically create spans for modules such as the Amazon SDK for JavaScript v3. OpenTelemetry provides the option to enable all library instrumentations or specify which library instrumentations to enable.

To enable all instrumentations, install the @opentelemetry/auto-instrumentations-node package:

npm install @opentelemetry/auto-instrumentations-node
        

Next, update the configuration to enable all library instrumentations as shown below.

const { getNodeAutoInstrumentations } = require('@opentelemetry/auto-instrumentations-node');

...

const sdk = new NodeSDK({
    resource: resource,
     instrumentations: [getNodeAutoInstrumentations()],
     textMapPropagator: new AWSXRayPropagator(),
    traceExporter: _traceExporter
});

You can use the ADOT auto-instrumentation for Node.js to automatically configure OpenTelemetry for your Node.js applications. By using ADOT Auto-Instrumentation, you do not need to make manual code changes to trace incoming requests, or to trace libraries such as the Amazon SDK or HTTP clients. For more information, see Tracing and metrics with the Amazon Distro for OpenTelemetry JavaScript Auto-Instrumentation.

ADOT auto-instrumentation for Node.js supports:

Express.js

With the X-Ray SDK to trace incoming HTTP requests received by Express.js applications, the two middlewares AWSXRay.express.openSegment(<name>) and AWSXRay.express.closeSegment() were required to wrap all of your defined routes in order to trace them.

app.use(xrayExpress.openSegment('defaultName'));

...

app.use(xrayExpress.closeSegment());
          

Restify

To trace incoming HTTP requests received by Restify applications, the middleware from the X-Ray SDK was used by running enable from the aws-xray-sdk-restify module on the Restify Server:

var AWSXRay = require('aws-xray-sdk');
var AWSXRayRestify = require('aws-xray-sdk-restify');

var restify = require('restify');
var server = restify.createServer();
AWSXRayRestify.enable(server, 'MyApp'));
          

Express.js

Tracing support for incoming requests for Express.js is provided by the OpenTelemetry HTTP instrumentation and OpenTelemetry express instrumentation. Install the following dependencies with npm:

npm install --save @opentelemetry/instrumentation-http @opentelemetry/instrumentation-express
          

Update the OpenTelemetry SDK Configuration to enable instrumentation for the express module:

const { HttpInstrumentation } = require('@opentelemetry/instrumentation-http');
const { ExpressInstrumentation } = require('@opentelemetry/instrumentation-express');
...

const sdk = new NodeSDK({
  ...
  
  instrumentations: [
    ...
    // Express instrumentation requires HTTP instrumentation
    new HttpInstrumentation(),
    new ExpressInstrumentation(),
  ],
});
          

Restify

For Restify applications, you will need the OpenTelemetry Restify instrumentation. Install the following dependency:

npm install --save @opentelemetry/instrumentation-restify
          

Update the OpenTelemetry SDK Configuration to enable instrumentation for the restify module:

const { RestifyInstrumentation } = require('@opentelemetry/instrumentation-restify');
...

const sdk = new NodeSDK({
  ...
  
  instrumentations: [
    ...
    new RestifyInstrumentation(),
  ],
});
          

To instrument outgoing Amazon requests from Amazon SDK, you instrumented clients like the following example:

import { S3, PutObjectCommand } from '@aws-sdk/client-s3';

const s3 = AWSXRay.captureAWSv3Client(new S3({}));

await s3.send(new PutObjectCommand({
  Bucket: bucketName,
  Key: keyName,
  Body: 'Hello!',
}));
      

Tracing support for downstream Amazon SDK calls to DynamoDB, Amazon S3, and others is provided by the OpenTelemetry Amazon SDK instrumentation. Install the following dependency with npm:

npm install --save @opentelemetry/instrumentation-aws-sdk
      

Update the OpenTelemetry SDK Configuration with the Amazon SDK instrumentation.

import { AwsInstrumentation } from '@opentelemetry/instrumentation-aws-sdk';
...

const sdk = new NodeSDK({
  ...

  instrumentations: [
    ...
    new AwsInstrumentation()
  ],
});
      

To instrument outgoing HTTP requests with X-Ray, it was required to instrument clients. For example, see below.

Individual HTTP clients

var AWSXRay = require('aws-xray-sdk');
var http = AWSXRay.captureHTTPs(require('http'));
      

All HTTP clients (global)

var AWSXRay = require('aws-xray-sdk');
AWSXRay.captureHTTPsGlobal(require('http'));
var http = require('http');

Tracing support for Node.js HTTP clients is provided by the OpenTelemetry HTTP Instrumentation. Install the following dependency with npm:

npm install --save @opentelemetry/instrumentation-http
      

Update the OpenTelemetry SDK Configuration as follows:

const { HttpInstrumentation } = require('@opentelemetry/instrumentation-http');
...

const sdk = new NodeSDK({
  ...
  
  instrumentations: [
    ...
    new HttpInstrumentation(),
  ],
});
      

Using X-Ray, the aws-xray-sdk package code was required to manually create segments and their child sub-segments to trace your application.

var AWSXRay = require('aws-xray-sdk');

AWSXRay.enableManualMode();

var segment = new AWSXRay.Segment('myApplication');

captureFunc('1', function(subsegment1) {
  captureFunc('2', function(subsegment2) {

  }, subsegment1);
}, segment);

segment.close();
segment.flush();
      

You can create and use custom spans to monitor the performance of internal activities that are not captured by instrumentation libraries. Note that only spans of kind Server are converted into X-Ray segments, all other spans are converted into X-Ray sub-segments. For more information, see Segments.

You will need a Tracer instance after you have configured the OpenTelemetry SDK in Tracing Setup to create Spans. You can create as many Tracer instances as needed, but it is common to have one Tracer for an entire application.

const { trace, SpanKind } = require('@opentelemetry/api');

// Get a tracer instance
const tracer = trace.getTracer('your-tracer-name');

...

  // This span will appear as a segment in X-Ray
  tracer.startActiveSpan('server', { kind: SpanKind.SERVER }, span => {
    // Do work here

    // This span will appear as a subsegment in X-Ray
    tracer.startActiveSpan('operation2', { kind: SpanKind.INTERNAL }, innerSpan => {
      // Do more work here

      innerSpan.end();
    });
    span.end();
  });
      

Adding annotations and metadata to traces with OpenTelemetry SDK

You can also add custom key-value pairs as attributes in your spans. Note that by default, all these span attributes will be converted into metadata in X-Ray raw data. To ensure that an attribute is converted into an annotation and not metadata, add the attribute's key to the list of the aws.xray.annotations attribute. For more information, see Enable The Customized X-Ray Annotations.

  tracer.startActiveSpan('server', { kind: SpanKind.SERVER }, span => {
    span.setAttribute('metadataKey', 'metadataValue');
    span.setAttribute('annotationKey', 'annotationValue');

    // The following ensures that "annotationKey: annotationValue" is an annotation in X-Ray raw data.
    span.setAttribute('aws.xray.annotations', ['annotationKey']);

    // Do work here

    span.end();
  });
      

After you enable Active Tracing for your Lambda function, the X-Ray SDK was required without additional configuration. Lambda creates a segment representing the Lambda handler invocation, and you created sub-segments or instrument libraries using the X-Ray SDK without any additional configuration.

You can automatically instrument your Lambda with Amazon vended Lambda layers. There are two solutions:

Manually creating Spans with Lambda instrumentation

While the ADOT JavaScript Lambda Layer provides automatic instrumentation for your Lambda function, you might find the need to perform manual instrumentation in your Lambda, for example, to provide custom data or to instrument code within the Lambda function itself that is not covered by library instrumentations.

To perform manual instrumentation alongside the automatic instrumentation, you will need to add @opentelemetry/api as a dependency. The version of this dependency is recommended to be the same version of the same dependency that is used by the ADOT JavaScript SDK. You can use the OpenTelemetry API to manually create spans in your Lambda function.

To add the @opentelemetry/api dependency using NPM:

npm install @opentelemetry/api