Question: Vercel OpenTelemetry Collector for Datadog

Help
1

What is the status of the Vercel OpenTelemetry Collector for Datadog? It has been in Beta for sometime now.

I’m interested in using it. However, it lacks support for metrics and logs and this prevents me from using Datadog for observability.

Also, is there a different between OpenTelemetry logs and the logs shipped to Datadog via the Log Drain integration?

If I could get even just OTEL metrics and traces sent from Vercel to Datadog that would greatly simplify my system.

Thanks :folded_hands:

1 Like
2

I just watched this video and it helped me understand the difference between OTEL logs and which might be injected via a Log Drain.

Dude, Where’s My Error?: How OpenTelemetry Records Errors, and Why It Does It Like That

1 Like
3

Hey @kaizen-insights-core. It sounds like you’ve found some of the info you needed from that video. Are there any other questions you have about OTEL and Log Drains?

4

I understand the difference between logs and OTEL logs now.

I would like to know when the Vercel OpenTelemetry Collector for Datadog will be GA and have feature parity with the OTEL collector for New Relic. The Datadog integration is missing OTEL metrics and logs.

Thanks!

6

Great question. I believe feature support comes down to the individual OTEL integrations which are owned by their respective APM vendors, but I’m checking with the team to be sure.

There’s also the option to use custom OTEL exporters without an integration for other APM vendors

1 Like
7

The custom OTEL exporter is interesting but it only supports traces. It doesn’t support OTEL metrics and logs like the New Relic beta collector.

8

Hi Corey! I’m an engineer on the observability team.

@vercel/otel does have support for logs and metrics. A metric reader/log record processor will need to be set in the config for it to work.

When using the vercel otel collector: yes it is true that, as of today, datadog is the only provider which does not support logs/metrics.

1 Like
9

@ethanshea - I saw the code you linked and tried to use a logRecordProcessor to customize the logs before it is forwarded to Datadog. However this code here is not doing anything, the onEmit never gets called:

export function register() {
  registerOTel({
    serviceName: SERVICE_NAME,
    attributes: {
      'service.name': SERVICE_NAME,
      'service.version': process.env.VERCEL_DEPLOYMENT_ID || 'unavailable',
      env: process.env.VERCEL_ENV || 'unavailable',
      environment: process.env.VERCEL_ENV || 'unavailable',
      'deployment.environment': process.env.VERCEL_ENV || 'unavailable'
    },
    spanProcessors: [new DatadogSpanProcessor(), 'auto'],
    logRecordProcessor: new DatadogLogProcessor()
  });
}

class DatadogLogProcessor implements LogRecordProcessor {
  forceFlush(): Promise<void> {
    return Promise.resolve();
  }

  shutdown(): Promise<void> {
    return Promise.resolve();
  }

  onEmit(logRecord: LogRecord, context?: Context): void {
    console.log('emitting');

    // Get trace context from the log record
    // const traceId = logRecord.spanContext?.traceId;
    // const spanId = logRecord.spanContext?.spanId;
    // if (traceId && spanId) {
    //   logRecord.attributes = {
    //     ...logRecord.attributes,
    //     dd: {
    //       trace_id: getDatadogTraceId(traceId),
    //       span_id: getDatadogSpanId(spanId)
    //     }
    //   };
    // }
  }
}

My goal is to add dd.trace_id and dd.span_id to the logs. I successfully do this to the spans with my DatadogSpanProcessor. But by your message I see that this is probably not possible with @vercel/otel correct? Is it possible without the otel package? As it seems logs forward to datadog even without installing the otel package.

10

@kaizen-insights-core

It is possible to get traces from OTEL sent to Datadog. In my previous reply you see I have a DatadogSpanProcessor. That implementation is this:

class DatadogSpanProcessor implements SpanProcessor {
  forceFlush(): Promise<void> {
    return Promise.resolve();
  }

  shutdown(): Promise<void> {
    return Promise.resolve();
  }

  onStart(span: Span, _context: Context): void {
    const spanContext = span.spanContext();

    span.setAttribute(
      'dd.trace_id',
      spanContext.traceId ? getDatadogTraceId(spanContext.traceId) : 'unavailable'
    );
    span.setAttribute(
      'dd.span_id',
      spanContext.spanId ? getDatadogSpanId(spanContext.spanId) : 'unavailable'
    );
  }

  onEnd(_span: ReadableSpan): void {
    // No additional processing needed on span end
  }
}

function getDatadogSpanId(otelSpanId: string) {
  return BigInt(`0x${otelSpanId}`).toString();
}

function getDatadogTraceId(otelTraceId: string) {
  const traceIdEnd = otelTraceId.slice(otelTraceId.length / 2);
  return BigInt(`0x${traceIdEnd}`).toString();
}
11

Is there any altnerative ways to get trace ids on Datadog logs? Maybe using Datadog pipeline to remap requestId to the dd.trace_id and dd.span_id?

Or is there some point I can patch the console.* methods so it can customize the log payload?

12

Thank @ethanshea. I was able to get traces sent to Datadog using the the basic configuration.

import { registerOTel } from '@vercel/otel';

export function register() {
    const serviceName = '@corp/name';
    const environment = getEnvironment();
    const version = process.env.VERCEL_GIT_COMMIT_SHA || 'unavailable';

    registerOTel({
        serviceName: serviceName,
        attributes: {
            env: environment,
            'deployment.environment': environment,
            'deployment.environment.name': environment,
            service: serviceName,
            'service.version': version,
            version: version,
        }
    });
}

Datadog now has OTEL endpoints for Metrics and Logs.

Do you know when this will be supported natively by the Vercel Datadog integration?

13

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.