OpenTelemetry.Instrumentation.Http 1.13.0

HttpClient and HttpWebRequest instrumentation for OpenTelemetry

This is an Instrumentation Library, which instruments System.Net.Http.HttpClient and System.Net.HttpWebRequest and collects metrics and traces about outgoing HTTP requests.

This component is based on the v1.23 of http semantic conventions. For details on the default set of attributes that are added, checkout Traces and Metrics sections below.

Steps to enable OpenTelemetry.Instrumentation.Http

Step 1: Install Package

Add a reference to the OpenTelemetry.Instrumentation.Http package. Also, add any other instrumentations & exporters you will need.

dotnet add package OpenTelemetry.Instrumentation.Http

Step 2: Enable HTTP Instrumentation at application startup

HTTP instrumentation must be enabled at application startup.

Traces

Starting with .NET 9, trace instrumentation is natively implemented, and the HttpClient library emits attributes defined in the OpenTelemetry Specification. When running on .NET 9+ this instrumentation library will not add/change/override any attributes set by the native instrumentation but it is still required for performing context propagation using the OpenTelemetry SDK and supports additional features not available in runtime (enrichment, filtering, etc.).

The following example demonstrates adding HttpClient instrumentation with the extension method .AddHttpClientInstrumentation() on TracerProviderBuilder to a console application. This example also sets up the OpenTelemetry Console Exporter, which requires adding the package OpenTelemetry.Exporter.Console to the application.

using OpenTelemetry;
using OpenTelemetry.Trace;

public class Program
{
    public static void Main(string[] args)
    {
        using var tracerProvider = Sdk.CreateTracerProviderBuilder()
            .AddHttpClientInstrumentation()
            .AddConsoleExporter()
            .Build();
    }
}

Following list of attributes are added by default on activity. See http-spans for more details about each individual attribute:

• error.type
• http.request.method
• http.request.method_original
• http.response.status_code
• network.protocol.version
• server.address
• server.port
• url.full - By default, the values in the query component of the url are replaced with the text Redacted. For example, ?key1=value1&key2=value2 becomes ?key1=Redacted&key2=Redacted. You can disable this redaction by setting the environment variable OTEL_DOTNET_EXPERIMENTAL_HTTPCLIENT_DISABLE_URL_QUERY_REDACTION to true.

Enrich Api can be used if any additional attributes are required on activity.

Metrics

The following example demonstrates adding HttpClient instrumentation with the extension method .AddHttpClientInstrumentation() on MeterProviderBuilder to a console application. This example also sets up the OpenTelemetry Console Exporter, which requires adding the package OpenTelemetry.Exporter.Console to the application.

using OpenTelemetry;
using OpenTelemetry.Metrics;

public class Program
{
    public static void Main(string[] args)
    {
        using var meterProvider = Sdk.CreateMeterProviderBuilder()
            .AddHttpClientInstrumentation()
            .AddConsoleExporter()
            .Build();
    }
}

Refer to this example to see how to enable this instrumentation in an ASP.NET Core application.

Refer to this example to see how to enable this instrumentation in an ASP.NET application.

Following list of attributes are added by default on http.client.request.duration metric. See http-metrics for more details about each individual attribute. .NET8.0 and above supports additional metrics, see list of metrics produced for more details.

• error.type
• http.request.method
• http.response.status_code
• network.protocol.version
• server.address
• server.port
• url.scheme

List of metrics produced

When the application targets NETFRAMEWORK, .NET6.0 or .NET7.0, the instrumentation emits the following metric:

Starting from .NET8.0, metrics instrumentation is natively implemented, and the HttpClient library has incorporated support for built-in metrics following the OpenTelemetry semantic conventions. The library includes additional metrics beyond those defined in the specification, covering additional scenarios for HttpClient users. When the application targets .NET8.0 and newer versions, the instrumentation library automatically enables all built-in metrics by default.

Note that the AddHttpClientInstrumentation() extension simplifies the process of enabling all built-in metrics via a single line of code. Alternatively, for more granular control over emitted metrics, you can utilize the AddMeter() extension on MeterProviderBuilder for meters listed in built-in-metrics-system-net. Using AddMeter() for metrics activation eliminates the need to take dependency on the instrumentation library package and calling AddHttpClientInstrumentation().

If you utilize AddHttpClientInstrumentation() and wish to exclude unnecessary metrics, you can utilize Views to achieve this.

Advanced configuration

Tracing

This instrumentation can be configured to change the default behavior by using HttpClientTraceInstrumentationOptions. It is important to note that there are differences between .NET Framework and newer .NET/.NET Core runtimes which govern what options are used. On .NET Framework, HttpClient uses the HttpWebRequest API. On .NET & .NET Core, HttpWebRequest uses the HttpClient API. As such, depending on the runtime, only one half of the "filter" & "enrich" options are used.

.NET & .NET Core

Filter HttpClient API

This instrumentation by default collects all the outgoing HTTP requests. It allows filtering of requests by using the FilterHttpRequestMessage function option. This defines the condition for allowable requests. The filter function receives the request object (HttpRequestMessage) representing the outgoing request and does not collect telemetry about the request if the filter function returns false or throws an exception.

The following code snippet shows how to use FilterHttpRequestMessage to only allow GET requests.

using var tracerProvider = Sdk.CreateTracerProviderBuilder()
    .AddHttpClientInstrumentation(
        // Note: Only called on .NET & .NET Core runtimes.
        (options) => options.FilterHttpRequestMessage =
            (httpRequestMessage) =>
            {
                // Example: Only collect telemetry about HTTP GET requests.
                return httpRequestMessage.Method.Equals(HttpMethod.Get);
            })
    .AddConsoleExporter()
    .Build();

It is important to note that this FilterHttpRequestMessage option is specific to this instrumentation. OpenTelemetry has a concept of a Sampler, and the FilterHttpRequestMessage option does the filtering after the Sampler is invoked.

Enrich HttpClient API

This instrumentation library provides options that can be used to enrich the activity with additional information. These actions are called only when activity.IsAllDataRequested is true. It contains the activity itself (which can be enriched) and the actual raw object.

HttpClientTraceInstrumentationOptions provides 3 enrich options: EnrichWithHttpRequestMessage, EnrichWithHttpResponseMessage and EnrichWithException. These are based on the raw object that is passed in to the action to enrich the activity.

Example:

using System.Net.Http;

var tracerProvider = Sdk.CreateTracerProviderBuilder()
    .AddHttpClientInstrumentation((options) =>
    {
        // Note: Only called on .NET & .NET Core runtimes.
        options.EnrichWithHttpRequestMessage = (activity, httpRequestMessage) =>
        {
            activity.SetTag("requestVersion", httpRequestMessage.Version);
        };
        // Note: Only called on .NET & .NET Core runtimes.
        options.EnrichWithHttpResponseMessage = (activity, httpResponseMessage) =>
        {
            activity.SetTag("responseVersion", httpResponseMessage.Version);
        };
        // Note: Called for all runtimes.
        options.EnrichWithException = (activity, exception) =>
        {
            activity.SetTag("stackTrace", exception.StackTrace);
        };
    })
    .Build();

.NET Framework

Filter HttpWebRequest API

This instrumentation by default collects all the outgoing HTTP requests. It allows filtering of requests by using the FilterHttpWebRequest function option. This defines the condition for allowable requests. The filter function receives the request object (HttpWebRequest) representing the outgoing request and does not collect telemetry about the request if the filter function returns false or throws an exception.

The following code snippet shows how to use FilterHttpWebRequest to only allow GET requests.

using var tracerProvider = Sdk.CreateTracerProviderBuilder()
    .AddHttpClientInstrumentation(
        // Note: Only called on .NET Framework.
        (options) => options.FilterHttpWebRequest =
            (httpWebRequest) =>
            {
                // Example: Only collect telemetry about HTTP GET requests.
                return httpWebRequest.Method.Equals(HttpMethod.Get.Method);
            })
    .AddConsoleExporter()
    .Build();

It is important to note that this FilterHttpWebRequest option is specific to this instrumentation. OpenTelemetry has a concept of a Sampler, and the FilterHttpWebRequest option does the filtering after the Sampler is invoked.

Enrich HttpWebRequest API

This instrumentation library provides options that can be used to enrich the activity with additional information. These actions are called only when activity.IsAllDataRequested is true. It contains the activity itself (which can be enriched) and the actual raw object.

HttpClientTraceInstrumentationOptions provides 3 enrich options: EnrichWithHttpWebRequest, EnrichWithHttpWebResponse and EnrichWithException. These are based on the raw object that is passed in to the action to enrich the activity.

Example:

using System.Net;

var tracerProvider = Sdk.CreateTracerProviderBuilder()
    .AddHttpClientInstrumentation((options) =>
    {
        // Note: Only called on .NET Framework.
        options.EnrichWithHttpWebRequest = (activity, httpWebRequest) =>
        {
            activity.SetTag("requestVersion", httpWebRequest.Version);
        };
        // Note: Only called on .NET Framework.
        options.EnrichWithHttpWebResponse = (activity, httpWebResponse) =>
        {
            activity.SetTag("responseVersion", httpWebResponse.Version);
        };
        // Note: Called for all runtimes.
        options.EnrichWithException = (activity, exception) =>
        {
            activity.SetTag("stackTrace", exception.StackTrace);
        };
    })
    .Build();

Processor, is the general extensibility point to add additional properties to any activity. The Enrich option is specific to this instrumentation, and is provided to get access to raw request, response, and exception objects.

When overriding the default settings provided by instrumentation or adding additional telemetry, it is important to consider the sequence of callbacks.

It is generally recommended to use EnrichWithHttpResponseMessage or EnrichWithHttpWebResponse for any activity enrichment that does not require access to exceptions or request object in case of .NET Framework, as the instrumentation library populates all telemetry following the OTel specification before this callback. The following is the sequence in which these callbacks are executed:

1. Processor OnStart
2. EnrichWithHttpRequestMessage (.NET) / EnrichWithHttpWebRequest (.NET Framework)
3. EnrichWithException both
4. EnrichWithHttpResponseMessage (.NET) / EnrichWithHttpWebResponse (.NET Framework)
5. Processor OnEnd

As an example, if you need to override the default DisplayName set by the library you can do so as follows:

.AddHttpClientInstrumentation(o =>
{
  o.EnrichWithHttpResponseMessage = (activity, response) =>
  {
      // .NET only, access request object if needed.
      // response.RequestMessage
      activity.DisplayName = "CustomDisplayName";

      // Overrides the value
      activity.SetTag("url.full", "CustomUrl");

      // Removes the tag
      activity.SetTag("network.protocol.version", null);
  };
});

RecordException

This instrumentation automatically sets Activity Status to Error if the Http StatusCode is >= 400. Additionally, RecordException feature may be turned on, to store the exception to the Activity itself as ActivityEvent.

Activity duration and http.client.request.duration metric calculation

Activity.Duration and http.client.request.duration values represents the time the underlying client handler takes to complete the request. Completing the request includes the time up to reading response headers from the network stream. It doesn't include the time spent reading the response body.

Troubleshooting

This component uses an EventSource with the name "OpenTelemetry-Instrumentation-Http" for its internal logging. Please refer to SDK troubleshooting for instructions on seeing these internal logs.

References

• OpenTelemetry Project