OTLP Integration

Integrate OTLP gRPC client and Listener server, check handler assembly, data conversion and reliability limitations.

OTLP is Telemetry Data Transfer Protocol. A batch can contain multiple Resources, each resource has multiple Scopes, and includes metric data points, logs, or traces. Applications that compress all layers into a list can easily lose source, attribute, or data point semantics.

Client Package Responsibilities

Zongsoft.Diagnostics.Protocols.Client provides the generated message type and gRPC client stub. It does not automatically instrument, ingest, batch, retry, or build SDK export pipelines. Ordinary applications start from diagnostics configuration; only direct protocol calls need to construct requests by themselves.

Discussions There is no use case for directly constructing OTLP Export requests, so a fictitious client for empty requests is no longer provided. The sender should use the existing diagnostics export configuration; when direct manipulation of the protocol is required, check the services and DTOs generated by the client project.

The business exporter also populates resources, scopes, and data points, handles responses and partial successes, and controls queues, batch sizes, and retry scopes. Generative API versions do not equal line protocol stability, and it is not recommended to directly use these types as business long-term storage models.

Server-side Assembly

Deploy gRPC and protocol servers in the plugin web host:

TelemetryServer.deploy(fragment)

[plugins zongsoft web grpc]
nuget:Zongsoft.Web.Grpc

[plugins zongsoft diagnostics protocols server]
nuget:Zongsoft.Diagnostics.Protocols.Server

Listener's Logs, Metrics, and Traces services are mapped through the gRPC label. The HTTP/2 endpoint of http://*:4317 is declared as a package option. Actual deployment should limit the address, transport security and sender access; ordinary HTTP/1 JSON requests cannot verify this gRPC entry.

Register Handler

The receiving service converts the protocol message into a skeleton model and dispatches it to the handler collection. The following is the actual plugin registration used in the framework samples:

Source: framework/Zongsoft.Diagnostics/protocols/server/samples/Zongsoft.Diagnostics.Protocols.Server.Samples.plugin (excerpt; see source for context).

Zongsoft.Diagnostics.Protocols.Server.Samples.plugin

<extension path="/Workbench/Diagnostics/Telemetry/Listener/Metrics">
	<object name="MetricHandler" type="Zongsoft.Diagnostics.Protocols.Server.Samples.MetricHandler, Zongsoft.Diagnostics.Protocols.Server.Samples" />
</extension>

Source: framework/Zongsoft.Diagnostics/protocols/server/samples/MetricHandler.cs (excerpt; see source for context).

MetricHandler.cs

protected override ValueTask OnHandleAsync(IEnumerable<Telemetry.Metrics.Meter> meters, Parameters parameters, CancellationToken cancellation)
{
	foreach(var meter in meters)
		Terminal.WriteLine(CommandOutletDumper.Dump(meter));

	return ValueTask.CompletedTask;
}

The type needs to be provided by the application, and the assembly and server plugin dependencies are declared in the manifest. Server sample uses a typed handler to print metrics; printing is for observation only and does not provide a database, persistent queue, or failure retries.

Current Conversion and Reliability Boundaries

🚨 The current implementation does not work as a proven lossless general-purpose OTLP sink: metric processing re-establishes the collection in the resource loop, and the final dispatch cannot retain all metrics of the previous resource; Gauge is also projected as a framework Counter related model. These behaviors should be verified and handled before integrating multiple resource batches or relying on specific metric semantics.

Timestamps are converted from Unix nanoseconds to milliseconds with a loss of precision. Handlers are executed through parallel dispatch, individual handler exceptions are logged and isolated, and the response does not report these failures one by one. Without a handler, it is also impossible to infer from the successful response that the data was consumed.

When reliable reception is required, it should be clearly defined when to queue or place a request, how to handle overload, and how failures are discovered by the sender or operation and maintenance. Current protocol responses and handler failure semantics must be incorporated into the design; simply catching and printing exceptions in the handler is not enough to ensure no data is lost.

Verification Order

First verify the HTTP/2 endpoint and service label, then mount the record handler, send a synthetic batch containing a known number of resources, scopes, and data points, and compare the received results. It then checks for empty batches, handler failures, cancellations, and target storage failures before connecting to the real telemetry source.

Source code basis: Dispatch and time conversion, Indicator conversion, Client generated project.