Use the decisions of one tail sampling processor to route traces to separate exporters #44790
Description
Component(s)
processor/tailsampling
Is your feature request related to a problem? Please describe.
I have a use case where I have two separate backends (exporters) where I need to send traces to.
There is one backend that should receive all traces after a fairly standard set of sampling policies:
- all HTTP traces for a specific route
- all error traces
- 1 % of other traces
The other backend should only receive traces for a specific attribute that is set somewhere in the trace and only available in one span.
The current solution is to have two tail sampling processors with two sets of policies but this is rather heavy on memory since both have their own separate buffer.
We also need to set decision_wait to 60 seconds since some traces when running in a timeout can take up to that amount. This makes having two tail sampling processors even worse.
Describe the solution you'd like
The ideal solution would be to have a single tail sampling processor which evaluates its policies and ideally mark the spans with an attribute indicating which policy has matched.
That way we could use a routing connector afterwards to route the sampled traces to the correct exporter.
Describe alternatives you've considered
We tried using a single tail sampling processor with a routing connector already but since the attribute we use to sample the traces for the second exporter is only available in some spans of a trace, only these spans are being routed to the correct exporter.
We also tried a forwarding connector and use a filter processor afterwards but also here only the spans containing the attribute are being kept.
Additional context
No response
Tip
React with 👍 to help prioritize this issue. Please use comments to provide useful context, avoiding +1 or me too, to help us triage it. Learn more here.