Method of controlling attributes on signals other than resources

[thompson-tomo]

With the current schema model, we have a way in which we can include/exclude attributes from resources. We do not however have a defined approach for achieving this with spans or events.

To solve this, I can see a couple of options:

• Replicate the solution from resources ie included/excluded properties to each of the applicable signals. This allows full granular control over the attributes with differences possible between resource detectors & spans.
• Move the current properties from being under resources to be at the top level so that they apply to all signals.
• Hybrid of the 2 previous solutions. Which means adding the properties to the top level and all applicable signals. The definition on the signal type takes precedence over that from the top level.

[jack-berg]

We limit the scope of the schema to model concepts which are part of the spec: https://github.com/open-telemetry/opentelemetry-configuration/blob/main/CONTRIBUTING.md#what-properties-are-part-of-schema

What you're describing sounds like a span processor / log record processor / measurement processor which filters attributes in a configurable way. Its possible to build custom processors to this effect, but for us to include them in the schema, they would need to be added as "built-in" processors in the specification (i.e. like the batch span processor is built-in).

[thompson-tomo]

The use case which I am thinking of is not quite a processor but is more about the instrumentation. Let me use an example which will hopefully help.

I have a library which instruments db calls, in particular to a MySQL db. As such it is following https://opentelemetry.io/docs/specs/semconv/database/mysql/#spans. That spec defines attributes such as db.response.returned_rows as opt-in hence I am looking for a way to opt-in including it in the signals.

Also looking for the opposite, as db.query.text is only recommended I don't see the need to be capturing it hence want to be able to exclude it.

This exclusion might be done by a processor but it could be done by the instrumentation implementation directly.

[jack-berg]

Ah I see. Yes configuring instrumentations to collect opt-in attributes is indeed a different type of thing. Can't configure the SDK to collect something the instrumentation never records!

The .instrumentation/development.general type is the place for things like this.

When I initially contributed it, I added schema properties for concepts which are indicated to be configurable in semantic-conventions. But we really need better cooperation between semantic-conventions and opentelemetry-configuration to see this vision through. An ideal world might look like:

• semantic-conventions publishes a structured representation (maybe JSON schema) of which things should be configurable for instrumentations
• opentelemetry-configuration consumes that data from semantic-conventions, and automatically maps it to appropriate types / properites in .instrumentation/development.
• Instrumentation accesses the configuration at initialization time, and leverages these well-known configuration options to automatically do things like record opt-in properties

[thompson-tomo]

Agree it would be great to have closer collaboration between the different components.

Looking at your suggestion it is a nice idea however I feel it might be too difficult to achieve in practise especially with multi registry solutions.

What I am thinking is:

• We replicate the resource detector approach to include/exclude attributes on the other signal types. This way it is quick and easy to include an attribute on all spans but not have it added to metrics.
• We could also have the same properties on the base definition of an instrumentation which would override those from the signal type.

An alternative approach to the exclude/include properties which I like more would be to have something like:

  detection/development:
    # Configure attributes provided by resource detectors.
    attributes:
      - key:
        # used for templated attributes ie http.request.header
        basekey:
        # optional user defined value to use for the attribute
        value:
        # optional boolean option to control if the attribute is included
        include:

In terms of collaboration with semconv, I think it would be useful if we could have a way to take the telemetry definitions resolved/emitted from weaver and validate a config file. A useful first step would be to be able to indicate which attributes have a user definable value.