Sampling at the span level

[adriangb]

Please correct me if I'm wrong but it seems to me that not only the built-in samplers but the concept of sampling in general is built around sampling traces. Is there any history of the concept of sampling spans? The use case where I think this makes a lot of sense is something like:

with tracer.start_as_current_span('do work'):
    for chunk in some_stream_with_millions_of_items:
        with tracer.start_as_current_span('process chunk'):
            pass

In this case, it'd be very useful to sample the chunk processing at say 1%. If there are millions of them I don't want/need millions of spans and assuming they're all similar chunks sampling is perfect to get representative statistics of the duration of processing each chunk, etc. But it seems like right now my only choice is to sample the entire trace or none of it. Is this right?

[svrnm]

There have been discussions around this before, take a look at the following:

• Equivalent of "logLevel" for spans #3205
• Overriding Decisions of the Built-in Trace Samplers via a "sample.priority" Like Span Attribute #3725

[adriangb]

Thanks for the links. I believe #3725 is discussing sampling of traces based on span attributes, although the discussion is large so I may be misunderstanding.

#3205 seems interesting, I'll give it a read 😄