Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

use SPI to manage components #1412

Open
wants to merge 57 commits into
base: 2.x
Choose a base branch
from

Conversation

brettmc
Copy link
Collaborator

@brettmc brettmc commented Oct 18, 2024

To resolve our long-standing race conditions stemming from using composer's autoload->files to register SDK components at runtime, this changes things so that:

  • components can be registered in various composer.json files, under extra.spi
  • existing _register.php files manually register through SPI, but will be removed from autoload_files if SPI is allowed to run
  • SDK fetches factories from SPI instead of maintaining static arrays

If we move ahead with this approach, a follow-up PR could remove our various late-binding providers.

Copy link

netlify bot commented Oct 18, 2024

Deploy Preview for opentelemetry-php canceled.

Name Link
🔨 Latest commit 5ffe83b
🔍 Latest deploy log https://app.netlify.com/sites/opentelemetry-php/deploys/671f1d38fe1753000839d348

Copy link

codecov bot commented Oct 18, 2024

Codecov Report

Attention: Patch coverage is 44.97817% with 126 lines in your changes missing coverage. Please review.

Project coverage is 72.07%. Comparing base (a19455c) to head (67891d0).
Report is 3 commits behind head on 2.x.

Files with missing lines Patch % Lines
src/Config/SDK/_register.php 0.00% 26 Missing ⚠️
src/SDK/_register.php 0.00% 17 Missing ⚠️
src/Contrib/Otlp/_register.php 0.00% 6 Missing ⚠️
src/SDK/Propagation/B3MultiPropagatorFactory.php 0.00% 6 Missing ⚠️
src/SDK/Propagation/B3PropagatorFactory.php 0.00% 6 Missing ⚠️
src/SDK/Propagation/BaggagePropagatorFactory.php 0.00% 6 Missing ⚠️
.../Propagation/CloudTraceOneWayPropagatorFactory.php 0.00% 6 Missing ⚠️
...rc/SDK/Propagation/CloudTracePropagatorFactory.php 0.00% 6 Missing ⚠️
...SDK/Propagation/JaegerBaggagePropagatorFactory.php 0.00% 6 Missing ⚠️
src/SDK/Propagation/JaegerPropagatorFactory.php 0.00% 6 Missing ⚠️
... and 15 more
Additional details and impacted files

Impacted file tree graph

@@             Coverage Diff              @@
##                2.x    #1412      +/-   ##
============================================
- Coverage     72.31%   72.07%   -0.24%     
- Complexity     2729     2758      +29     
============================================
  Files           401      405       +4     
  Lines          8148     8200      +52     
============================================
+ Hits           5892     5910      +18     
- Misses         2256     2290      +34     
Flag Coverage Δ
8.2 71.91% <44.97%> (-0.22%) ⬇️
8.3 71.87% <44.97%> (-0.26%) ⬇️
8.4 71.89% <44.97%> (-0.38%) ⬇️
8.5 71.98% <44.97%> (-0.21%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
...entProvider/Propagator/TextMapPropagatorJaeger.php 0.00% <ø> (ø)
src/Contrib/Grpc/GrpcTransportFactory.php 73.17% <100.00%> (+1.37%) ⬆️
src/Contrib/Otlp/LogsExporterFactory.php 97.61% <100.00%> (+0.18%) ⬆️
src/Contrib/Otlp/MetricExporterFactory.php 98.03% <100.00%> (+0.12%) ⬆️
src/Contrib/Otlp/SpanExporterFactory.php 97.50% <100.00%> (+0.20%) ⬆️
src/SDK/Common/Export/Http/PsrTransportFactory.php 81.25% <100.00%> (+2.67%) ⬆️
src/SDK/Common/Export/Http/PsrUtils.php 92.64% <100.00%> (ø)
...DK/Common/Export/Stream/StreamTransportFactory.php 91.11% <100.00%> (+0.86%) ⬆️
src/SDK/Common/Services/Loader.php 100.00% <100.00%> (ø)
src/SDK/Logs/Exporter/ConsoleExporterFactory.php 100.00% <100.00%> (ø)
... and 36 more

... and 14 files with indirect coverage changes


Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a19455c...67891d0. Read the comment docs.

@brettmc brettmc changed the title generate registry from composer autoload-dump ptototype: generate registry from composer autoload-dump Oct 20, 2024
@brettmc brettmc changed the title ptototype: generate registry from composer autoload-dump prototype: generate registry from composer autoload-dump Oct 20, 2024
To resolve our long-standing race conditions stemming from using composer's autoload->files
to registry SDK components at runtime, this changes things so that:
- components are registed in various composer.json files, under the extra.opentelemetry.registry key
- a composer script is registered to write this config out to a JSON file
- the SDK Registry is modified to make manually registering components a no-op (currently behind a flag, OTEL_PHP_EXPERIMENTAL_JSON_REGISTRY),
  and instead configure itself from the generated JSON file

If we move ahead with this approach, a follow-up PR could tidy up the Registry and remove our various late-binding providers.
@brettmc brettmc force-pushed the composer-extra-registry branch from ffb28df to abd9944 Compare October 21, 2024 00:48
@Nevay
Copy link
Contributor

Nevay commented Oct 21, 2024

An alternative idea if we want to avoid duplicating the "load implementations from composer.json" approach:

The main difference between the solution in this PR and an SPI-based approach is the addition of a name/key for each implementation within the composer.json configuration. This information could instead be added as additional method in the interfaces1 to allow loading them using SPI.

Registry is currently used for two different types of implementations:

  • factories used to initialize the SDK from environment variables
  • transport factory implementations, which are also used outside of environment variables configuration

Transport factory implementations

We can extend TransportFactoryInterface with the necessary information to populate the transport factories:

$factories = iterator_to_array(ServiceLoader::load(TransportFactoryServiceInterface::class));
array_multisort(
    array_map(static fn($factory) => $factory->priority(), $factories),
    SORT_DESC,
    $factories,
);
$factoriesByProtocol = [];
foreach ($factories as $factory) {
    $factoriesByProtocol[$factory->protocol()] ??= $factory;
}

self::$transportFactories = $factoriesByProtocol;
interface TransportFactoryServiceInterface extends TransportFactoryInterface {

    public function protocol(): string;
    
    public function priority(): int;
}

Factories to initialize the SDK from environment variables

We could align the implementation with the file-based implementation by adding a new, from the current implementation independent interface2 similar to Configuration\ComponentProvider and deprecate the current registry methods and factory interfaces.
Very basic interface definition:

namespace OpenTelemetry\Config\SDK\Environment;

/**
 * @template T
 */
interface ComponentProvider {

    /**
     * @return T
     */
    public function createPlugin(): mixed;

    public function name(): string;
}

Alternatively we could follow the approach mentioned for transport factory implementations and add a name(/priority) method to the factory interfaces and continue using the Registry.

Footnotes

  1. See also Java ServiceLoader - Designing Services:

    A service should declare as many methods as needed to allow service providers to communicate their domain-specific properties and other quality-of-implementation factors. An application which obtains a service loader for the service may then invoke these methods on each instance of a service provider, in order to choose the best provider for the application.

  2. FWIW I've been using the following Env\Loader interface for env-based configuration, its implementations are loaded via SPI:

    /**
     * @template T
     */
    interface Loader {
    
        /**
         * @return T
         */
        public function load(EnvResolver $env, LoaderRegistry $registry, Context $context): mixed;
    
        public function name(): string;
    }
    

@brettmc
Copy link
Collaborator Author

brettmc commented Oct 22, 2024

updated to use SPI, and implemented the idea of having factories declare their key & priority.

@haad
Copy link

haad commented Oct 23, 2024

  • The Vneno i

Copy link
Contributor

@ChrisLightfootWild ChrisLightfootWild left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is an exciting set of changes!

Just left a few bits for consideration/discussion.

src/SDK/Common/Export/Http/PsrTransportFactory.php Outdated Show resolved Hide resolved
src/SDK/Common/Export/TransportFactoryInterface.php Outdated Show resolved Hide resolved
src/SDK/Logs/LogRecordExporterFactoryInterface.php Outdated Show resolved Hide resolved
src/SDK/Metrics/MetricExporterFactoryInterface.php Outdated Show resolved Hide resolved
src/Contrib/Zipkin/SpanExporterFactory.php Outdated Show resolved Hide resolved
Copy link
Contributor

@xvilo xvilo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One small code-style comment was placed.

I understand that the Registry stuff will not be used when using extra.spi. Is it correct that third-party libraries (in my case, proprietary code/packages within a company) can also add their instrumentation through the SPI way, as shown in composer.json?

It would be nice to pair this with some documentation regardless, e.g. a 'How to add instrumentation into your composer package' section. It would be a good way to demo/reflect/document the preferred way of including AutoInstrumentations.

src/Extension/Propagator/B3/B3MultiPropagatorFactory.php Outdated Show resolved Hide resolved
@brettmc
Copy link
Collaborator Author

brettmc commented Oct 23, 2024

I understand that the Registry stuff will not be used when using extra.spi. Is it correct that third-party libraries (in my case, proprietary code/packages within a company) can also add their instrumentation through the SPI way, as shown in composer.json?

@xvilo That's right. It's still up for debate exactly how registry/SPI should interact, and which one would be the default. The SPI mechanism will definitely allow custom components like we have now.

composer.json Outdated Show resolved Hide resolved
@brettmc brettmc added the breaking A breaking change label Jan 7, 2025
To resolve our long-standing race conditions stemming from using composer's autoload->files
to registry SDK components at runtime, this changes things so that:
- components are registed in various composer.json files, under the extra.opentelemetry.registry key
- a composer script is registered to write this config out to a JSON file
- the SDK Registry is modified to make manually registering components a no-op (currently behind a flag, OTEL_PHP_EXPERIMENTAL_JSON_REGISTRY),
  and instead configure itself from the generated JSON file

If we move ahead with this approach, a follow-up PR could tidy up the Registry and remove our various late-binding providers.
@brettmc brettmc marked this pull request as ready for review January 10, 2025 02:32
@brettmc brettmc requested a review from a team as a code owner January 10, 2025 02:32
@brettmc
Copy link
Collaborator Author

brettmc commented Jan 10, 2025

@open-telemetry/php-approvers @Nevay I think this is ready for review: NB that it targets 2.x branch and has some BC breakage, which I've started to document in docs/upgrading.md

src/SDK/Common/Services/Loader.php Outdated Show resolved Hide resolved
composer.json Outdated Show resolved Hide resolved
src/SDK/Common/Export/Http/PsrUtils.php Outdated Show resolved Hide resolved
src/SDK/Common/Services/Loader.php Outdated Show resolved Hide resolved
src/SDK/Common/Services/Loader.php Outdated Show resolved Hide resolved
src/SDK/Common/Services/Loader.php Outdated Show resolved Hide resolved
tests/Unit/SDK/Common/Services/LoaderTest.php Outdated Show resolved Hide resolved
in most cases, we can use existing _register.php files rather than using
config.spi - this removes some repetition, and means the experience without
SPI plugin enabled should be closer to when it is enabled.
Merge 3x different _register.php's from SDK into one at the top level.
in most cases, we can use existing _register.php files rather than using
config.spi - this removes some repetition, and means the experience without
SPI plugin enabled should be closer to when it is enabled.
Merge 3x different _register.php's from SDK into one at the top level.
per code review feedback
add SpiLoadableInterface, which is used by factories. Not applied to the factories in Context, due to
dependency (Context should not depend on API or SDK)
@brettmc brettmc force-pushed the composer-extra-registry branch from 5fc05ec to 3e80235 Compare January 13, 2025 05:19
this allows moving all SPI config into _register.php files
deptrac.yaml Outdated Show resolved Hide resolved

namespace OpenTelemetry\Context\Propagation;

interface TextMapPropagatorFactoryInterface
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should IMO live in the SDK similar to ComponentProvider. It doesn't make much sense to implement the env-based factory w/o also implementing a corresponding component provider.

(Sidenote: would still prefer if we move ComponentProvider as well as an abstraction for env-based factories into SDK independent package(s))

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved things around in 0571d3b - WDYT?

src/SDK/Common/Services/Loader.php Outdated Show resolved Hide resolved
composer.json Outdated Show resolved Hide resolved
src/API/composer.json Outdated Show resolved Hide resolved
reorganise extension propagator factories into SDK, and add a deptrac rule to allow
SDK->Extension dependency. Add PackageDependency attribute to the SDK-housed
extension propagator factories.
@brettmc brettmc requested review from a team and Nevay January 21, 2025 00:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
breaking A breaking change
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants