Reading project properties and items from a logger

Until MSBuild 16.10, project properties and items were mostly available from ProjectStartedEventArgs.Properties and Items respectively.

However sometimes the Properties and Items on ProjectStartedEventArgs could be null, so it is important to check these for null before accessing them. The reason they could be null is if that particular project built in a different process, the properties and items were not sent across process boundaries. For an 8-process /m build only ~12% of all projects actually had properties and items available.

An additional problem is that projects are built multiple times (for multiple targets and global properties), so you could end up with numerous ProjectStartedEventArgs all sharing the same duplicated properties and items.

To fix this, in MSBuild 16.10 a new mode was introduced where properties and items are instead always available on ProjectEvaluationFinishedEventArgs, even if the project evaluated in a different process. There is also no duplication because for each project you can look up its evaluation by the args.BuildEventContext.EvaluationId, so many builds of the same project point back to the same evaluation.

This is the PR that introduced this new functionality: https://github.com/dotnet/msbuild/issues/6287

To opt in to the new behavior, a logger needs to declare that it supports it: https://github.com/dotnet/msbuild/blob/f3d77bee4368de1f8c2d0819e4d85e22cba2dbda/src/Build/Logging/BinaryLogger/BinaryLogger.cs#L114 https://github.com/dotnet/msbuild/blob/f3d77bee4368de1f8c2d0819e4d85e22cba2dbda/src/Build/Logging/BinaryLogger/BinaryLogger.cs#L148-L151

If Traits are not available for the logger source code, you can either read the environment variable here: https://github.com/dotnet/msbuild/blob/f3d77bee4368de1f8c2d0819e4d85e22cba2dbda/src/Shared/Traits.cs#L187-L191 or just always set it to true.

To force opt into the new (better) behavior, you can set the MSBUILDLOGPROPERTIESANDITEMSAFTEREVALUATION environment variable.

To summarize:

1. try cast to IEventSource4 and call eventSource4.IncludeEvaluationPropertiesAndItems(); if available
2. Subscribe to both ProjectStarted and ProjectEvaluationFinished (available from StatusEventRaised: https://github.com/dotnet/msbuild/blob/f3d77bee4368de1f8c2d0819e4d85e22cba2dbda/src/Framework/IEventSource.cs#L146), and check Properties and Items on both for null, and iterate if not null. A logger needs to handle Properties and Items being present on either ProjectStartedEventArgs or ProjectEvaluationFinishedEventArgs, and either one can be null depending on which mode MSBuild is operating in.
3. To locate an evaluation for a ProjectStarted event, build a dictionary of all evaluations by id (by listening to all ProjectEvaluationFinished) and then look up the evaluation from ProjectStartedEventArgs.BuildEventContext.EvaluationId.

The benefits of properties and items being available on ProjectEvaluationFinished are that they will always be available even for projects built in another process, and there will be less duplication and data, which makes builds faster and binary logs smaller.

There's an effort to move all the loggers to the new behavior. The way MSBuild 16.10 initially shipped, as long as any logger opts in, the behavior is turned on. However this broke some legacy loggers that haven't been updated yet. Here's an example issue: https://github.com/dotnet/msbuild/issues/6498

We're changing it so the old behavior is preserved unless all loggers opt in.

This is the PR: https://github.com/dotnet/msbuild/pull/6520

There's a latest PR in progress (https://github.com/dotnet/msbuild/pull/10243) that flips the default to on if there are both loggers that support IEventSource4 and loggers that don't support it. Currently even if there's a single logger that doesn't support IEventSource4 the legacy behavior is used.