Line number and source code (via pdb) for maui android

[tranb3r]

Hi,

I'm uploading pdb files to Sentry for a maui android app.
I've managed to get line number and source code in error reports.
However I'm having a couple of issues.

First, it does not seem to work when I enable LLVM or full Trimming.

<TrimMode>full</TrimMode>
<EnableLLVM>true</EnableLLVM>

Second, it seems to work only for the code of the app, not for external nuget packages, even if I manually upload the nuget's pdb.

Are these issues known limitations? Am I missing something?

Thanks

[jamescrosswell]

Hey @tranb3r,

First, it does not seem to work when I enable LLVM or full Trimming.

When enabling LLVM, I'm not seeing any debug information files that could be used to symbolicate native code... I've started this discussion in the dotnet/maui repo to see if I can get a bit more information on how we could do this:

• Debug information (to map to source) when EnableLLVM is true? dotnet/maui#23932

If you can upvote that discussion, it might help us get an answer more quickly.

[tranb3r]

What about Trimming?

[jamescrosswell]

What about Trimming?

I tested this with the Sentry.Samples.Maui app and I am seeing source context for errors even when publishing the app fully trimmed.

At first, I was seeing "old source code" associated with the exception. I think that's because I was running this locally so the source hadn't yet been committed and pushed to github where it would be available via source link.

However, if I set <EmbedAllSources>true</EmbedAllSources> in the csproj file though, then exceptions are shown in fully trimmed apps, including the correct source context, even for source code that I haven't yet committed and pushed to a repository that Sentry has access to.

[tranb3r]

Indeed, it works with trimming on the sample app.
However, it only works when the error is happening in the top level assembly.
If you add a class library and trigger an error from the class library itself, it does not anymore (although it works when disabling trimming).

[jamescrosswell]

Second, it seems to work only for the code of the app, not for external nuget packages, even if I manually upload the nuget's pdb.

I think this should work. The Sentry Docs talk about various built in symbol servers that Sentry uses

@bitsandfoxes @bruno-garcia can you think of any caveats to this or why/when this might not work?

[tranb3r]

I've done more tests for nugets.

It seems to work sometimes, but it depends on the nuget:

• it works for Flurl.Http 4.0.2 (symbols in snupkg, no PublicKeyToken)
• it works for SecureHttpClient 2.2.6 (symbols in snupkg, no PublicKeyToken)
• it does not work for Sentry 4.9.0 (although it also has symbols in snupkg)
• it does not work for CommunityToolkit.Mvvm 8.2.2 (symbols alongside the dll)
• it does not work for Microsoft.Extensions.Http 9.0.0

Is it just a coincidence that both working nugets in this list do not have a PublicKeyToken ?

[jamescrosswell]

Hm, all of these seem to have source context using Source link (although Community Toolkit doesn't have a Symbol Server):

• Flurl.Http: Valid with Symbol Server
• SecureHttpClient: Valid with Symbol Server
• Sentry: Valid with Symbol Server
• CommunityToolkit.Mvvm: Valid
• Microsoft.Extensions.Http: Valid with Symbol Server

Maybe as @vaind suggested, we'd need a reproducible example for this one as there are quite a few moving parts with a build like this.

[vaind]

EnableLLVM

Looks like Android now has AOT compilation. I don't think we support that yet as it requires custom code for other platforms

[vaind]

but it's fair to say that I may not remember this correctly, since there's so many ways and combinations how you can build stuff...

We've got some handling on-device with AssemblyReader but that doesn't seem to help in this case.

we'll definitely need a reproducible example and look into this closely, frame by frame, to determine if the info is available there and whether it allows us to do symbolication

[tranb3r]

@jamescrosswell

I've created a sample for my issues, based on the sample app for maui.
It's available here: https://github.com/tranb3r/sentry-demos-dotnet-maui/tree/tests-pdb
I hope it will help you figure out a solution for my issues.

Trimming issue

No source context for code in class library, when trimming is enabled

Enable trimming in DotNetMaui.csproj: <TrimMode>full</TrimMode>

No issue for code in class library, without trimming

No issue for code in main assembly, with or without trimming

Nuget issue : source context is not always available, it depends on the nuget package

Remark about PDB upload

EmbedAllSources=false is not always working for me.
I need to disable SourceLink for PDB upload to work properly.

<EnableSourceLink>false</EnableSourceLink>
<PublishRepositoryUrl>false</PublishRepositoryUrl>

Make sure you get the following files uploaded or the tests won't work:

EnableLLVM

When enabling LLVM, I don't see the same files in the uploaded PDB page (2 files instead of 4).
Maybe this explains why I can't get Source Context in errors reported by Sentry?

[jamescrosswell]

Hm, that sounds like a lot of secret handshakes and like the process is far too fragile to be usable/useful for most SDK users.

manually upload files with sentry-cli (not the msbuild task), with debug verbosity.

@tranb3r what command are you running, exactly, to perform the uploads? Is it the same as what the MSBuild task runs (only with debug verbosity) or are you running a different command for some reason?

[jamescrosswell]

OK, I finally managed to get the source context to show for a Mac Catalyst app by setting:

  <MtouchLink>None</MtouchLink>

... which, obviously, you woudn't want to do in a production application since it makes the binary huge.

Not to mention:

  <TrimMode>partial</TrimMode>
  <Optimize>false</Optimize>

All in all, I had the feeling I was making random changes and getting random results. Things get uploaded to Sentry. Sometimes it works and sometimes it doesn't but it's hard to know why, either way. It's very much a black box.

It would be great if there was some way to get "logs" from Sentry detailing how it succeeds or fails to resolve symbols and source context for stack traces.

[jamescrosswell]

@tranb3r I've created a features request in the Sentry repo for this:

• Expose logs for symbolication and source context resolution sentry#75705

[tranb3r]

that sounds like a lot of secret handshakes and like the process is far too fragile to be usable/useful for most SDK users.

It's only necessary when iterating on a developer machine, like we've done.
In a CI context, it would be much simpler, since the assemblies version would be increased (so there would be no caching).

@tranb3r what command are you running, exactly, to perform the uploads? Is it the same as what the MSBuild task runs (only with debug verbosity) or are you running a different command for some reason?

The reason why I've chosen to perform the uploads with scentry-cli instead of msbuild is because when you include the Sentry.Maui package in a class library, the Sentry target is executed both when building the class library and when building the application, resulting in the pdb upload being done twice.
First upload would be before trimming, second upload after trimming... I imagine this could cause issues with Sentry server's caching.
So I've decided not to trust the msbuild target and run the command after the build. But the command should be the same.
sentry-cli-Windows-x86_64.exe debug-files upload -o XXX -p YYY --include-sources --log-level debug [PATH]

All in all, I had the feeling I was making random changes and getting random results.

I had this feeling too, until I figured out the steps I've described, that work for me.

It's very much a black box. It would be great if there was some way to get "logs" from Sentry detailing how it succeeds or fails to resolve symbols and source context for stack traces.

This is a very good idea, I agree it would help a lot when trying to understand what's going on.

[bitsandfoxes]

Just to have mentioned it and to help with further debugging: The actual event-json contains further information regarding the symbolication status.
i.e. source link status, debug symbol ID and

"data": {
    "symbolicator_status": "symbolicated"
},