Continuous profiling not working with NextJS instrumentation

[austinwoon]

Getting no profiles in the APM section after setting the following configs to tracer on NextJS.

const appTracer = tracer.init({
  runtimeMetrics: true,
  logInjection: true,
  profiling: true,
})

Could I check if this is a bug or I have to enable something else on my end? Thank you.

[austinwoon]

Hey, after some debugging we found that the issue was that prebuilds were not being added in standalone mode in nextjs.

Have created a upstream fix here awaiting merge. Tagging maintainers as this is an actual bug that will prevent nextjs standalone users from using datadog profiling @rochdev @juan-fernandez

[rochdev]

Thank you for the upstream fix! Do you think this could be done in a more generic way? We have more native extensions than just profiling and we have more coming as well. Having to remember to update a list in an external project, while doable, is definitely not ideal.

[austinwoon]

Thank you for the upstream fix! Do you think this could be done in a more generic way? We have more native extensions than just profiling and we have more coming as well. Having to remember to update a list in an external project, while doable, is definitely not ideal.

To be honest, I'm not sure if this is a unique case because vercel has their own tree-shaking logic for standalone mode. Perhaps other bundlers have accounted for not treeshaking modules that require binaries somehow.

For this specific case, the most intuitive DX I could imagine that would not require too many changes on vercel's end would be to have them introduce a feature in the nft module that allows users to mark some modules to not be tree-shaken.

Then, your team could add some specific code snippets for users to include in next.config.js when using standalone mode to build.

This seems to be the best middleground without sprinkling moar AST magic 🧙 into their code haha (if you are interested, can have a look at vercel/nft#392, which was a fix for another module, this involved just hardcoding another way to handle imports which banked on the fact that node-gyp-build was imported in the parent folder)

Hope this makes sense!

[luizmiranda7]

Hey, @austinwoon do you know of any workaround I could apply to my repo since I'm not able to upgrade the Nextjs version from 13.x to 14.x?

[austinwoon]

Hey, @austinwoon do you know of any workaround I could apply to my repo since I'm not able to upgrade the Nextjs version from 13.x to 14.x?

Manually install this package and transfer into to stand alone mode's node modules package. Not ideal but will work.

[Neurostep]

I have a similar issue when running the NextJS v14.2.10 in a standalone mode. When the function _startProfiler is called, I get the following error

Error: No native build was found for runtime=node abi=108 platform=darwinglibc arch=x64

Is there a way to work around this in the current NextJS version and the latest dd-trace-js? Maybe add something to the standalone config? Any help would be appreciated 🙏

/cc @austinwoon Maybe you can help here?

[szegedi]

Hi @Neurostep, Attila from Datadog profiling team here. @austinwoon's comment just above yours includes a very tersely formulated workaround, I'll try to expand on it in in more detail. This workaround is unfortunately still required today as the dependency bump in next.js to pull in the fix from the nft library is only slated for next.js versions 14.3.0 and 15.0.0. 14.3.0 in particular seems to be in a release limbo for months now.

In our own testing with next.js, we noticed that the application version emitted by the bundler in its .next/standalone directory will reference dd-trace and its dependencies (@datadog/pprof included) in its package.json, but the dependencies are not installed. Executing

cd .next/standalone
npm update

will complete the missing dependencies and make the profiler work. There's few more libraries under the @datadog namespace that have native components – there's one used for Node specific metrics gathering, one for our AppSec/IAST library, so it might make sense to do this and make sure the standalone app is fully fleshed out.

If you're concerned that this brings in too many dependencies, you can also try to create an empty sample application, use npm to install dd-trace into it, then copy only node_modules/@datadog/pprof over to .next/standalone directory of your production app. E.g. presuming you use dd-trace 5.22.0, current latest:

mkdir sample-app
cd sample-app
npm init -y
npm install [email protected]
cp -r node_modules/@datadog/pprof $YOUR_APP/.next/standalone/node_modules/@datadog

It does sound from your error message that you already might have a part of the profiler library (supposedly the JS files only) installed, and it's only the native code shared libraries that are missing; those would normally reside in node_modules/@datadog/pprof/prebuilds. You might have luck copying only those over, but it's probably more robust to just copy all of the node_modules/@datadog/pprof.

Hope that helps!

[Neurostep]

Hi @szegedi 👋 Thank you for the thorough explanation, that is very helpful 🙏

I have tried to fix the problem by copying the @datadog/pprof to the standalone directory and it worked 🎉 . One correction though: the cp command needs to be slightly updated as follows:

cp -r node_modules/@datadog/pprof $YOUR_APP/.next/standalone/node_modules/@datadog

Otherwise, we end up having the @datadog/pprof/pprof directory and the prebuilds would be missing

[szegedi]

the cp command needs to be slightly updated as follows:

Thank you! I updated the instructions to reflect that.