3rd party plugin boiler plates attempts to import '../../../../src/core/server' which doesn't exist on production installations of Kibana.

[btychon]

Kibana version:
8.10.4

Elasticsearch version:
8.10.4

Server OS version:
Ubuntu 22

Browser version:
Evergreen

Browser OS version:
Windows or Linux

Original install method (e.g. download page, yum, from source, etc.):
Download and install Kibana. Then install the 3rd party boiler plate plugin.

Describe the bug:
The boiler plate generator node /kibana/scripts/generate_plugin.js produces a file /kibana/plugins/testplugin/server/index.ts

That file contains an import statement:
import { PluginInitializerContext } from '../../../src/core/server';

When running Kibana in development mode the import statement works because the dependency exists at /kibana/src/core/server. However if the plugin is installed on a production version of Kibana (offline install or within a container) the import statement fails because that file doesn't exist.

Steps to reproduce:

1. Generate a boiler plate Kibana plugin and download ZIP.
2. Install the production version of Kibana by downloading (don't use yarn start). Install the plugin ZIP artifact.
3. Try running Kibana and visit the "Hello World" boiler plate plugin.

Expected behavior:
The import statement should work whether the plugin is running Kibana in Development or Production mode. There is no file /kibana/src/core/server on the latter.

Either:

1. Fix the boiler plate so the import statement works. Maybe it needs to reference another path/file?
2. Include the file /kibana/srce/core/server on the offline version of Kibana. It used to exist in earlier releases.

Screenshots (if relevant):

Errors in browser console (if relevant):

Provide logs and/or server output (if relevant):
[FATAL][root] Reason: Cannot find module '../../../src/core/server'

Any additional context:

[elasticmachine]

Pinging @elastic/kibana-operations (Team:Operations)

[mistic]

Thanks for reporting this. We have moved into isolated packages and the import import { PluginInitializerContext } from '../../../src/core/server'; should be import { PluginInitializerContext } from '@kbn/core/server'; like in the examples https://github.com/elastic/kibana/blob/main/examples/content_management_examples/server/index.ts#L9

[btychon]

I replaced the relative import statements with your name-spaced NPM module. Everything works great in development mode and the API's are working as expected, such as determining "dark mode".

yarn build succeeds without any errors and places the artifact in the build directory. Our company's production plugin ZIP used to be around 7MB and now it's a little over 2MB. I'm curious how you were able to shave that many bytes off of the build? Anyway, here's the log from running the build step.

When I download and explore the ZIP file I can see that both front-end and server package.json dependencies are all in the node_modules folder. The /public/ directory is missing but maybe that's intentional because I can see the JS bundles in the /target/public/ directory instead. The 404 is difficult to identify through the browser's Network console because Kibana proxies requests through a very large integer prefix.

The problem is that when we try installing the build artifact for the boilerplate plugin into a "Kibana production installation" we get a 404 error for the JS bundle immediately after login (again). This is similar to the problem I was getting in my "Kibana development env" before I knew to run yarn dev --watch in the plugin directory.

Can you please try creating a boilerplate plugin and install it into a "Kibana production installation" to confirm the plugin shows in the left nav after logging in and you can navigate to its home page where you should see a "Congratulations" notice?

[mistic]

@btychon you're right. Looks like our brotli compression is broken when creating an external plugin for production environment. I already know what the problem is and will report back once it is fixed. For now, as a workaround, please delete the generated .br files from your target/public folder created as part of your plugin build.

[btychon]

I unzipped the artifact and deleted everything in /kibana/tychon/target/public/*.br. Then I re-zipped and tried installing however the plugin isn't showing up.

[btychon]

There's something else weird going on related to the file sizes which must offer a clue beyond deleting .br files. Our last plugin artifact from ELK 7.13.1 is 16MB and the new artifact from ELK 8.10.4 is only 2MB.

I expanded the ZIP to see what's causing the size difference. The /target/build/ folder should have all of our React and front-end source code but it's showing a deficit of 5MB in ELK 8.10.4. If I remove the .br files the size difference is exacerbated a tiny bit which is going in the wrong direction.

If I look inside the /target/public/ directory it seems that tychon.chunk.1.js contains the entire application code for ELK 7.13.1. However, in ELK 8.10.4, curiously there are a number of other chunks except for tychon.chunk.1.js, which brings to mind the possibility of a fencepost error

There is also a huge discrepancy in the node_modules folder before it's compressed into a ZIP (25MB difference).

Within node_modules, the 7.13.1 folder appears to have a bunch of "peer dependencies". However, there's no peerDependencies key in our package.json. These are the two versions of node_modules after unpacking the ZIP artifact. When I do a yarn install from the plugin on either 7.13.1 or 8.10.4 the number of files are essentially identical, about 225 entries. Clearly webpack is acting differently, or more aggressive, to reduce the final size by tree shaking or something. Whatever it's doing must the source of another problem.

[mistic]

@btychon we had indeed a problem with decompressing the generated brotli assets for external built plugins which will be fixed as soon as #170203 gets merged.

I've made some tests with some example plugins and things seems to be working okay now.

Regarding what you're mentioning, I'm not aware we have changed how we install node modules in the generated archive upon the external plugin build (the sourcecode for that matter is the same in between version 7.13 and 8.10).

I would advise you to check if the dependencies in the package.json (for production) are the same in both those examples you're sharing as well as the yarn version you're using there.

I'm going to close this issue as the original problem is not a problem anymore. If you find further problems please lets open another issue.

Thank you very much for helping us catching those problems 🙇

[btychon]

BTW... you guys did some magic with the plugin's artifact size. It went from 16MB to 2MB.

1. You removed 4 MP from the JS junks because 7.13.1 had a bunch of minified parts of the codebase commented out for some reason.
2. The WebPack upgrade and "tree shaking" removed about another 10 MB from node_modules.