How should one programmatically generated shared-data?

[jmuchovej]

I have ~700 files that need to be included in shared-data when building a wheel. Ideally there's a programmatic way to do this. However the only way I've seen this done is how JupyterHub does it, which relies on setup.py.

This is my first time using hatch, so I'm still grokking how to do this. Would the appropriate way to populate shared-data be to write a plugin of some kind?

I see in the docs that one should use the BuilderInterface, but I wonder if I should instead be using the WheelBuilderConfig as a base and modifying the behavior of of the shared_data property?

hatch/backend/src/hatchling/builders/wheel.py

Lines 225 to 251 in 94dd377

	@property
	def shared_data(self):
	if self.__shared_data is None:
	shared_data = self.target_config.get('shared-data', {})
	if not isinstance(shared_data, dict):
	raise TypeError(f'Field `tool.hatch.build.targets.{self.plugin_name}.shared-data` must be a mapping')
	for i, (source, relative_path) in enumerate(shared_data.items(), 1):
	if not source:
	raise ValueError(
	f'Source #{i} in field `tool.hatch.build.targets.{self.plugin_name}.shared-data` '
	f'cannot be an empty string'
	)
	elif not isinstance(relative_path, str):
	raise TypeError(
	f'Path for source `{source}` in field '
	f'`tool.hatch.build.targets.{self.plugin_name}.shared-data` must be a string'
	)
	elif not relative_path:
	raise ValueError(
	f'Path for source `{source}` in field '
	f'`tool.hatch.build.targets.{self.plugin_name}.shared-data` cannot be an empty string'
	)
	self.__shared_data = normalize_inclusion_map(shared_data, self.root)
	return self.__shared_data

Additionally, how would I go about using a local plugin/hook? Searching through the docs, I didn't see a clear answer on how to do that (save for building a project and doing a pip install – which seems overkill given this modification would (presumably) only ever apply to the project I'm working on).

[ofek]

Can you link an example?

Local build hook:

• https://hatch.pypa.io/latest/plugins/build-hook/custom/
• https://hatch.pypa.io/latest/plugins/build-hook/reference/#hatchling.builders.hooks.plugin.interface.BuildHookInterface
• https://github.com/ipython/ipyparallel/blob/main/hatch_build.py

[ofek]

cc @blink1073 has that not been migrated because shared-data needs to be dynamic?

[ofek]

@jmuchovej can you try https://hatch-jupyter-builder.readthedocs.io/en/latest/source/how_to_guides/migrating_data_files_projects.html?

[jmuchovej]

Hmm... looking at the source for hatch_jupyter_builder (and running the migration on a quick run at setup.py), it's not too clear that those files would, in-fact, be moved to {sys.prefix}/path/to/file.

I'll need to finish preparing this for publishing before I can really test and verify that this works. Will circle back when I'm done with that. 🙂

Thanks for the responsiveness and guidance!

[blink1073]

@jmuchovej, you could look at ipykernel for inspiration.

You can run hatch new --init instead of using hatch-jupyter-builder if you aren't currently using jupyter-packaging, and move files around as you need in your build hook, then refer to them by path in your tool.hatch.build.targets.wheel.shared-data metadata.

[mturoci]

Has there been any resolution for dynamic shared-data? The static declaration works as expected, but I would need a dynamic way. I would expect something like this could work:

from hatchling.builders.hooks.plugin.interface import BuildHookInterface


class CustomBuildHook(BuildHookInterface):
    def initialize(self, version, build_data):
        build_data['extra_metadata']['shared-data'] = { 'my-folder/to/share': '.' }

But unfortunately, it does not.

The only ugly working solution I came up with is to:

• Set the statictool.hatch.build.targets.wheel.shared-data.
• Create a tmp folder in the initialize hook.
• Copy the necessary files over.
• Clean it up once finished in the finalized hook.

I assume, there is a less hacky solution though.