Include data files in Python package with pyproject.toml and hatch

[wigging]

I'm building a Python package using pyproject.toml and hatch. The structure of the project is shown below. The package located at src/mypackage uses data files in CSV format which are in the src/data directory. When I install the package with pip the data files do not get installed. How can I install the package and include the data files?

mypackage
├── README.md
├── examples
│   └── example.py
├── pyproject.toml
├── src
│   ├── data
│   │   └── fruits.csv
│   └── mypackage
│       ├── __init__.py
│       └── fruit_reader.py

The read_fruits() function in fruit_reader.py reads the data from the CSV file; but when I install the package the CSV file is not installed. I'm not sure if the path is correctly defined, so that may be an issue as well.

# fruit_reader.py

import pandas as pd
from pathlib import Path

def read_fruits():
    """
    Read fruit data from CSV file.
    """
    path = Path(__file__).parent / 'data/fruits.csv'
    print(f'File path is \n{path}')

    df = pd.read_csv(path)
    print(f'Dataframe is \n{df}')

# pyproject.toml

[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

[project]
name = "mypackage"
version = "0.1"
authors = [
  { name="Example Author", email="[email protected]" },
]
description = "A small example package"
readme = "README.md"
requires-python = ">=3.9"
classifiers = [
  "Programming Language :: Python :: 3",
  "License :: OSI Approved :: MIT License",
  "Operating System :: OS Independent",
]

I tried to include the data files as shown below but this causes the whole package to not get installed.

[tool.hatch.build]
include = [
  "/src/data/*.csv",
]

[ofek]

Hey! You should put data under src/mypackage

[wigging]

Yeah, src/mypackage/data seems to be the expected structure. Are there any situations where you would have folders in src other than the package folder?

[ofek]

Tests, or multiple packages like https://github.com/psf/black/tree/main/src

[wigging]

I thought the tests folder is supposed to be outside the src folder? But having supporting packages in src makes sense.

[ofek]

Tests can go anywhere you want as long as they aren't included in the wheel