Directory Structure

Directory Structure

Podder Task CLI generates the following directories and files.

[name]
├── README.md
├── config
│     └── [name]
│            └── model.json
├── data
│     └── [name]
│            └── dummy_model.pth
├── input
│     └──[name]
├── manage.py
├── output
│     └── [name]
├── poetry.lock
├── processes
│     ├── __init__.py
│     └── [name]
│              ├── __init__.py
│              └── process.py
├── pyproject.toml
└── tests
    ├── __init__.py
    └── [name]
           ├── __init__.py
           └── test_process.py

[name] will be replaced with the name you set on the podder new [name] command

processes/[name]

This is the place to put your code. The entry point is always process.py. You need to inherit podder_task_foundation.Process and override the following 2 methods.

def initialize(self, context: Context) -> None:
    # Initializing
    # System call this method only once when the initialization process

def execute(self, input_payload: Payload, output_payload: Payload, context: Context):
    # Actual process execution code

config/[name]

This is the place to put configuration data for your code.

You can put JSON / YAML / TOML files which include any of configuration information such as parameters for the process, name of model/data files.

For instance, you can put model.json include the following JSON data.

{
    "name": "yourmodel.pth",
	  "parameters": {
        "paramA": 123,
		  "paramB": "test",
    }
}

Then you can access with context object which is passed as a parameter to initialize and execute methods.

# You can get "yourmodel.pth"
model_name = context.config.get(“model.name”)
# You can get dict ( ["paramA":123, "paramB":"test"] )
parameters = context.config.get(“model.parameters”)
# You can get 123
parameters = context.config.get(“model.parameters.paramA”)

data/[name]

This is the place to put all your data files ( model files, text data… ) which you want to use inside your module.

For instance, you use PyTorch and need to load yourmodel.pth to execute the module, you can write it as follows.

self._model = TheModelClass()
model_path = context.file.get_data_file(context.config.get("model.name"))
self._model.load_state_dict(torch.load(model_path))

Don’t forget to load your model on the initialize method, ( and load to instance property such as self._model. Do not load model on the execute method to load model only once at the initialization process.