Skip to content

Commit

Permalink
Rename zoo_keeper -> zookeeper (#16)
Browse files Browse the repository at this point in the history
* Rename zoo_keeper -> zookeeper

* Update readme

* Rename folder

* Add readme badges
  • Loading branch information
lgeiger authored May 30, 2019
1 parent 1e935f1 commit c741e21
Show file tree
Hide file tree
Showing 14 changed files with 29 additions and 27 deletions.
20 changes: 11 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,22 +1,24 @@
# Zoo Keeper
# Zookeeper

[![Azure DevOps builds](https://img.shields.io/azure-devops/build/plumerai/zookeeper/11.svg?logo=azure-devops)](https://plumerai.visualstudio.com/zookeeper/_build/latest?definitionId=11&branchName=master) [![Azure DevOps coverage](https://img.shields.io/azure-devops/coverage/plumerai/zookeeper/11.svg?logo=azure-devops)](https://plumerai.visualstudio.com/zookeeper/_build/latest?definitionId=11&branchName=master) [![PyPI - Python Version](https://img.shields.io/pypi/pyversions/zookeeper.svg)](https://pypi.org/project/zookeeper/) [![PyPI](https://img.shields.io/pypi/v/zookeeper.svg)](https://pypi.org/project/zookeeper/) [![PyPI - License](https://img.shields.io/pypi/l/zookeeper.svg)](https://github.com/plumerai/zookeeper/blob/master/LICENSE) [![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/ambv/black) [![Join the community on Spectrum](https://withspectrum.github.io/badge/badge.svg)](https://spectrum.chat/larq)

A small library for managing deep learning models, hyper parameters and datasets designed to make training deep learning models easy and reproducible.

## Getting Started

Zoo Keeper allows you to build command line interfaces for training deep learning models with very little boiler plate using [click](https://click.palletsprojects.com/) and [TensorFlow Datasets](https://www.tensorflow.org/datasets/). It helps you structure your machine learning projects in a framework agnostic and effective way.
Zoo Keeper is heavily inspired by [Tensor2Tensor](https://github.com/tensorflow/tensor2tensor) and [Fairseq](https://github.com/pytorch/fairseq/) but is designed to be used as a library making it lightweight and very flexible. Currently Zoo Keeper is limited to image classification tasks but we are working on making it useful for other tasks as well.
Zookeeper allows you to build command line interfaces for training deep learning models with very little boiler plate using [click](https://click.palletsprojects.com/) and [TensorFlow Datasets](https://www.tensorflow.org/datasets/). It helps you structure your machine learning projects in a framework agnostic and effective way.
Zookeeper is heavily inspired by [Tensor2Tensor](https://github.com/tensorflow/tensor2tensor) and [Fairseq](https://github.com/pytorch/fairseq/) but is designed to be used as a library making it lightweight and very flexible. Currently zookeeper is limited to image classification tasks but we are working on making it useful for other tasks as well.

### Installation

```console
pip install zoo-keeper
pip install zookeeper
pip install colorama # optional for colored console output
```

### Registry

Zoo Keeper uses registries to keep track of you data preprocessing, models and hyperparameters.
Zookeeper keeps track of data preprocessing, models and hyperparameters to allow you to reference them by name from the commandline.

#### Datasets and Preprocessing

Expand All @@ -26,7 +28,7 @@ In the following we will use [MNIST](http://yann.lecun.com/exdb/mnist) and defin
```python
import tensorflow as tf

from zoo_keeper import cli, build_train, HParams, registry
from zookeeper import cli, build_train, HParams, registry

@registry.register_preprocess("mnist")
def default(image, training=False):
Expand Down Expand Up @@ -78,7 +80,7 @@ class basic(HParams):

### Training loop

To train the models registered above we will need to write a custom training loop. Zoo Keeper will then tie everything together:
To train the models registered above we will need to write a custom training loop. Zookeeper will then tie everything together:

```python
@cli.command()
Expand Down Expand Up @@ -111,11 +113,11 @@ if __name__ == "__main__":
cli()
```

If you want to register your models in separate files, make sure to import them before calling `cli` to allow Zoo Keeper to properly register them. To install your CLI as a executable command checkout the [`setuptools` integration](http://click.palletsprojects.com/en/7.x/setuptools/) of Click.
If you want to register your models in separate files, make sure to import them before calling `cli` to allow zookeeper to properly register them. To install your CLI as a executable command checkout the [`setuptools` integration](http://click.palletsprojects.com/en/7.x/setuptools/) of Click.

#### Usage

Zoo Keeper already ships with `prepare`, `plot`, and `tensorboard` commands, but now also includes the `train` command we created above:
Zookeeper already ships with `prepare`, `plot`, and `tensorboard` commands, but now also includes the `train` command we created above:

```console
python examples/train.py --help
Expand Down
2 changes: 1 addition & 1 deletion azure-pipelines.yml
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ jobs:
displayName: "pytest"
condition: ne(variables['coverage'], 'true')

- script: pytest . --junitxml=junit/test-results.xml --cov=zoo_keeper --cov-report=xml --cov-report=html --cov-config=.coveragerc
- script: pytest . --junitxml=junit/test-results.xml --cov=zookeeper --cov-report=xml --cov-report=html --cov-config=.coveragerc
displayName: "pytest coverage"
condition: eq(variables['coverage'], 'true')

Expand Down
2 changes: 1 addition & 1 deletion examples/train.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
from zoo_keeper import cli, build_train, registry, HParams
from zookeeper import cli, build_train, registry, HParams
import tensorflow as tf


Expand Down
4 changes: 2 additions & 2 deletions setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,14 +7,14 @@ def readme():


setup(
name="zoo-keeper",
name="zookeeper",
version="0.1.0",
author="Plumerai",
author_email="[email protected]",
description="A small library for managing deep learning models, hyper parameters and datasets",
long_description=readme(),
long_description_content_type="text/markdown",
url="https://github.com/plumerai/zoo-keeper",
url="https://github.com/plumerai/zookeeper",
packages=find_packages(),
license="Apache 2.0",
install_requires=["click>=7.0", "matplotlib>=3.0.3", "tensorflow-datasets>=1.0.2"],
Expand Down
4 changes: 0 additions & 4 deletions zoo_keeper/__init__.py

This file was deleted.

4 changes: 4 additions & 0 deletions zookeeper/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
from zookeeper.hparam import HParams
from zookeeper.cli import build_train, cli

__all__ = ["build_train", "cli", "HParams"]
10 changes: 5 additions & 5 deletions zoo_keeper/cli.py → zookeeper/cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ def build_train(function):
)
@click.option(
"--output-prefix",
default=os.path.expanduser("~/zoo-keeper-logs"),
default=os.path.expanduser("~/zookeeper-logs"),
help="Directory prefix used to save model checkpoints and logs.",
)
@click.option(
Expand Down Expand Up @@ -76,7 +76,7 @@ def train(
validationset,
**kwargs,
):
from zoo_keeper import registry
from zookeeper import registry

dataset = registry.get_dataset(
dataset_name,
Expand Down Expand Up @@ -118,7 +118,7 @@ def prepare(datasets, data_dir):
@click.option("--dataset", help="Tensorflow dataset name.")
@click.option(
"--output-prefix",
default=os.path.expanduser("~/zoo-keeper-logs"),
default=os.path.expanduser("~/zookeeper-logs"),
help="Directory prefix used to save model checkpoints and logs.",
)
@click.option("--output-dir", "--logdir", help="Directory containing checkpoints.")
Expand All @@ -141,14 +141,14 @@ def tensorboard(model, dataset, output_prefix, output_dir):
@click.option("--data-dir", type=str, help="Directory with training data.")
@click.option(
"--output-prefix",
default=os.path.join(os.path.expanduser("~/zoo-keeper-logs"), "plots"),
default=os.path.join(os.path.expanduser("~/zookeeper-logs"), "plots"),
help="Directory prefix used to save plots",
)
@click.option(
"--format", default="pdf", type=click.Choice(["png", "pdf", "ps", "eps", "svg"])
)
def plot(dataset, preprocess_fn, data_dir, output_prefix, format):
from zoo_keeper import registry, data_vis
from zookeeper import registry, data_vis

output_dir = Path(output_prefix).joinpath(dataset, preprocess_fn)
output_dir.mkdir(parents=True, exist_ok=True)
Expand Down
4 changes: 2 additions & 2 deletions zoo_keeper/cli_test.py → zookeeper/cli_test.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
from zoo_keeper import registry, cli, build_train, HParams, data
from zookeeper import registry, cli, build_train, HParams, data
from click.testing import CliRunner
import click
from unittest import mock
Expand Down Expand Up @@ -155,7 +155,7 @@ def test_cli_wrong_hparams():
def test_tensorboard(os_system):
result = runner.invoke(cli, ["tensorboard", "foo", "--dataset", "bar"])
assert result.exit_code == 0
logdir = path.expanduser("~/zoo-keeper-logs/bar/foo")
logdir = path.expanduser("~/zookeeper-logs/bar/foo")
os_system.assert_called_once_with(f"tensorboard --logdir={logdir}")


Expand Down
File renamed without changes.
2 changes: 1 addition & 1 deletion zoo_keeper/data_test.py → zookeeper/data_test.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
import pytest
from unittest import mock
from zoo_keeper import data
from zookeeper import data


@mock.patch("os.makedirs")
Expand Down
File renamed without changes.
File renamed without changes.
2 changes: 1 addition & 1 deletion zoo_keeper/hparam_test.py → zookeeper/hparam_test.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
"""Tests for hparam."""
import pytest
import click
from zoo_keeper.hparam import HParams
from zookeeper.hparam import HParams


@pytest.fixture
Expand Down
2 changes: 1 addition & 1 deletion zoo_keeper/registry.py → zookeeper/registry.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
import tensorflow_datasets as tfds
from zoo_keeper.data import Dataset
from zookeeper.data import Dataset

MODEL_REGISTRY = {}
HPARAMS_REGISTRY = {}
Expand Down

0 comments on commit c741e21

Please sign in to comment.