Skip to content

Commit

Permalink
Rewrite dev docs
Browse files Browse the repository at this point in the history 
Misc. tweaks elsewhere
  • Loading branch information
@tchamberlin
tchamberlin committed Dec 11, 2023
1 parent 9a7e9e4 commit cd0552c
Show file tree
Hide file tree
Showing 8 changed files with 249 additions and 124 deletions.
Empty file added docs/source/for_developers/dev_guide.rst
View file
Empty file.
56 changes: 30 additions & 26 deletions docs/source/for_developers/doc_standards.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,50 +2,54 @@
Documentation
*************

Setting Up Sphinx Autobuilds
============================
How does it work, and how can you contribute to it?

Here are the steps to set up Sphinx autobuilds so that you can check your documentation edits live.
How to Contribute to Dysh Docs
==============================

1. First, navigate to your `dysh` root directory and activate the `hatch` environment.
Local Builds
++++++++++++

.. code-block:: bash
The first thing you'll want to do is get the docs building locally. This makes it very easy to develop, and you can be *nearly certain* that the docs will build on Read the Docs if they have successfully built locally (assuming you've rememembered to commit all your files...)

$ hatch shell
To run a live docs server (i.e. reload on changes)

2. Next, copy the environment file template.
.. code-block:: sh
.. code-block:: bash
cd "$DYSH_REPO_ROOT"
hatch run docs
(dysh) $ cp .env.template .env
You can pass any arguments through to ``sphinx-autobuild``, e.g. to bind to a specific host and port:

3. Add values for ``DOCS_ROOT``, ``DOCS_HOST``, and ``DOCS_PORT`` in `.env`
4. Start the autobuild
.. code-block:: sh
.. code-block:: bash
cd "$DYSH_REPO_ROOT"
hatch run docs --host 0.0.0.0 --port 8001
(dysh) $ source .env
(dysh) $ startdocs
5. Go to `http://{$DOCS_HOST}:{$DOCS_PORT}` in a web browser. You should now see the documentation with live edits as you save changes.
.. note::
Do not commit the `.env` file to `git`.
Flowcharts/Diagrams
+++++++++++++++++++

Docstring Format
================
Where possible, please prefer Mermaid to pre-rendered flowcharts/diagrams. It makes it easy to embed the full source in the docs, which makes future edits significantly easier.

Gotta format the docstrings

Mermaid Diagrams
================

Diagrams can be directly in these text files by using the `sphinxcontrib-mermaid` package. Here's an example:
Dysh supports Mermaid Diagrams via the ``sphinxcontrib-mermaid`` package. Here's an example:

.. mermaid::

flowchart LR
A[Item 1] --> B[Item 2]
B --> C[Item 3]

To learn more, see the `package documentation <https://sphinxcontrib-mermaid-demo.readthedocs.io/en/latest/>`_. Mermaid also offers an `online editor <https://mermaid.live>`_ which can be used to design diagrams.
To learn more, see the `sphinxcontrib-mermaid documentation <https://sphinxcontrib-mermaid-demo.readthedocs.io/en/latest/>`_. Mermaid also offers an `online editor <https://mermaid.live>`_ which can be used to design diagrams.

Read the Docs
=============

Dysh's documentation is hosted on Read the Docs, at https://readthedocs.org/projects/dysh/. Docs are built directly from the Dysh GitHub repo.

RTD hosts several different active versions of Dysh documentation:

- ``latest``: built from the latest commit on ``main``
- ``stable``: built from that latest stable version tag
- ``release-X.Y``/``vX.Y``: built from the latest commit on a given release branch
177 changes: 123 additions & 54 deletions docs/source/for_developers/git_workflows.rst
View file
Original file line number Diff line number Diff line change
@@ -1,84 +1,153 @@
*************
Git Workflows
*************
************
Git Workflow
************

Branches
========
Background
==========

The development team employs a GitFlow workflow with personal branching. This means that code in the `main` branch should always be in a releasable state. Developers should maintain their own development branches and commit changes to a `release-x.y.z` branch. When it's time to release, a dedicated team member will merge the `release-x.y.z` branch with the `main` branch and tag it accordingly.
Dysh uses the `OpenAstronomy model for its branching and release structure <https://packaging-guide.openastronomy.org/en/latest/releasing.html#releasing-from-branches>`_. In practice, this essentially means that we are using a standard "feature branch" workflow, where new features are added via Pull Requests based on ``main``.

If you are an external collaborator, you will need to create Pull Requests from a fork of Dysh, but the instructions below should otherwise be fairly similar.

Developers are welcome to manage their git workflow however they want, but using the `GitHub CLI <https://cli.github.com/>`_ or `GitHub Desktop <https://desktop.github.com/>`_ simplify many operations.

How to Contribute
=================


There are two types of workflow:
1. Contributing to a future release. This is the most common workflow
2. Patching a previous release. This involves changes to a "release branch" -- changes will *not* go on `main`

Below is a diagram demonstrating both of these workflows. They are explored in more depth farther down.

.. mermaid::

%%{init: { 'theme': 'base' } }%%
%%{init: { 'logLevel': 'debug', 'theme': 'base', 'gitGraph': {'showBranches': true, 'showCommitLabel':true,'mainBranchOrder': 10}} }%%

gitGraph
commit id: "1"
commit id: "2"
branch release-x.y.z order: 1
commit id: "3"
branch personB-devel order: 3
branch personA-devel order: 2
commit id: "4"
checkout personB-devel
commit id: "5"
commit id: "6"
checkout personA-devel
commit id: "7"
commit id: "8"
checkout release-x.y.z
merge personA-devel id: "9"
commit id: "10"
checkout personB-devel
merge release-x.y.z id: "11"
commit id: "12"
commit id: "13"
checkout release-x.y.z
merge personB-devel id: "14"

commit id: "freeze 0.2" tag: "v0.3dev"
branch v0.2 order: 11

checkout v0.2
commit id: "start v0.2 beta test" tag: "v0.2.0b1"

branch v0.2.0b1_fix order: 14
commit id: "fix bugs from beta"
checkout v0.2
merge v0.2.0b1_fix id: "Release v0.2.0rc1" tag: "v0.2.0rc1"
commit id: "Release v0.2.0" tag: "v0.2.0"
commit id: "Bug fix for v0.2" tag: "v0.2.1"
checkout v0.2

checkout main
branch feature_x order:3
commit id: "start feature x"
commit id: "finish feature x"
checkout main
merge release-x.y.z id: "15" tag: "release-x.y.z"
merge feature_x id: "freeze 0.3" tag: "v0.4dev"

Releases
========
branch v0.3 order:100
commit id: "start v0.3 beta test" tag: "v0.3.0b1"
checkout main

Release branches will be locked once work on the next release begins.
branch feature_y order:1
checkout feature_y
commit id:"implement feature y"
checkout main

Setting up your own development branch
======================================
Contribute to ``main`` (Add to a Future Release)
++++++++++++++++++++++++++++++++++++++++++++++++

In the directory you want to work in, set up the repo:

`$ git clone git@github.com:GreenBankObservatory/dysh.git`
Primary development is always done on a feature branch off of `main`. The only way for code to get into `main` is via a `Pull Request <https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/about-pull-requests>`_ on the `Dysh GitHub <https://github.com/GreenBankObservatory/dysh>`_. A Pull Request is essentially a group of commits with a common purpose, and it is associated with a discussion thread. It encapsulates both the code changes themselves, and the explanation of why they are being made, how they work, etc.

`$ cd dysh`
Here is the lifecycle of a simple Pull Request:

`$ git checkout [branch-name]`
.. code-block:: sh
with the [branch-name] replaced by the current release or other branch. Currently, this is "release-0.2.0". Then setup your own development branch with your first name followed by "-devel":
git checkout main
git fetch origin main
git checkout -b my-cool-new-feature
# <make some code changes>
git commit -am "Desc of changes"
# <make some code changes>
git commit -am "More changes"
git push origin my-cool-new-feature
`$ git checkout [name-devel]`
At this point, the developer considers the work complete, and the Dysh GitHub has a new branch, ``my-cool-new-feature``, and the developer's repo will look like:

Now make and activate a python virtual environment so your work doesn't impede or break other concurrent projects. Whenever you do some work, make sure you are in your own development branch. When you are ready to merge changes made by other developers into your own branch,
.. mermaid::

`$ git checkout release-0.2.0`
%%{init: { 'logLevel': 'debug', 'theme': 'base', 'gitGraph': {'showBranches': true, 'showCommitLabel':true}} }%%

`$ git pull`
gitGraph
commit id:" "
branch my-cool-new-feature
checkout my-cool-new-feature
commit id: "Desc of changes"
commit id: "More changes"

`$ git checkout [name-devel]`
Now they can open a Pull Request in the Dysh repo on GitHub.

`$ git merge release-0.2.0`
At this point, a few things happen:

When you are ready to commit changes, review what's been changed with
1. GitHub CI kicks off, which runs code quality checks and unit tests
2. Another developer will review your code contribution
3. Discussion of the new feature will occur
4. Eventually, the PR will be accepted or rejected
- If it is accepted, it will be merged into ``main`` and be a part of the next release
- If it is rejected, it will go nowhere

`$ git status`

and then add the intended files using
If accepted, the Dysh repo will now look like:

`$ git add [path/to/changed/file]`
.. mermaid::

check `dysh/.gitignore` to make sure you are not adding ignored files (virtual environment data, `_build/`, etc.). Then commit and push with
%%{init: { 'logLevel': 'debug', 'theme': 'base', 'gitGraph': {'showBranches': true, 'showCommitLabel':true}} }%%

`$ git commit -m "[this is my commit message]"`
gitGraph
commit id:" "
branch my-cool-new-feature
checkout my-cool-new-feature
commit id: "Desc of changes"
commit id: "More changes"
checkout main
merge my-cool-new-feature

`$ git push`

The first time you run this, it will give a command about setting the origin upstream. Simply copy and run that command. Users of GitHub Desktop can also achieve all of these above steps using the app interface. Next, go to the `dysh GitHub page <https://github.com/GreenBankObservatory/dysh/>`_ and submit a pull request.


Contribute to a Previous Release (Add to a release branch)
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

In this example, release ``v0.2.0`` has been released, and development has started on ``v0.3.0`` (on ``main``). Then, a bug is reported in ``v0.2.0``, necessitating a bug fix.


.. mermaid::

%%{init: { 'logLevel': 'debug', 'theme': 'base', 'gitGraph': {'showBranches': true, 'showCommitLabel':true}} }%%

gitGraph

commit id: "freeze 0.2" tag: "v0.3dev"
branch v0.2

checkout v0.2

checkout main
commit id: " "
checkout v0.2

commit id: "Release 0.2.0" tag: "v0.2.0"
branch v0.2.0-bug-fix
commit id: "fix bug"
checkout v0.2
merge v0.2.0-bug-fix id: "Release v0.2.1"
commit id: "Release 0.2.1" tag: "v0.2.1"

checkout v0.2

checkout main
commit id: " "
9 changes: 5 additions & 4 deletions docs/source/for_developers/index.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,15 +2,16 @@
For Developers
**************

Here's some info that developers might find useful
Documentation specifically for people who want to contribute code to Dysh

.. toctree::
:maxdepth: 2

contributing
install
setup_dev_env
git_workflows
doc_standards

install
github_integrations
git_workflows
black
terminal_messages
78 changes: 78 additions & 0 deletions docs/source/for_developers/setup_dev_env.rst
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@
************************************
Setting Up a Development Environment
************************************


Pre-Requisites
==============

There are a few core things you will need to develop Dysh:

1. A Linux computer (Mac and Windows should also work, but we don't have docs for those)
2. Python (specifically you should be using the minimum-supported version of Python; see ``pyproject.toml``)
3. `pre-commit <https://pre-commit.com/>`_: used to enforce code quality checks
4. `Hatch <https://hatch.pypa.io/latest/install/#pipx>`_ (optional, technically)

Python
------

See site-specific instructions [TODO] for GBO, UMD, etc. instructions -- the necessary version of Python is probably already available somewhere.


If you do not already have access to the minimum-supported version of Python, you will need to install it. This can be done manually via `the official site <https://www.python.org/downloads/>`_, but it is much easier to use pyenv.


Once you can run ``python --version`` and see the correct version pop up, you are ready to move on.

pre-commit
----------



Dysh Development Environment
============================

Start to finish, how do you initialize a Dysh development environment?


Via Hatch
---------

.. code-block:: sh
git clone [email protected]:GreenBankObservatory/dysh.git
cd dysh
hatch shell #
hatch run test # Prove that the tests can run and pass
Manual Virtual Environment
--------------------------

In some environments it makes more sense to manage your environments yourself. For example, at GBO we have a variety of hosts that all mount a common filesystem, but can have different OS versions. Here the standard "one env per project per user" breaks down, and it is better to manage things yourself.

.. code-block:: sh
git clone [email protected]:GreenBankObservatory/dysh.git
cd dysh
python3.9 -m venv /path/to/venv # Create your venv
source /path/to/venv/bin/activate # Activate your venv
pip install -e .[all] # Install dysh in edit mode, including all extra deps
hatch run test # Use hatch to trigger pytest inside your manually-created venv
Advanced Cases
==============

If you need to exactly replicate the Python dependencies of either:
1. A given CI deployment
2. A given Dysh deployment at GBO (or elsewhere)

You will instead be relying on the lockfile (``requirements.txt``) to provide exact version pins of all of your dependencies.

With the same version of Python as the environment you are trying to replicate, and with the same OS (or as similar as possible):

.. code-block:: sh
python3.9 -m venv /path/to/venv # Create your venv
source /path/to/venv/bin/activate # Activate your venv
pip install -r requirements.txt # Install from lockfile
Loading

0 comments on commit cd0552c

Please sign in to comment.