Add mock documentation

README.md

@@ -1,29 +1,286 @@
-# prototype-ci-reading
+# Proposed Quality Workflow (QW) user guide - release 1
 
-Quick prototype for processing PR and issue data in github actions
+The Quality Workflow tool (QW) aims to help reduce the friction in creating and maintaining traceability matrices (later release could target hazards and risks).
 
-## Ensuring that we can access these parts in CI
+# Background and use-case
 
-Issues
+Within medical device software there are several design and development stages that should be tracked, versioned and a traceability matrix is a standard way of presenting this information. The Quality Management System used in WEISS is document based, which requires substantial manual effort in tracking linkages, updating of versions and ensuring that all approvals are current.
 
-- [x] Title
-- [x] First comment
-- [x] PR number
-- [x] Assignee(s)
-  - If we're using names, they're not required on github so could end up being blank
-  - We could just use github logins and have part of the config that you have to map gh logins
-    to names as you want them on the report information
-- [x] Label
+The QW allows users to track the different changes in design and development requirements, outputs, verification and validations. These different stages are linked together using github or gitlab as part of the normal development process. QW also will ensure that versioning of items are updated if their contents have changed, and notify when manual verification should be rerun due to changes in requirements or outputs.
 
-PRs
+# Setup
 
-- [x] Title
-- [x] First comment
-- [x] Assignee(s)
-- [x] Users who's final reviews were APPROVED
-- [x] Label
-- [x] [Linked Issues](https://github.com/cli/cli/discussions/7097#discussioncomment-5229031)
+## Installation
 
-## Prototyping of QW tool
+The `qw` tool requires python 3.9 or greater, if your project is in python then we suggest adding it to the developer requirements. Otherwise you may want to create a virtual environment for your python, one option is the [venv](https://docs.python.org/3.9/library/venv.html) module.
 
-Messing around with python in [src](src)
+For Unix:
+
+```shell
+python -m venv env --prompt qw # create virtual environment directory called `venv`
+source venv/bin/activate  # use the virtual environment
+pip install git+https://github.com/UCL-ARC/qw.git#egg=qw  # install qw
+echo "venv" >> .gitignore. # ensure git ignores the `venv` directory
+```
+
+## Configuration using Github
+
+### Initial setup and adding to repository
+
+The `qw` tool creates a `.qw` directory that allows versions of each design and development stage to be tracked at release time. We suggest adding this directory to your version control, so that there is a shared baseline at each release. In the first step it also creates a set of github actions and templates to ensure the tool can work correctly. To allow the `qw` to interact with github you will need to create a fine-grained personal access token. The final step of configuration then updates your github repository with rules , and add templates for each stage of the quality workflow (and another template if there is an issue or pull request that doesn't relate to the quality management documentation).
+
+#### Initialise `qw`
+
+At the top level of your git repository, run the intialisation step for github
+
+```shell
+qw init --repo github.com:username/reponame --service github
+```
+
+> INFO: A ".qw" directory has been created
+> INFO: Github actions for qw have been added to your ".github" directory
+> INFO: Templates added to your ".github" directory: "User need", "Design input", "Design output", "Pull request"
+> INFO: Please commit the files after you have finished your setup
+> INFO: Rulesets and branch protection added for branches matching "main" to "stefpiatek/dummy-medical-software"
+
+If you have a different development flow than pull requests into main/master then edit the `qw` ruleset for the repository, adding extra branches to the ruleset (e.g. `develop`)
+
+<!--
+  - [name=Stef] or do we just define what workflow they have to use? Could also give a comma separated list of branch names in the --git-service option?
+-->
+
+You can omit `--service github` if the repo is at `github.com` (as in
+this case) and you can omit `--repo <url>` if you have the desired
+repository configured as the git remote named `upstream`.
+
+#### Setup Personal Access Token
+
+- Follow the [github instructions](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens#creating-a-fine-grained-personal-access-token) we suggest the following configuration options:
+
+  | field             | value, user defined values defined by `${...}`              |
+  | ----------------- | ----------------------------------------------------------- |
+  | Token name        | `qw-${repository_name}`                                     |
+  | Expiration        | `custom...` then choose 364 days from today                 |
+  | Description       | `QW tool for ${repository_name}`                            |
+  | Repository access | `Only select repositories` -> Select the project repository |
+  | Permissions       | Only add repository permissions, see next table             |
+
+  | repository permissions | value          |
+  | ---------------------- | -------------- |
+  | Actions                | Read and write |
+  | Contents               | Read-only      |
+  | Issues                 | Read and write |
+  | Metadata               | Read-only      |
+  | Pull Requests          | Read and write |
+
+  Then click the `Generate token` button and copy the token (you can't get its value again if you close the page).
+
+- Add the personal access token to your machine's keychain (uses [keyring](https://pypi.org/project/keyring/))
+
+  ```shell
+  qw login
+  ```
+
+  <!--
+     - [name=Stef] For this I guess we should implement it in keyring using: service=`qw:github.com` username=`owner/repo`?
+     - [name=Stef] Should try and get a windows user early on to test that keyring works as expected
+  -->
+
+  > Please copy the personal access token and hit enter:
+
+  After you've done this, the system will check the personal access token. If you've configured for the remote URL <https://github.com/stefpiatek/dummy-medical-software> you would see:
+
+  > INFO: Can access "stefpiatek/dummy-medical-software"
+
+### Setup for a new user, on an existing project that uses QW
+
+The local and github repository have already been configured to work with `qw`, you'll just need to [add the personal access token](#setup-personal-access-token)
+
+### Customising configuration
+
+QW initialises with a `System` component for design inputs (used for generating a unique identifier, short code value is `X`).
+You can add extra components to the configuration using qw:
+
+```shell
+qw config add-service --name="Drug dosage" --short-code=D
+```
+
+> INFO: Added "Drug dosage (D)" to local components, please commit this file
+> INFO: Added tag "qw-component-d" to github
+
+## Configuration using Gitlab
+
+Intentionally left blank at the moment for brevity. Will aim for being able to implement this.
+
+# Using QW with github
+
+QW uses existing issues and pull requests to track the different design and development stages along with managing risks and their mitigations.
+
+## Creating QW items
+
+### User needs
+
+- In your github repository, Add a new issue, selecting the User needs template
+  ![](https://hackmd.io/_uploads/Syo1w1oy6.png)
+- Fill out the required fields and any other information if it exists
+  ![](https://hackmd.io/_uploads/rJG7_1okT.png)
+- Hit `Submit new issue` and the issue will be rendered like this: ![](https://hackmd.io/_uploads/BktGcJo1p.png)
+- The first comment of this issue will be used by `qw` for tracking, but you can edit anything manually after the `Other information` header and this will not interfere with the tool
+
+### Design inputs
+
+- In your github repository, Add a new issue, selecting the User needs template
+  ![](https://hackmd.io/_uploads/Syo1w1oy6.png)
+- Fill out the required fields and any other information that may be useful
+  ![](https://hackmd.io/_uploads/B1X07JVZ6.png)
+  - The options for requirements type are:
+    - Functional
+    - System inputs and outputs
+    - Application programming interface
+    - Alarms and warnings
+    - Security
+    - User interface
+    - Data definition and database
+    - Installation and acceptance
+    - Operation methods and maintenance
+    - Networking
+    - User maintenance
+    - Regulatory
+- Hit `Submit new issue` and theissue will be rendered like this:
+  ![](https://hackmd.io/_uploads/H1-xiDBWT.png)
+  - Note that the `design-component-d` has been added as "Drug dosage" was added as a component type with a short code of `D`.
+- The first comment of this issue will be used by `qw` for tracking, but you can edit anything manually after the `Other information` header and this will not interfere with the tool
+- The parent issue will have been updated with this issue number
+  ![](https://hackmd.io/_uploads/S1OvqDrW6.png)
+
+### Design outputs and design verification
+
+- Create a pull request from github and follow the instructions on the template
+  ![](https://hackmd.io/_uploads/BkYpd7ikp.png)
+  - If the `qw-ignore` tag is added, then this PR does is not related to the medical device aspect of the software
+- In this example, the pull request contains the design outputs and design verification. These can be added separately, where the design verification would also be linked to the design input.
+  ![](https://hackmd.io/_uploads/ryp39Xj1a.png)
+- QW will check that the chain of design items are able to be processed and fully signed off,
+  when this is successful the github action will pass
+  ![](https://hackmd.io/_uploads/H1Ix67i16.png)
+- QW requires a pull request that is labelled as a `design-validation` or `design-verification` to have at least one automated test that targets a specific `design-output`, `design-input` or `user-need`
+  - Tests are tracked manually in `.qw/test_mapping.csv` where multiple issues are separated by `;`, in the following structure
+    test name | issue(s) targetted
+    -- | --
+    InputVerifierTests::testWeightIsNegative | 40
+    InputVerifierTests::testConversionOfWeightUnits | 4;23
+    acceptance_test_user_entry.docx | 2
+  - The development team may want to write a script that allows them to get the names of all tests run and ensure that automated tests listed in the csv still exist, bonus points if you make this an automated test that fails if a test doesn't exist
+  - Adding the manual verification test scripts to version control is ideal, but you may also list the name
+  - the QW CI task will fail if:
+    - No parent issue is given for a PR
+    - A test doesn't exist which is mapped to the parent issue
+    - The structure of the `test_mapping.csv` has been altered, or `;` has not been used to separate targeted issues
+    - A test name is duplicated in `test_mapping.csv`
+
+### Design validation
+
+- QW does not require design validation documentation to be added to the repository,
+  but you may add the test script.
+  Ideally this would be in a plain text format. We suggest creating a
+  `validation` directory and storing the validation information there
+- As with [Design outputs and design verification](#design-outputs-and-design-verification), update the `.qw/test_mapping.csv` with the test script(s) that fulfill the user need.
+- Create a pull request from github and follow the instructions on the template
+  ![](https://hackmd.io/_uploads/BkYpd7ikp.png)
+  - A validation links to a user need, so link this type of issue in the PR
+  - Assign the pull request to the team member who will authorise the validation.
+    Once they have validated it and signed any required paperwork,
+    they should approve the PR.
+
+### Non-QW items
+
+- If you'd like to create an issue that doesn't relate to the medical device then use the `Non QW item` issue template.
+  This tags the issue with `qw-ignore` so that QW does not parse the issue.
+  ![](https://hackmd.io/_uploads/SkWcEzQgp.png)
+- If you'd like to create a pull request that doesn't relate to the medical device then tag the pull request with `qw-ignore`.
+
+## Closing QW items when they are not resolved by a PR
+
+- There may be times where an QW-tracked issue is required to be closed not by a PR.
+- You may close the issue as either `completed` or `not planned` ![](https://hackmd.io/_uploads/Sy22Bi9x6.png)
+- Then please add the `qw` tag that matches the reason. The options for tags are:`qw-close-duplicate`, `qw-close-wont-fix`, `qw-close-duplicate`, , `qw-close-cannot-replicate`, `qw-close-not-a-defect`
+- [name=Stef] Alternatively we could have a manual closure reason in the issue template, and then that is required to be filled with free text as a reason?
+
+## Versioning of QW items
+
+The `qw freeze` function is used to save the current state of the qw items in github, and to ensure that the versions of items are updated if their information has changed.
+This should be run regularly, which will update or add to the `.qw` directory.
+These changes should be committed and added to the main development branch by a pull request as a [Non QW item](#non-qw-items).
+
+```shell
+qw freeze
+```
+
+An example incrementing a tag upon update:
+
+> Running WQ freeze
+> Found 47 QW items
+>
+> INFO: Design Input https://github.com/stefpiatek/dummy-medical-software/issues/6 has been updated since last saved
+>
+> Previous data:
+>
+> - Description : Warfarin dosage should be calculated using based on patient age, gender and weight
+> - Parent User need: https://github.com/stefpiatek/dummy-medical-software/issues/5
+>
+> Current Data:
+>
+> - Description : Warfarin dosage in mg/kg should be calculated using based on patient age, gender and weight
+> - Parent User need: https://github.com/stefpiatek/dummy-medical-software/issues/5
+>
+> Would you like to increment the version? (y/n): <prompt response from user - y>
+> INFO: Updated tag to "qw-v2"
+> INFO: There are 2 design outputs that link to this Design Input, please ensure one of them has the "qw-v2" tag if it resolves the updated information
+>
+> - https://github.com/stefpiatek/dummy-medical-software/pull/7
+> - https://github.com/stefpiatek/dummy-medical-software/pull/12
+>   ...
+>   INFO: :heavy_check_mark: 12 new QW items added to state
+>   INFO: :heavy_check_mark: 47 QW items checked
+>   INFO: Please commit the changes in the ".qw" directory
+
+Example response when the change is trivial and does not warrant a change in the version of the QW item:
+
+> ...
+> Would you like to increment the version? (y/n): <prompt response from user - n>
+> INFO: Tag kept at "qw-v1", data for the Design Input has been updated to the current state
+> ...
+
+### Creating a documentation release
+
+When you're ready to update the documentation in your Quality Management System (QMS), you can use the QW tool's `release` command.
+Running this will:
+
+- Ensure that all issues and pull requests have been marked with `qw-ignore` or one of the `qw-` item tags, raising an error (and stopping) to ensure all items are tagged
+- Ensure that all QW items have versions
+- Ensure the entire chain `design validation -> design verification -> design output -> design input -> user need` is consistent with QW rules, starting from the furthest stage of QW items. So if there is no `design validation`, then the chain will start from `design verification`. If there was only a `user need` and `design input`s, then only these would be validated
+- Create word documents based on the QW template for export
+- [name=Stef] Optionally? Create an html page that shows a burndown graph for each of the QW item types, showing the number completed and outstanding over time. Would this be useful?
+
+```shell
+qw release qms_docs
+```
+
+> Creating a release for QW
+> Creating "qms_docs" directory if it doesn't already exist, and overwriting any previously exported files
+> INFO: :heavy_check_mark: 47 QW items checked
+> INFO: :heavy_check_mark: Documents have been written to the "qms_docs" directory
+
+# FAQ and common issues
+
+- Can I import an existing project into QW
+  - There's nothing stopping this, though each issue and pull request would need to match the required format, and be tagged appropriately by the time you'd like to run a release. This may be a reasonable amount of work and we expect issues may need to be created.
+-
+
+# Open questions
+
+- Which document parts are the most useful for export, and would we want the user to be able to edit the template (update styles and placeholder text?)
+- Would the burndown chart be useful to get an overview for development?
+- Gate who can authorise a PR, or can anyone?
+- Should we include a change request at this stage, or keep it with the risk management side of things
+- closed not by a PR - would be useful to get a steer on what is required or most useful