Symbol

.readthedocs.yaml

@@ -0,0 +1,25 @@
+# Read the Docs configuration file for Sphinx project
+
+# Required
+version: 2
+
+# Set the OS, Python version and other tools you might need
+build:
+  os: ubuntu-22.04
+  tools:
+    python: "3.9"
+
+# Build documentation in the "docs/" directory with Sphinx
+sphinx:
+  configuration: docs/conf.py
+
+# Optionally build your docs in additional formats such as PDF and ePub
+# formats:
+#   - pdf
+#   - epub
+
+# Optional but recommended, declare the Python requirements required
+# to build your documentation
+python:
+  install:
+    - requirements: requirements.txt

CHANGELOG.md

@@ -0,0 +1,81 @@
+# Changelog
+
+All notable changes to this project will be documented in this file.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/) (with possible titles of 'added', 'changed', 'deprecated', 'removed', 'fixed', or 'security'),
+and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+This should align with:
+* Releases on [PyPI](https://pypi.org/project/kailo-beewell-dashboard/#history)
+* Releases on [GitHub](https://github.com/kailo-beewell/kailo_beewell_dashboard_package/releases) (which are like a non-portable changelog only displayed to users within GitHub)
+
+## 0.2.0
+
+**Release date:** 1st March 2024
+
+**Contributors:** Amy Heather
+
+Modified package so it can be used to produce the **synthetic symbol** #BeeWell survey dashboard, as well as the standard survey dashboard.
+
+https://github.com/kailo-beewell/kailo_beewell_dashboard_package/compare/main...amy
+
+### Added
+
+* Created `CHANGELOG.md` (with backdated entry for 0.1.0)
+* Created `CITATION.cff`
+* Created package documentation using Sphinx and readthedocs - `docs/` and `.readthedocs.yaml`
+* `create_group_list()` in `bar_charts.py` which creates a correctly formatted list of strings depending on number of inputs (e.g. 'a', 'a and b', 'a, b and c')
+* `grammar.py` - contains `lower_first()` which converts first letter of string to lower case, unless all other letters are upper case
+* Add `aggregate_demographic()` to `create_and_aggregate_data.py`
+* Add new alternative inputs, outputs and process to existing functions that were developed for standard survey, so they can be used to output equivalent content for symbol survey. This includes
+    * `authentication.py` - custom login screen text
+    * `create_and_aggregate_data.py` - two possible sets of groups to aggregate by
+    * `explore_results.py` - custom page text, and providing survey type to functions like `filter_by_group()`
+    * `import_data.py` - add names of data for symbol survey as for session_state and as in TiDB Cloud, and simplified import function
+    * `page_setup.py` - to use name 'symbol' survey in the page menu
+    * `reshape_data.py` - for differences with symbol (e.g. exc SEN)
+    * `response_labels.py` - new function `create_symbol_response_label_dict()`
+    * `static_report.py` - converting some of the processes into standalone functions, so they can be imported to old function making static standard report and new function making static symbol report
+    * `who_took_part.py` - modifications for symbol like different headers and no descriptive text
+
+### Changed
+
+* Moved all About page text to `reuse_text.py`
+* In `requirements.txt`, upgraded pip to 24.0 and streamlit to 1.31.1, and add packages used to produce documentation (sphinx, sphinx-rtd-theme, myst-parser, sphinx-autoapi)
+
+### Removed
+
+* Removed option to compare data import from TiDB to imported CSVs from the project directory (as function wasn't being used) in `import_data.py`
+
+### Fixed
+
+* Hide responses (and non-response) on 'who took part' if n<10 for a given response option (more strict than elsewhere on dashboard, which just requires n>=10 for the entire question). For this, modified `survey_responses()` in `bar_charts.py`
+
+## 0.1.0
+
+**Release date:** 21st February 2024
+
+**Contributors:** [Amy Heather](https://github.com/amyheather)
+
+First release of kailo-beewell-dashboard package on PyPi. Contains functions for production of **synthetic standard** #BeeWell survey dashboard (including static PDF version, which can be produced using the dashboard).
+
+### Added
+
+Functions used to generate and process data, and to produce the dashboard:
+* `authentication.py` - for user authentication using Django
+* `bar_charts.py` - to create bar charts of proportions of each survey responses, or ordered bar charts comparing scores
+* `bar_charts_text.py` - dictionary with descriptions to go above bar charts on the 'explore results' page
+* `convert_image.py` - converts a plotly figure to HTML string
+* `create_and_aggregate_data.py` - functions used to create and process the pupil-level data
+* `explore_results.py` - functions used for the 'explore results' page
+* `import_data.py` - connects to TiDB Cloud and imports data to session state
+* `page_setup.py` - page configuration, styling, formatting
+* `reshape_data.py` - to reshape data or extract a certain element, with functions often used across multiple different pages
+* `response_labels.py` - dictionary of labels to each of the question response options
+* `reuse_text.py` - two sections of text that were reused on different pages of the dashboard vs PDF report
+* `score_descriptions.py` - simple descriptions used on the 'explore results' page to support score interpretation
+* `static_report.py` - uses same functions as on the dashboard, to produce a static HTML report (containing same information as in dashboard)
+* `stylable_container.py` - produces stylised containers for streamlit
+* `summary_rag.py` - produces the red-amber-green boxes and 'summary' page introduction and table
+* `switch_page.py` - function to switch page programmatically
+* `who_took_part.py` - functions used for the 'who took part' page

CITATION.cff

@@ -0,0 +1,29 @@
+# This CITATION.cff file was generated with cffinit.
+# Visit https://bit.ly/cffinit to generate yours today!
+
+cff-version: 1.2.0
+title: kailo-beewell-dashboard
+message: >-
+  If you use this package, please cite it using the metadata
+  from this file.
+type: software
+authors:
+  - given-names: Amy
+    family-names: Heather
+    email: [email protected]
+    affiliation: University of Exeter
+    orcid: 'https://orcid.org/0000-0002-6596-3479'
+repository-code: >-
+  https://github.com/kailo-beewell/kailo_beewell_dashboard_package
+repository-artifact: 'https://pypi.org/project/kailo-beewell-dashboard/'
+abstract: >-
+  Tools to support creation of #BeeWell survey dashboards
+  for the Kailo project
+keywords:
+  - python
+  - streamlit
+  - '#BeeWell'
+  - Kailo
+license: MIT
+version: 0.2.0
+date-released: '2024-02-21'

README.md

@@ -1,10 +1,15 @@
 # `kailo-beewell-dashboard`: tools to support creation of #BeeWell survey dashboards for the Kailo project.
 
-[![ORCID: Amy Heather](https://img.shields.io/badge/ORCID-0000--0002--6596--3479-brightgreen)](https://orcid.org/0000-0002-6596-3479)
+[![PyPI package](https://img.shields.io/badge/PyPI_package-0.2.0-2596be.svg)](https://pypi.org/project/kailo-beewell-dashboard/0.2.0/)
 [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
+[![ORCID: Amy Heather](https://img.shields.io/badge/ORCID_Amy_Heather-0000--0002--6596--3479-brightgreen)](https://orcid.org/0000-0002-6596-3479)
 
 This package contains functions that are used in the creation of the various #BeeWell survey dashboards for Kailo. They have been compiled into a single package to prevent code duplication between the different repositories.
 
+**Package on PyPI:** https://pypi.org/project/kailo-beewell-dashboard/
+
+**Package documentation:** http://kailo-beewell-dashboard.readthedocs.io/
+
 ## Features
 
 1. Functions used to generate and aggregate synthetic data
@@ -13,3 +18,20 @@ This package contains functions that are used in the creation of the various #Be
 ## How to install?
 
 `pip install kailo-beewell-dashboard`
+
+## Citation
+
+If you use this package, please include the following citation
+
+> Heather, Amy. (2024). kailo-beewell-dashboard: tools to support creation of #BeeWell survey dashboards for the Kailo project (0.2.0). https://pypi.org/project/kailo-beewell-dashboard/.
+
+```tex
+@software{forecast_tools,
+  author = {Heather, Amy},
+  title = {kailo-beewell-dashboard},
+  date = {2024-03-01},
+  version = {0.2.0},
+  publisher = {PyPI},
+  url = {https://pypi.org/project/kailo-beewell-dashboard/}
+}
+```

docs/Makefile

@@ -0,0 +1,20 @@
+# Minimal makefile for Sphinx documentation
+#
+
+# You can set these variables from the command line, and also
+# from the environment for the first two.
+SPHINXOPTS    ?=
+SPHINXBUILD   ?= sphinx-build
+SOURCEDIR     = .
+BUILDDIR      = _build
+
+# Put it first so that "make" without argument is like "make help".
+help:
+	@$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
+
+.PHONY: help Makefile
+
+# Catch-all target: route all unknown targets to Sphinx using the new
+# "make mode" option.  $(O) is meant as a shortcut for $(SPHINXOPTS).
+%: Makefile
+	@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)

docs/authentication.md

@@ -0,0 +1,37 @@
+# Authentication
+
+This guide provides a step-by-step on how I set up the user authentication for this dashboard, which was based on these tutorials:
+* [https://towardsdatascience.com/secure-your-streamlit-app-with-django-bb0bee2a6519](https://towardsdatascience.com/secure-your-streamlit-app-with-django-bb0bee2a6519)
+* [https://towardsdatascience.com/streamlit-access-control-dae3ab8b7888](https://towardsdatascience.com/streamlit-access-control-dae3ab8b7888)
+
+## How to set up user authentication
+
+1. If not already in environment, install django (add to requirements.txt, recreate environment)
+2. Create a Django app. In terminal:
+    * `django-admin startproject config .`
+
+    This produced a config folder containing 5 .py files.
+3. Create a superuser to manage all other users:
+    * `python3 manage.py migrate`
+    * `python3 manage.py createsuperuser`
+
+    Then enter user as `kailobeewell` and the dartington email address. I have record of password personally which I can share with other team members.
+
+4. Start the server: `python3 manage.py runserver`.
+5. Open web-browser and go to http://localhost:8000/admin/. Login with the superuser username and password just created.
+6. Add some users by clicking '+ Add' next to Users, then entering a username and password for each user. For the synthetic dashboard, I create some basic logins (which need to be more secure for actual dashboard). For each user followed the format of:
+    * Username: schoola
+    * Password: schoolapassword
+7. Created the authentication.py script (see utilities folder)
+8. On each of the app pages, imported the check_password() function from authentication.py, then sat all the page code under it, eg:
+```
+import streatmlit as st
+from utilities.authentication import check_password
+
+if check_password():
+    st.title('Page title')
+    st.write('Page content)
+    ...
+```
+
+The actions above (migrate, superuser, adding users) will have generated and modified a db.sqlite3. Make sure you push this up to GitHub repository - I found that the app failed on deployment without it.

docs/changelog_link.md

@@ -0,0 +1,2 @@
+```{include} ../CHANGELOG.md
+```

docs/conf.py

@@ -0,0 +1,41 @@
+# Configuration file for the Sphinx documentation builder.
+
+# -- Project information -----------------------------------------------------
+
+project = 'kailo-beewell-dashboard'
+copyright = '2024, Amy Heather'
+author = 'Amy Heather'
+
+# -- General configuration ---------------------------------------------------
+
+extensions = [
+    'myst_parser',  # To use markdown as well as reStructuredText
+    'autoapi.extension'  # Auto generate module and function documentation
+]
+
+# Location of files for auto API
+autoapi_dirs = ['../kailo_beewell_dashboard']
+
+# File types for documentation
+source_suffix = ['.rst', '.md']
+
+templates_path = ['_templates']
+
+# Location of toctree
+master_doc = 'contents'
+
+# List of patterns, relative to source directory, that match files and
+# directories to ignore when looking for source files.
+# This pattern also affects html_static_path and html_extra_path.
+exclude_patterns = ['_build', '_templates', 'Thumbs.db', '.DS_Store']
+
+language = 'English'
+
+# -- Options for HTML output -------------------------------------------------
+
+html_theme = 'sphinx_rtd_theme'
+
+# Add any paths that contain custom static files (such as style sheets) here,
+# relative to this directory. They are copied after the builtin static files,
+# so a file named "default.css" will overwrite the builtin "default.css".
+html_static_path = ['_static']

docs/contents.rst

@@ -0,0 +1,14 @@
+Site contents
+*************
+
+.. toctree::
+
+   Home <index>
+   environments
+   package_maintenance
+   authentication
+   hosting_data
+   standard_survey_scores
+   streamlit_community_cloud
+   changelog_link
+   Documentation for modules and functions (API Reference) <autoapi/index>

docs/environments.md

@@ -0,0 +1,25 @@
+# Environments
+
+This package and the accompanying dashboard repositories use virtual environments. This page contains tips and advice relating to these, in case they are unfamiliar to you.
+
+## Virtual environment wrapper
+
+I recommend using virtualenvwrapper to manage your python virtual environments. This is because:
+* It stores all your environments in one place (so you can easily see a list of your environments)
+* The syntax for deleting an environment prevents you from accidentally deleting folders, as it has a specify command, rather than a general delete command that could apply to an environment or a folder with the same name
+
+To do so, you'll need to have `pip`, `python`, `virtualenv` and `virtualenvwrapper` installed on your machine.
+
+Commands:
+* Create environment - `mkvirtualenv env_kailo_dashboards`
+* Enter environment -  `workon env_kailo_dashboards`
+* Install requirements into environment - `pip install -r requirements.txt`
+* See list of all available environments - `workon`
+* List contents of active environment - `pip list`
+* Delete environment - `rmvirtualenv env_kailo_dashboards`
+
+## Streamlit Community Cloud
+
+Streamlit Community Cloud only appears to work with virtual environment - states compatability with environment.yml but failed when I and my colleague attempted it.
+
+Therefore, we us a virtual environment with the requirements.txt file provided and python version 3.9.12 (with community cloud set up on python 3.9).