Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

23.02.ug #7

Open
wants to merge 48 commits into
base: 23.02.ug
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
48 commits
Select commit Hold shift + click to select a range
f110bf5
Wait_finished method for job API (regarding #240) (#242)
JonaOtto Sep 9, 2022
f60220c
Fixed mem_per_cpu setting in job class (#243)
JonaOtto Sep 10, 2022
7b370f7
fix typos (#252)
schluenz Dec 23, 2022
da978a6
Fix creating RPM packages (#248)
tazend Jan 3, 2023
a2fa6e1
Support updating end_time in slurm_update_reservation (#255)
pllopis Jan 6, 2023
ace7785
Fix formatting error for reservation_list example (#256)
pllopis Jan 6, 2023
2617092
Raise ValueError on slurm_update_reservation (#257)
pllopis Jan 6, 2023
ffc419f
Actually retrieve the batch script contents (#258)
tazend Feb 4, 2023
61d4740
Brought __rpc_num2string in sync with slurm 22.05 message types (#261)
wresch Feb 9, 2023
dd73ddf
bump pyslurm version (#263)
tazend Feb 26, 2023
f3dbb73
Transition from sphinx to mkdocs for documentation (#271)
tazend Mar 10, 2023
3b89325
Migrate pyslurm.pyx from reST to google docstring style (#273)
tazend Mar 15, 2023
f506d63
Support for Slurm 23.02.X (#277)
tazend Apr 7, 2023
23d436e
Revert usage of return type hints (#281)
tazend Apr 12, 2023
3dc607a
Reorganize slurm header definitions into seperate files (#282)
tazend Apr 30, 2023
0043080
Big rework (#283)
tazend May 1, 2023
b81d491
Update documentation and do some restructuring (#285)
tazend May 5, 2023
4ec608e
drop the doc/ directory in doc_files for building rpm (#286)
wpoely86 May 5, 2023
be9395c
some more doc fixes (#287)
tazend May 5, 2023
788f445
just use the copyright notice again that was also previously used (#288)
tazend May 6, 2023
42471d8
some more improvements (#291)
tazend May 12, 2023
538bf42
Start maintaining a changelog (#293)
tazend May 18, 2023
d5b0076
fix paths in MANIFEST.in (#295)
elelayan May 24, 2023
8ebc5c5
Rework Partition API (#296)
tazend May 27, 2023
a84d23c
Allow db job modification and some fixes (#297)
tazend May 27, 2023
37f7e22
Few fixes and handle modification consistently (#300)
tazend Jun 2, 2023
3a603e1
Fix parsing user names in JobSearchFilter (#303)
tazend Jun 5, 2023
485722b
Implement DB Associations API and a little refactoring. (#304)
tazend Jun 25, 2023
0a65047
Refactor + improve docs (#305)
tazend Jul 13, 2023
fe572c7
More doc improvements and other fixes (#306)
tazend Jul 18, 2023
0e4a327
Update CHANGELOG (#307)
tazend Jul 18, 2023
a0dfa6b
Fix recursively transforming collections to JSON (#308)
tazend Jul 18, 2023
2810957
Bump version to 23.2.2 (#309)
tazend Jul 18, 2023
b435d68
Fix a reference to an invalid property and return the correct value f…
robgics Aug 24, 2023
4ecdfa0
Nodes: Some Bugfixes and new attributes (#318)
tazend Aug 26, 2023
7ebf64a
Add new attributes to db.Jobs and truncate_time option to db.JobFilte…
tazend Sep 8, 2023
84a3d11
Merge branch '22.05.ug' into 23.02.ug
itkovian Oct 2, 2023
2ee2696
bump: workflow image to ubuntu 22.04 LTS
itkovian Oct 13, 2023
5b76298
bump: slurm version in docker images to 23.02.6
itkovian Oct 13, 2023
32d1e69
bump: back to ubuntu 20.04 LTS because of py3.6
itkovian Oct 13, 2023
141303f
fix: other slurm image
itkovian Oct 13, 2023
3067a70
bump: actions to newer version
itkovian Oct 13, 2023
cff9c8a
bump: python 3.9 version to 3.9.18
itkovian Oct 17, 2023
3f94b5d
fix: verbose building
itkovian Oct 17, 2023
7194d82
feat: support for building on el9
itkovian Jan 10, 2024
43e242d
fix: rpm is called python3-devel
itkovian Jan 10, 2024
b8d437b
fix: remove cython upper bound verison limit
itkovian Jan 10, 2024
2482522
fix: remove exceprtion raise
itkovian Jan 10, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Update documentation and do some restructuring (PySlurm#285)
* update README

- include logo
- remove testing section (maybe it back separetely in the documentation)
- remove workflow-badge (will only fail anyway, add back later when the
  building process has been updated)

* update the documentation

* move pyslurm.core.common package to pyslurm.utils

* move pyslurm.core.db package to pyslurm.db

* rename JobStats to JobStatistics
  • Loading branch information
tazend authored May 5, 2023
commit b81d491aec980aacc2be775dac229e0a2894f2db
128 changes: 21 additions & 107 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,18 +1,23 @@
# PySlurm
# <img src="logo.png" alt="PySlurm Logo">

[![PySlurm](https://github.com/PySlurm/pyslurm/actions/workflows/pyslurm.yml/badge.svg?branch=main)](https://github.com/PySlurm/pyslurm/actions/workflows/pyslurm.yml)
pyslurm is the Python client library for the [Slurm Workload Manager](https://slurm.schedmd.com)

## Overview

PySlurm is the Python client library for the [Slurm](https://slurm.schedmd.com) HPC Scheduler.

## Prerequisites
## Requirements

* [Slurm](https://slurm.schedmd.com) - Slurm shared library and header files
* [Python](https://www.python.org) - >= 3.6
* [Cython](https://cython.org) - >= 0.29.30 but < 3.0

This PySlurm branch is for the Slurm Major-Release 23.02
This Version is for Slurm 23.02.x

## Versioning

In pyslurm, the versioning scheme follows the official Slurm versioning. The
first two numbers (`MAJOR.MINOR`) always correspond to Slurms Major-Release,
for example `23.02`.
The last number (`MICRO`) is however not tied in any way to Slurms `MICRO`
version, but is instead PySlurm's internal Patch-Level. For example, any
pyslurm 23.02.X version should work with any Slurm 23.02.X release.

## Installation

Expand All @@ -21,14 +26,14 @@ By default, it is searched inside `/usr/include` for the Header files and in
For Slurm installations in different locations, you will need to provide
the corresponding paths to the necessary files.

You can specify these Paths with environment variables (recommended), for example:
You can specify those with environment variables (recommended), for example:

```shell
export SLURM_INCLUDE_DIR=/opt/slurm/23.02/include
export SLURM_LIB_DIR=/opt/slurm/23.02/lib
```

Then you can proceed to install PySlurm, for example by cloning the Repository:
Then you can proceed to install pyslurm, for example by cloning the Repository:

```shell
git clone https://github.com/PySlurm/pyslurm.git && cd pyslurm
Expand All @@ -40,105 +45,14 @@ pip install .

Also see `python setup.py --help`

## Release Versioning

PySlurm's versioning scheme follows the official Slurm versioning. The first
two numbers (MAJOR.MINOR) always correspond to Slurms Major-Release, for example
`23.02`. The last number (MICRO) is however not tied in any way to Slurms
MICRO version. For example, any PySlurm 23.02.X version should work with any
Slurm 23.02.X release.

## Documentation

The API documentation is hosted at <https://pyslurm.github.io>.

To build the docs locally, use [Sphinx](http://www.sphinx-doc.org) to generate
the documentation from the reStructuredText based docstrings found in the
pyslurm module once it is built:

```shell
cd doc
make clean
make html
```

## Testing

PySlurm requires an installation of Slurm.

### Using a Test Container

To run tests locally without an existing Slurm cluster, `docker` and
`docker-compose` is required.

Clone the project:

```shell
git clone https://github.com/PySlurm/pyslurm.git
cd pyslurm
```

Start the Slurm container in the background:

```shell
docker-compose up -d
```

The cluster takes a few seconds to start all the required Slurm services. Tail
the logs:

```shell
docker-compose logs -f
```

When the cluster is ready, you will see the following log message:

```text
Cluster is now available
```

Press CTRL+C to stop tailing the logs. Slurm is now running in a container in
detached mode. `docker-compose` also bind mounds the git directory inside the
container at `/pyslurm` so that the container has access to the test cases.

Install test dependencies:

```shell
pipenv sync --dev
```

Execute the tests inside the container:

```shell
pipenv run pytest -sv scripts/run_tests_in_container.py
```

When testing is complete, stop the running Slurm container:

```shell
docker-compose down
```

### Testing on an Existing Slurm Cluster

You may also choose to clone the project and run tests on a node where Slurm is
already compiled and installed:

```shell
git clone https://github.com/PySlurm/pyslurm.git
cd pyslurm
pip install .
./scripts/configure.sh
pipenv sync --dev
pipenv run pytest -sv
```

## Contributors

PySlurm is made by [contributors like
pyslurm is made by [contributors like
you](https://github.com/PySlurm/pyslurm/graphs/contributors).

## Help
## Support

Feel free to ask questions in the [GitHub
Discussions](https://github.com/orgs/PySlurm/discussions)

Ask questions on the [PySlurm Google
Group](https://groups.google.com/forum/#!forum/pyslurm)
Found a bug or you are missing a feature? Feel free to [open an Issue!](https://github.com/PySlurm/pyslurm/issues/new)
1 change: 1 addition & 0 deletions doc_requirements.txt
Original file line number Diff line number Diff line change
Expand Up @@ -4,3 +4,4 @@ setuptools
mkdocstrings[python]
mike
mkdocs-material
mkdocs-awesome-pages-plugin
53 changes: 1 addition & 52 deletions docs/index.md
Original file line number Diff line number Diff line change
@@ -1,52 +1 @@
---
hide:
- navigation
---
# PySlurm: Slurm Interface to python

This module provides a low-level Python wrapper around the Slurm C-API using Cython.

::: pyslurm.config
handler: python

::: pyslurm.front_end
handler: python

::: pyslurm.hostlist
handler: python

::: pyslurm.job
handler: python

::: pyslurm.jobstep
handler: python

::: pyslurm.node
handler: python

::: pyslurm.partition
handler: python

::: pyslurm.reservation
handler: python

::: pyslurm.slurmdb_events
handler: python

::: pyslurm.slurmdb_reservations
handler: python

::: pyslurm.slurmdb_clusters
handler: python

::: pyslurm.slurmdb_jobs
handler: python

::: pyslurm.statistics
handler: python

::: pyslurm.topology
handler: python

::: pyslurm.trigger
handler: python
--8<-- "README.md"
1 change: 1 addition & 0 deletions docs/logo.png
3 changes: 3 additions & 0 deletions docs/reference/.pages
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
title: API Reference
nav:
- ...
10 changes: 10 additions & 0 deletions docs/reference/config.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
---
title: Config
---

!!! warning
This API is currently being completely reworked, and is subject to be
removed in the future when a replacement is introduced

::: pyslurm.config
handler: python
3 changes: 3 additions & 0 deletions docs/reference/db/.pages
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
title: Database
nav:
- ...
10 changes: 10 additions & 0 deletions docs/reference/db/cluster.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
---
title: Cluster
---

!!! warning
This API is currently being completely reworked, and is subject to be
removed in the future when a replacement is introduced

::: pyslurm.slurmdb_clusters
handler: python
6 changes: 6 additions & 0 deletions docs/reference/db/connection.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
---
title: Connection
---

::: pyslurm.db.Connection
handler: python
10 changes: 10 additions & 0 deletions docs/reference/db/event.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
---
title: Event
---

!!! warning
This API is currently being completely reworked, and is subject to be
removed in the future when a replacement is introduced

::: pyslurm.slurmdb_events
handler: python
4 changes: 4 additions & 0 deletions docs/reference/db/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
# pyslurm.db

The `pyslurm.db` package contains all functionality to interact with the Slurm
Database Daemon (slurmdbd)
13 changes: 13 additions & 0 deletions docs/reference/db/job.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
---
title: Job
---

!!! note
This supersedes the [pyslurm.slurmdb_job](../old/db/job.md) class, which
will be removed in a future release

::: pyslurm.db.Job
handler: python

::: pyslurm.db.Jobs
handler: python
6 changes: 6 additions & 0 deletions docs/reference/db/jobsearchfilter.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
---
title: JobSearchFilter
---

::: pyslurm.db.JobSearchFilter
handler: python
6 changes: 6 additions & 0 deletions docs/reference/db/jobstats.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
---
title: JobStatistics
---

::: pyslurm.db.JobStatistics
handler: python
9 changes: 9 additions & 0 deletions docs/reference/db/jobstep.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
---
title: JobStep
---

::: pyslurm.db.JobStep
handler: python

::: pyslurm.db.JobSteps
handler: python
10 changes: 10 additions & 0 deletions docs/reference/db/reservation.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
---
title: Reservation
---

!!! warning
This API is currently being completely reworked, and is subject to be
removed in the future when a replacement is introduced

::: pyslurm.slurmdb_reservations
handler: python
9 changes: 9 additions & 0 deletions docs/reference/exceptions.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
---
title: Exceptions
---

::: pyslurm.PyslurmError
handler: python

::: pyslurm.RPCError
handler: python
10 changes: 10 additions & 0 deletions docs/reference/frontend.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
---
title: Frontend
---

!!! warning
This API is currently being completely reworked, and is subject to be
removed in the future when a replacement is introduced

::: pyslurm.front_end
handler: python
10 changes: 10 additions & 0 deletions docs/reference/hostlist.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
---
title: Hostlist
---

!!! warning
This API is currently being completely reworked, and is subject to be
removed in the future when a replacement is introduced

::: pyslurm.hostlist
handler: python
Loading