Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

build(python): bump python version to use v3.11 #2164

Merged
merged 64 commits into from
Oct 29, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
64 commits
Select commit Hold shift + click to select a range
cf6622e
test
MartinBelthle Sep 30, 2024
549faff
also test the binary
MartinBelthle Sep 30, 2024
3d0c49a
fix little mypy issue
MartinBelthle Sep 30, 2024
7d41963
still 51 tests failures
MartinBelthle Sep 30, 2024
1cdda23
fix issue but still 52 test failures
MartinBelthle Sep 30, 2024
acb7ec2
still 47 test failures
MartinBelthle Oct 1, 2024
abe010b
45
MartinBelthle Oct 1, 2024
c1ff88f
43
MartinBelthle Oct 1, 2024
f922cea
fix lots of tests
MartinBelthle Oct 1, 2024
2fc280e
23 failures
MartinBelthle Oct 1, 2024
2a705ee
use StrEnum
MartinBelthle Oct 1, 2024
c86bc8a
fix 2 mypy issues still 10 test failures
MartinBelthle Oct 1, 2024
663e293
fix last failing tests
MartinBelthle Oct 1, 2024
534df03
fix all mypy issues
MartinBelthle Oct 1, 2024
6271165
fix test
MartinBelthle Oct 1, 2024
d18a215
try to bump pyinstaller to make desktop version work
MartinBelthle Oct 1, 2024
50a2bb4
fix test that failed the CI
MartinBelthle Oct 1, 2024
133635d
update requirements to work only with 3.11
MartinBelthle Oct 1, 2024
7e67b01
simplify small parts of code with new python api
MartinBelthle Oct 1, 2024
0e598b6
use 3.11 inside other files than .py
MartinBelthle Oct 1, 2024
134ada2
bump version inside dockerfile
MartinBelthle Oct 1, 2024
1d7c52d
merge with dev
MartinBelthle Oct 2, 2024
7ea7c69
fix typing
MartinBelthle Oct 2, 2024
72fa3c8
Merge branch 'dev' into bump-python/use-3.11-version
MartinBelthle Oct 7, 2024
6a1303b
try to bump pyinstaller
MartinBelthle Oct 7, 2024
02845d3
test debug=True inside EXE .spec file
MartinBelthle Oct 7, 2024
f80dfbe
add debug True always
MartinBelthle Oct 7, 2024
1e58a52
try to change stdout
MartinBelthle Oct 8, 2024
925f841
merge with dev
MartinBelthle Oct 14, 2024
096f54a
Merge branch 'dev' into bump-python/use-3.11-version
MartinBelthle Oct 15, 2024
449b28c
revert changes inside gh actions
MartinBelthle Oct 15, 2024
1bb22f4
revert changes on .spec file
MartinBelthle Oct 15, 2024
b6bb16b
remove useless scripts
MartinBelthle Oct 16, 2024
63e5160
update Dockerfile
MartinBelthle Oct 16, 2024
5940e4c
use pip instead of pip3
MartinBelthle Oct 16, 2024
3633afc
try to fix error with pip
MartinBelthle Oct 16, 2024
00ccf74
try to install pip
MartinBelthle Oct 16, 2024
6214bd8
try to understand permission error
MartinBelthle Oct 16, 2024
ef73be0
use debug venv
MartinBelthle Oct 16, 2024
0583595
use venv not at the root
MartinBelthle Oct 16, 2024
e32e40a
try to undrestand
MartinBelthle Oct 16, 2024
c13cfd2
im lost
MartinBelthle Oct 16, 2024
b9f9b6a
im lost
MartinBelthle Oct 16, 2024
3b4f235
put pip inside .sh
MartinBelthle Oct 16, 2024
47d50dc
use pip3 instead of pip
MartinBelthle Oct 16, 2024
40ad874
try to put pip inside docker file instead of .sh
MartinBelthle Oct 16, 2024
37511c9
use python3.11 instead of python3
MartinBelthle Oct 16, 2024
8ff02d6
force pip reinstall
MartinBelthle Oct 16, 2024
930c6ee
try to use pip outside ouf venv
MartinBelthle Oct 16, 2024
a110c03
use pip instead of pip3
MartinBelthle Oct 16, 2024
9f6185e
use +x instead of 777
MartinBelthle Oct 16, 2024
d86c2da
try to use pip outside of venv
MartinBelthle Oct 16, 2024
9eab197
last chance
MartinBelthle Oct 16, 2024
3ce1b70
never deactivate the venv
MartinBelthle Oct 16, 2024
8efb042
....
MartinBelthle Oct 16, 2024
5982b0d
remove deactivate keyword
MartinBelthle Oct 16, 2024
b18856b
It should work once antares-ci is good
MartinBelthle Oct 16, 2024
cf44d58
use no cache for pip
MartinBelthle Oct 17, 2024
d9838bd
Merge branch 'dev' into bump-python/use-3.11-version
MartinBelthle Oct 18, 2024
ae65af7
merge with dev
MartinBelthle Oct 23, 2024
30b7b96
Merge branch 'dev' into bump-python/use-3.11-version
MartinBelthle Oct 24, 2024
9b8584c
add doc inside gui.py
MartinBelthle Oct 24, 2024
5a5f66b
rebase with dev
MartinBelthle Oct 25, 2024
87f7c81
use strEnum instead of str, Enum
MartinBelthle Oct 25, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/deploy.yml
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ jobs:
- name: 🐍 Set up Python
uses: actions/setup-python@v5
with:
python-version: 3.8
python-version: 3.11

- name: 🐍 Install development dependencies
run: |
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/license_header.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ jobs:
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: 3.8
python-version: 3.11
- name: Install dependencies
run: |
python -m pip install --upgrade pip
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ jobs:
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: 3.8
python-version: 3.11
- name: Install dependencies
run: |
python -m pip install --upgrade pip
Expand Down Expand Up @@ -46,7 +46,7 @@ jobs:
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: 3.8
python-version: 3.11
- name: Install dependencies
run: |
python -m pip install --upgrade pip
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/worker.yml
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ jobs:
- name: 🐍 Set up Python
uses: actions/setup-python@v5
with:
python-version: 3.8
python-version: 3.11

- name: 🐍 Install dependencies
run: |
Expand Down
8 changes: 2 additions & 6 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
FROM python:3.8-slim-bullseye
FROM python:3.11-slim-bullseye

# RUN apt update && apt install -y procps gdb

Expand All @@ -16,10 +16,6 @@ COPY ./scripts /scripts
COPY ./alembic /alembic
COPY ./alembic.ini /alembic.ini

RUN ./scripts/install-debug.sh

RUN pip3 install --no-cache-dir --upgrade pip \
&& pip3 install --no-cache-dir -r /conf/requirements.txt

RUN pip3 install --no-cache-dir --upgrade pip && pip3 install --no-cache-dir -r /conf/requirements.txt

ENTRYPOINT ["./scripts/start.sh"]
4 changes: 2 additions & 2 deletions antarest/core/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
import multiprocessing
import tempfile
from dataclasses import asdict, dataclass, field
from enum import Enum
from enum import StrEnum
from pathlib import Path
from typing import Dict, List, Optional

Expand All @@ -25,7 +25,7 @@
DEFAULT_WORKSPACE_NAME = "default"


class Launcher(str, Enum):
class Launcher(StrEnum):
SLURM = "slurm"
LOCAL = "local"
DEFAULT = "default"
Expand Down
4 changes: 2 additions & 2 deletions antarest/core/configdata/model.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
#
# This file is part of the Antares project.

from enum import Enum
from enum import StrEnum
from typing import Any, Optional

from sqlalchemy import Column, Integer, String # type: ignore
Expand Down Expand Up @@ -43,6 +43,6 @@ def to_dto(self) -> ConfigDataDTO:


# APP MAIN CONFIG KEYS
class ConfigDataAppKeys(str, Enum):
class ConfigDataAppKeys(StrEnum):
MAINTENANCE_MODE = "MAINTENANCE_MODE"
MESSAGE_INFO = "MESSAGE_INFO"
4 changes: 2 additions & 2 deletions antarest/core/interfaces/eventbus.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,14 +11,14 @@
# This file is part of the Antares project.

from abc import ABC, abstractmethod
from enum import Enum
from enum import StrEnum
from typing import Any, Awaitable, Callable, List, Optional

from antarest.core.model import PermissionInfo
from antarest.core.serialization import AntaresBaseModel


class EventType(str, Enum):
class EventType(StrEnum):
ANY = "_ANY"
STUDY_CREATED = "STUDY_CREATED"
STUDY_DELETED = "STUDY_DELETED"
Expand Down
4 changes: 2 additions & 2 deletions antarest/core/maintenance/model.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,10 +10,10 @@
#
# This file is part of the Antares project.

from enum import Enum
from enum import StrEnum


class MaintenanceMode(str, Enum):
class MaintenanceMode(StrEnum):
NORMAL_MODE = "NORMAL"
MAINTENANCE_MODE = "MAINTENANCE"

Expand Down
4 changes: 2 additions & 2 deletions antarest/core/model.py
Original file line number Diff line number Diff line change
Expand Up @@ -24,15 +24,15 @@
SUB_JSON = Union[ELEMENT, JSON, List[Any], None]


class PublicMode(str, enum.Enum):
class PublicMode(enum.StrEnum):
NONE = "NONE"
READ = "READ"
EXECUTE = "EXECUTE"
EDIT = "EDIT"
FULL = "FULL"


class StudyPermissionType(str, enum.Enum):
class StudyPermissionType(enum.StrEnum):
"""
User permission belongs to Study
"""
Expand Down
4 changes: 2 additions & 2 deletions antarest/core/tasks/model.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
import typing as t
import uuid
from datetime import datetime
from enum import Enum
from enum import Enum, StrEnum

from sqlalchemy import Boolean, Column, DateTime, ForeignKey, Integer, Sequence, String # type: ignore
from sqlalchemy.engine.base import Engine # type: ignore
Expand All @@ -28,7 +28,7 @@
from antarest.study.model import Study


class TaskType(str, Enum):
class TaskType(StrEnum):
EXPORT = "EXPORT"
VARIANT_GENERATION = "VARIANT_GENERATION"
COPY = "COPY"
Expand Down
4 changes: 2 additions & 2 deletions antarest/core/utils/archives.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@
import tempfile
import typing as t
import zipfile
from enum import Enum
from enum import StrEnum
from pathlib import Path

import py7zr
Expand All @@ -25,7 +25,7 @@
logger = logging.getLogger(__name__)


class ArchiveFormat(str, Enum):
class ArchiveFormat(StrEnum):
ZIP = ".zip"
SEVEN_ZIP = ".7z"

Expand Down
4 changes: 2 additions & 2 deletions antarest/eventbus/web.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@

import dataclasses
import logging
from enum import Enum
from enum import StrEnum
from http import HTTPStatus
from typing import List, Optional

Expand All @@ -32,7 +32,7 @@
logger = logging.getLogger(__name__)


class WebsocketMessageAction(str, Enum):
class WebsocketMessageAction(StrEnum):
SUBSCRIBE = "SUBSCRIBE"
UNSUBSCRIBE = "UNSUBSCRIBE"

Expand Down
12 changes: 12 additions & 0 deletions antarest/gui.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,18 @@
# SPDX-License-Identifier: MPL-2.0
#
# This file is part of the Antares project.

import os
import sys

# The Pyinstaller version we use has a known issue on windows and to fix it we need to implement this workaround.
# See issue description and workaround on pyinstaller website:
# https://pyinstaller.org/en/stable/common-issues-and-pitfalls.html#sys-stdin-sys-stdout-and-sys-stderr-in-noconsole-windowed-applications-windows-only
if sys.stdout is None:
sys.stdout = open(os.devnull, "w")
if sys.stderr is None:
sys.stderr = open(os.devnull, "w")

import argparse
import multiprocessing

Expand Down
2 changes: 1 addition & 1 deletion antarest/launcher/adapters/abstractlauncher.py
Original file line number Diff line number Diff line change
Expand Up @@ -116,6 +116,6 @@ def update_log(log_line: str) -> None:
channel=EventChannelDirectory.JOB_STATUS + job_id,
)
)
self.cache.put(f"Launch_Progress_{job_id}", launch_progress_dto.model_dump())
self.cache.put(f"Launch_Progress_{job_id}", launch_progress_dto.model_dump(mode="json"))

return update_log
8 changes: 4 additions & 4 deletions antarest/launcher/model.py
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@ def from_launcher_params(cls, params: t.Optional[str]) -> "LauncherParametersDTO
return cls.model_validate(from_json(params))


class LogType(str, enum.Enum):
class LogType(enum.StrEnum):
STDOUT = "STDOUT"
STDERR = "STDERR"

Expand All @@ -79,14 +79,14 @@ def to_suffix(self) -> str:
return "out.log"


class JobStatus(str, enum.Enum):
class JobStatus(enum.StrEnum):
PENDING = "pending"
FAILED = "failed"
SUCCESS = "success"
RUNNING = "running"


class JobLogType(str, enum.Enum):
class JobLogType(enum.StrEnum):
BEFORE = "BEFORE"
AFTER = "AFTER"

Expand Down Expand Up @@ -139,7 +139,7 @@ def json_schema_extra(schema: t.MutableMapping[str, t.Any]) -> None:
exit_code=0,
solver_stats="time: 1651s, call_count: 1, optimization_issues: []",
owner=UserInfo(id=0o007, name="James BOND"),
).model_dump()
).model_dump(mode="json")


class JobLog(Base): # type: ignore
Expand Down
2 changes: 1 addition & 1 deletion antarest/launcher/service.py
Original file line number Diff line number Diff line change
Expand Up @@ -185,7 +185,7 @@ def update(
self.event_bus.push(
Event(
type=EventType.STUDY_JOB_COMPLETED if final_status else EventType.STUDY_JOB_STATUS_UPDATE,
payload=job_result.to_dto().model_dump(),
payload=job_result.to_dto().model_dump(mode="json"),
permissions=PermissionInfo(public_mode=PublicMode.READ),
channel=EventChannelDirectory.JOB_STATUS + job_result.id,
)
Expand Down
6 changes: 1 addition & 5 deletions antarest/matrixstore/service.py
Original file line number Diff line number Diff line change
Expand Up @@ -101,11 +101,7 @@ def get_matrix_id(self, matrix: t.Union[t.List[t.List[float]], str]) -> str:
"""
# noinspection SpellCheckingInspection
if isinstance(matrix, str):
# str.removeprefix() is not available in Python 3.8
prefix = "matrix://"
if matrix.startswith(prefix):
return matrix[len(prefix) :]
return matrix
return matrix.removeprefix("matrix://")
elif isinstance(matrix, list):
return self.create(matrix)
else:
Expand Down
4 changes: 2 additions & 2 deletions antarest/service_creator.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@

import logging
import typing as t
from enum import Enum
from enum import StrEnum
from pathlib import Path

import redis
Expand Down Expand Up @@ -65,7 +65,7 @@
"""


class Module(str, Enum):
class Module(StrEnum):
APP = "app"
WATCHER = "watcher"
MATRIX_GC = "matrix_gc"
Expand Down
14 changes: 7 additions & 7 deletions antarest/study/business/aggregator_management.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@

import logging
import typing as t
from enum import Enum
from enum import StrEnum
from pathlib import Path

import numpy as np
Expand Down Expand Up @@ -54,31 +54,31 @@
logger = logging.getLogger(__name__)


class MCRoot(str, Enum):
class MCRoot(StrEnum):
MC_IND = "mc-ind"
MC_ALL = "mc-all"


class MCIndAreasQueryFile(str, Enum):
class MCIndAreasQueryFile(StrEnum):
VALUES = "values"
DETAILS = "details"
DETAILS_ST_STORAGE = "details-STstorage"
DETAILS_RES = "details-res"


class MCAllAreasQueryFile(str, Enum):
class MCAllAreasQueryFile(StrEnum):
VALUES = "values"
DETAILS = "details"
DETAILS_ST_STORAGE = "details-STstorage"
DETAILS_RES = "details-res"
ID = "id"


class MCIndLinksQueryFile(str, Enum):
class MCIndLinksQueryFile(StrEnum):
VALUES = "values"


class MCAllLinksQueryFile(str, Enum):
class MCAllLinksQueryFile(StrEnum):
VALUES = "values"
ID = "id"

Expand Down Expand Up @@ -308,7 +308,7 @@ def _process_df(self, file_path: Path, is_details: bool) -> pd.DataFrame:
# loop over the cluster id to extract the values of the actual columns
for cluster_id, dummy_component in cluster_dummy_product_cols:
for actual_col in actual_cols:
col_values = un_normalized_df[(cluster_id, actual_col, dummy_component)].tolist() # type: ignore
col_values = un_normalized_df[(cluster_id, actual_col, dummy_component)].tolist()
new_obj[actual_col] += col_values
new_obj[CLUSTER_ID_COL] += [cluster_id for _ in range(df_len)]
new_obj[TIME_ID_COL] += list(range(1, df_len + 1))
Expand Down
8 changes: 4 additions & 4 deletions antarest/study/business/area_management.py
Original file line number Diff line number Diff line change
Expand Up @@ -229,11 +229,11 @@ def from_model(
obj = {
"average_unsupplied_energy_cost": average_unsupplied_energy_cost,
"average_spilled_energy_cost": average_spilled_energy_cost,
**area_folder.optimization.filtering.model_dump(by_alias=False),
**area_folder.optimization.nodal_optimization.model_dump(by_alias=False),
**area_folder.optimization.filtering.model_dump(mode="json", by_alias=False),
**area_folder.optimization.nodal_optimization.model_dump(mode="json", by_alias=False),
# adequacy_patch is only available if study version >= 830.
**(
area_folder.adequacy_patch.adequacy_patch.model_dump(by_alias=False)
area_folder.adequacy_patch.adequacy_patch.model_dump(mode="json", by_alias=False)
if area_folder.adequacy_patch
else {}
),
Expand Down Expand Up @@ -363,7 +363,7 @@ def update_areas_props(
for area_id, update_area in update_areas_by_ids.items():
# Update the area properties.
old_area = old_areas_by_ids[area_id]
new_area = old_area.copy(update=update_area.model_dump(by_alias=False, exclude_none=True))
new_area = old_area.copy(update=update_area.model_dump(mode="json", by_alias=False, exclude_none=True))
new_areas_by_ids[area_id] = new_area

# Convert the DTO to a configuration object and update the configuration file.
Expand Down
2 changes: 1 addition & 1 deletion antarest/study/business/areas/renewable_management.py
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,7 @@ def json_schema_extra(schema: t.MutableMapping[str, t.Any]) -> None:
unit_count=100,
nominal_capacity=1000.0,
ts_interpretation="power-generation",
).model_dump()
).model_dump(mode="json")


class RenewableClusterCreation(RenewableClusterInput):
Expand Down
Loading
Loading