Skip to content

Commit

Permalink
doc: update the outage documentation, add e-mail template
Browse files Browse the repository at this point in the history
  • Loading branch information
praiskup committed Apr 29, 2024
1 parent 50b4b4e commit 4d9ed87
Show file tree
Hide file tree
Showing 2 changed files with 71 additions and 8 deletions.
30 changes: 22 additions & 8 deletions doc/maintenance/announce_outage.rst
Original file line number Diff line number Diff line change
Expand Up @@ -19,20 +19,34 @@ follow the steps outlined in this document.
Planned outage
--------------

1. Prepare the infrastructure ticket similar to `this old one <https://pagure.io/fedora-infrastructure/issue/10854>`_.
1. Red Hatters-only: Schedule a calendar event, and invite the ``Engineering
Outage Calendar`` room.

2. Send email to `copr-devel`_ mailing list informing about an upcomming
release. We usually copy-paste text of the infrastructure ticket created in a
previous step. Don't forget to put a link to the ticket at the end of the
email. See the `example <https://lists.fedoraproject.org/archives/list/[email protected]/message/FVVX3Y7IVRTFW3NYVBTWX3AK3BHNRATX/>`_.
2. Prepare `an infrastructure ticket
<https://pagure.io/fedora-infrastructure/new_issue>`_ using one of these
templates

3. Adjust the `Matrix channel`_ title so it contains a message similar to::
.. toctree::
:maxdepth: 1

Upgrading packages <announce_outage_example_01>

3. Send email to `copr-devel`_ mailing list informing about an upcomming
release. We usually copy-paste text of the infrastructure ticket, including
a back-reference to the *infra ticket*. Sending to::

[email protected]
[email protected]
OPT-IN [email protected]
OPT-IN [email protected]

4. Adjust the `Matrix channel`_ title so it contains a message similar to::

Planned outage 2022-08-17 20:00 UTC - https://pagure.io/fedora-infrastructure/issue/10854

4. Create a new "planned" `Fedora Status SOP`_ entry.
5. Create a new "planned" `Fedora Status SOP`_ entry.

6. Create warning banner on Copr homepage::
7. Create warning banner on Copr homepage::

copr-frontend warning-banner --outage_time "2022-12-31 13:00-16:00 UTC" --ticket 1234

Expand Down
49 changes: 49 additions & 0 deletions doc/maintenance/announce_outage_example_01.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
.. _outage_announce_01:

Upgrading packages
------------------

The Fedora Infra ticket and the e-mail may look like::

Subject: Fedora Copr Outage - Updating to a <Month> <Year> Version
------------------------------------------------------------------

There will be a Fedora Copr outage starting at

$ date --date "2023-11-28 13:30 UTC"

The outage will last approximately 2 hours. The build queue processing will
be stopped during the outage, and the Frontend/Web-UI will be down most of
the time (no new tasks accepted).

<EITHER>
The DNF packages and repositories (hosted on copr-backend) will be available
during this outage.

<OR>
The DNF packages and repositories (hosted on copr-backend) will be
unavailable for a short period, about 10 minutes, as the copr-backend
machine will be rebooted.

Reason for outage:
We will update the infrastructure machines to the latest packages that are
currently being developed.

Affected Services:
https://copr.fedorainfracloud.org/
https://download.copr.fedorainfracloud.org/results/

Upstream ticket:
https://github.com/fedora-copr/copr/issues/<ticket>

<e-mail backref to infra ticket>
Infrastructure ticket:
https://pagure.io/fedora-infrastructure/issue/11468

Please join Fedora Build System Matrix channel:
https://matrix.to/#/#buildsys:fedoraproject.org

Outage Contacts:
@nikromen (Jiří Kyjovský) / @frostyx (Jakub Kadlčík) / @praiskup (Pavel Raiskup)

Or comment on this ticket.

0 comments on commit 4d9ed87

Please sign in to comment.