Set up testing/CI and other basic package infrastructure

[sloede]

[...] we should perhaps set up testing/CI (similarly to TrixiShallowWater.jl) before merging this to main.

Originally posted by @tristanmontoya in #2 (comment)

List of things to set up (feel free to amend/check off):

• A basic Julia test setup (test/runtests.jl and friends, based on functionality in TrixiBase.jl)
• CI: testing (.github/workflows/ci.yml)
• Documenter: add Documenter key to GitHub repository secrets
• CI: Documenter
• CI: CompatHelper
• CI: TagBot
• Add .JuliaFormatter file (format #5)
• CI: FormatCheck (format #5)
• CI: SpellCheck (5dd0e57)
• CI: Dependabot (0b417e8)
• Codecov: enable repository in GitHub org settings
• Zenodo: enable repository for DOI retrieval
• Add badges to README: docs, license, DOI etc.
• Downstream CI tests for Trixi.jl

[sloede]

IMHO this is an excellent suggestion @tristanmontoya!

I've added some of the things that I recommend to get right up front, before starting to consider true content additions, to the OP above ☝️

Most of these things can be found in TrixiShallowWater.jl, a good place to get started is maybe this early state where a lot of the above-mentioned points are already in but not much else yet:
https://github.com/trixi-framework/TrixiShallowWater.jl/tree/c1cad34a12a4e17c716601292042107925a89886

The advantage of setting up these things first is that test turnaround times are much faster now, which helps to get everything right. I even recommend to temporarily remove Trixi.jl from this package's dependencies such that you can avoid the precompilation time during CI runs while testing everything.

Feel free to ping me in case of questions.

[tristanmontoya]

It seems like the CI badge sometimes says "failing" even when the default branch passes. I think it may be reporting the status of failing CI runs triggered by a PR. I tried adding ?branch=main following this tutorial, but the badge just changed to "no status" and I ended up changing it back.

[benegee]

Well spotted!

Maybe our URL scheme is outdated? And maybe the branch defaults to main?

Scheme as used in current Trixi.jl README

Trixi.jl

Trixi.jl main

Trixi.jl feature-t8codemesh-checkpointing

TrixiAtmo.jl

TrixiAtmo.jl main

TrixiAtmo.jl bg/moist-equations-lucas

Scheme as used in the docs

Trixi.jl

Trixi.jl main

Trixi.jl feature-t8codemesh-checkpointing

TrixiAtmo.jl

TrixiAtmo.jl main

TrixiAtmo.jl bg/moist-equations-lucas

[tristanmontoya]

Thanks for looking into that, I didn't catch that the rest of the URL was different! I think it makes sense to change the TrixiAtmo.jl one to https://github.com/trixi-framework/TrixiAtmo.jl/actions/workflows/ci.yml/badge.svg, and it should probably be changed similarly in Trixi.jl too (unless the Trixi devs have a reason for keeping the old URL scheme).

[benegee]

I think we should not run tests with coverage and upload the coverage data for ubuntu, macos, and windows. It seem to cause errors like

Response: {"message":"Can't add a job to a build that is already closed. Build 10060164422 is closed. See docs.coveralls.io/parallel-builds","error":true}`

Maybe we should do coverage testing for the ubuntu run only, in case we in general want to do testing for MacOS and Windows as well. @tristanmontoya

[tristanmontoya]

@benegee I don't fully understand all the details of the Trixi test setup yet, so I may be missing something, but it seems like the full tests are not running on Ubuntu, stopping after one or just a few time steps due to maxiters. For example, this happens here: https://github.com/trixi-framework/TrixiAtmo.jl/actions/runs/11787658352/job/32833359691#step:6:3350.

Could it be that the Ubuntu tests are checking coverage only, but not actually running fully? My understanding was that #23 would make Ubuntu have tests and coverage, while Windows and Mac would have tests only. Am I understanding correctly?

[benegee]

You are absolutely right! This change

 - name: Run tests without coverage
        if: ${{ matrix.os != 'ubuntu-latest' }}

in #23 was not very clever.