Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

use mtt action for integration tests #361

Merged
merged 4 commits into from
Apr 26, 2024
Merged

use mtt action for integration tests #361

merged 4 commits into from
Apr 26, 2024

Conversation

BuckarooBanzay
Copy link
Member

@BuckarooBanzay BuckarooBanzay commented Apr 21, 2024 •
edited
Loading

This PR removes the manual integration-testing files and switches to the mtt action.

It currently tests against latest engine release 5.8.0 and current master/main branches of the dependencies (game and mods)

Note: The test against 5.5.0 is removed now due to some gh-action limitations (BuckarooBanzay/mtt#4) IMO: we should let go of the old engine versions anyway

@BuckarooBanzay BuckarooBanzay added Enhancement New feature or request Cleanup Cleanup of bad code or other redundant/unnecessary stuff labels Apr 21, 2024
@BuckarooBanzay BuckarooBanzay marked this pull request as ready for review April 21, 2024 16:17
S-S-X
S-S-X approved these changes Apr 21, 2024
View reviewed changes
Copy link
Member

@S-S-X S-S-X left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd be happier if test specific stuff would be in some sub dirs dedicated for tests, I mean similar how it was.

Anyway a lot cleaner than before overall and seems like just two files anyway.

@BuckarooBanzay
Copy link
Member Author

I'd be happier if test specific stuff would be in some sub dirs dedicated for tests, I mean similar how it was.

There's only the mtt.lua and registered_nodes.txt in the "main" technic mod-folder, i could move them to a "test" directory if you want...

@OgelGames
Copy link
Contributor

OgelGames commented Apr 22, 2024 •
edited
Loading

I think it would be good to keep the .test directory for those files, even though it's only two; it keeps the mod directories cleaner, and they automatically get excluded from releases.

Actually now that I look at it closer, those are only tests for technic, so it would make more sense to keep them in there.

@github-actions GitHub Actions
Copy link

Click for detailed source code test coverage report

Test coverage report for Technic CNC 87.39% in 11/14 files:

File                             Hits Missed Coverage
-----------------------------------------------------
programs.lua                   263  0      100.00%
materials/technic_worldgen.lua 32   0      100.00%
materials/init.lua             14   0      100.00%
materials/default.lua          183  0      100.00%
materials/basic_materials.lua  17   0      100.00%
init.lua                       16   0      100.00%
digilines.lua                  55   0      100.00%
cnc.lua                        53   0      100.00%
formspec.lua                   104  7      93.69%
api.lua                        222  43     83.77%
pipeworks.lua                  25   13     65.79%
materials/moreblocks.lua       0    29     0.00%
materials/ethereal.lua         0    37     0.00%
materials/bakedclay.lua        0    13     0.00%

Test coverage report for technic chests 45.58% in 6/6 files:

File          Hits Missed Coverage
----------------------------------
chests.lua    102  18     85.00%
init.lua      34   18     65.38%
register.lua  85   78     52.15%
formspec.lua  76   94     44.71%
inventory.lua 10   100    9.09%
digilines.lua 2    61     3.17%

Test coverage report for technic 66.88% in 97/98 files:

File                                      Hits Missed Coverage
--------------------------------------------------------------
max_lag.lua                               12   0      100.00%
materials.lua                             71   0      100.00%
machines/register/init.lua                15   0      100.00%
machines/register/freezer_recipes.lua     16   0      100.00%
machines/other/init.lua                   6    0      100.00%
machines/MV/solar_array.lua               12   0      100.00%
machines/MV/init.lua                      17   0      100.00%
machines/MV/grinder.lua                   17   0      100.00%
machines/MV/generator.lua                 9    0      100.00%
machines/MV/freezer.lua                   17   0      100.00%
machines/MV/extractor.lua                 17   0      100.00%
machines/MV/electric_furnace.lua          17   0      100.00%
machines/MV/compressor.lua                17   0      100.00%
machines/MV/centrifuge.lua                17   0      100.00%
machines/MV/cables.lua                    40   0      100.00%
machines/MV/battery_box.lua               17   0      100.00%
machines/MV/alloy_furnace.lua             19   0      100.00%
machines/LV/solar_array.lua               11   0      100.00%
machines/LV/init.lua                      17   0      100.00%
machines/LV/grinder.lua                   17   0      100.00%
machines/LV/generator.lua                 10   0      100.00%
machines/LV/electric_furnace.lua          16   0      100.00%
machines/LV/compressor.lua                21   0      100.00%
machines/LV/cables.lua                    41   0      100.00%
machines/LV/battery_box.lua               15   0      100.00%
machines/LV/alloy_furnace.lua             18   0      100.00%
machines/HV/solar_array.lua               11   0      100.00%
machines/HV/init.lua                      12   0      100.00%
machines/HV/grinder.lua                   17   0      100.00%
machines/HV/generator.lua                 9    0      100.00%
machines/HV/electric_furnace.lua          17   0      100.00%
machines/HV/compressor.lua                17   0      100.00%
machines/HV/cables.lua                    39   0      100.00%
machines/HV/battery_box.lua               17   0      100.00%
legacy.lua                                33   0      100.00%
crafts.lua                                134  0      100.00%
machines/LV/led.lua                       80   1      98.77%
items.lua                                 113  2      98.26%
config.lua                                48   1      97.96%
machines/register/compressor_recipes.lua  41   1      97.62%
machines/register/grindings.lua           39   1      97.50%
machines/LV/geothermal.lua                82   3      96.47%
machines/register/solar_array.lua         48   2      96.00%
machines/LV/solar_panel.lua               44   2      95.65%
machines/network.lua                      403  21     95.05%
machines/register/alloy_recipes.lua       41   3      93.18%
tools/init.lua                            13   1      92.86%
init.lua                                  25   2      92.59%
machines/LV/water_mill.lua                73   6      92.41%
machines/register/cables.lua              109  9      92.37%
machines/compat/tools.lua                 58   6      90.62%
machines/register/recipes.lua             134  14     90.54%
machines/LV/lamp.lua                      131  17     88.51%
register.lua                              28   4      87.50%
machines/register/grinder_recipes.lua     109  16     87.20%
tools/flashlight.lua                      65   14     82.28%
machines/register/battery_box.lua         249  54     82.18%
util/throttle.lua                         9    2      81.82%
machines/init.lua                         38   9      80.85%
machines/LV/extractor.lua                 18   5      78.26%
radiation.lua                             279  81     77.50%
machines/power_monitor.lua                63   19     76.83%
machines/register/centrifuge_recipes.lua  23   7      76.67%
machines/register/machine_base.lua        165  51     76.39%
machines/other/coal_furnace.lua           3    1      75.00%
machines/switching_station.lua            84   32     72.41%
effects.lua                               5    2      71.43%
machines/overload.lua                     12   5      70.59%
machines/MV/wind_mill.lua                 52   22     70.27%
machines/supply_converter.lua             98   46     68.06%
machines/MV/hydro_turbine.lua             49   26     65.33%
machines/other/injector.lua               80   45     64.00%
machines/switching_station_globalstep.lua 28   16     63.64%
tools/multimeter.lua                      130  78     62.50%
machines/register/generator.lua           127  83     60.48%
machines/MV/tool_workshop.lua             64   43     59.81%
machines/other/coal_alloy_furnace.lua     90   62     59.21%
tools/cans.lua                            54   48     52.94%
tools/mining_lasers.lua                   37   35     51.39%
machines/LV/music_player.lua              48   47     50.53%
machines/other/constructor.lua            78   77     50.32%
tools/tree_tap.lua                        25   30     45.45%
tools/vacuum.lua                          16   20     44.44%
machines/register/common.lua              51   64     44.35%
machines/HV/forcefield.lua                111  158    41.26%
machines/HV/nuclear_reactor.lua           139  214    39.38%
helpers.lua                               62   98     38.75%
machines/HV/quarry.lua                    152  250    37.81%
tools/sonic_screwdriver.lua               17   33     34.00%
tools/chainsaw.lua                        39   80     32.77%
machines/compat/api.lua                   16   34     32.00%
chatcommands.lua                          19   45     29.69%
tools/mining_drill.lua                    66   195    25.29%
machines/other/anchor.lua                 16   74     17.78%
tools/prospector.lua                      15   100    13.04%
machines/register/extractor_recipes.lua   8    77     9.41%
machines/compat/digtron.lua               4    41     8.89%
mtt.lua                                   0    6      0.00%

Raw test runner output for geeks:

CNC:

++ Executing suite spec/api_spec.lua
++ Executing suite spec/digilines_spec.lua
++ Executing suite spec/interaction_spec.lua
++ Executing suite spec/technic_cnc_spec.lua⭆ Starting test set technic_cnc_spec.lua:17 ⯈ Technic CNC
W:	Unsupported registration overrides detected for core.registered_on_mods_loaded


31 successes / 0 failures / 0 errors / 0 pending : 14.828262 seconds

Chests:

++ Executing suite spec/api_spec.lua
W:	Configuration: invalid key	exclude_textures


5 successes / 0 failures / 0 errors / 0 pending : 0.361759 seconds

Technic:

++ Executing suite spec/api_spec.lua
E:	Loading common/fs from 5.4.1 failed, trying to use builtin
◌◌⭆ Starting test set api_spec.lua:204 ⯈ Technic API Machine use
W:	Unsupported registration overrides detected for core.registered_on_mods_loaded
◌
++ Executing suite spec/building_spec.lua
E:	Loading common/fs from 5.4.1 failed, trying to use builtin

++ Executing suite spec/hv_network_spec.lua
E:	Loading common/fs from 5.4.1 failed, trying to use builtin
⭆ Starting test set hv_network_spec.lua:11 ⯈ HV machine network
W:	Unsupported registration overrides detected for core.registered_on_mods_loaded

++ Executing suite spec/lv_network_spec.lua
E:	Loading common/fs from 5.4.1 failed, trying to use builtin
⭆ Starting test set lv_network_spec.lua:11 ⯈ LV machine network
W:	Unsupported registration overrides detected for core.registered_on_mods_loaded

++ Executing suite spec/network_spec.lua
E:	Loading common/fs from 5.4.1 failed, trying to use builtin

++ Executing suite spec/nodes_spec.lua
E:	Loading common/fs from 5.4.1 failed, trying to use builtin
⭆ Starting test set nodes_spec.lua:11 ⯈ Technic node placement
W:	Unsupported registration overrides detected for core.registered_on_mods_loaded

++ Executing suite spec/supply_converter_spec.lua
E:	Loading common/fs from 5.4.1 failed, trying to use builtin
◌
++ Executing suite spec/tools_compatibility_spec.lua
E:	Loading common/fs from 5.4.1 failed, trying to use builtin
⭆ Starting test set tools_compatibility_spec.lua:11 ⯈ Technic power tool compatibility
W:	Deprecated technic.register_power_tool use. Setting max_charge for oldlegacy:powertool
W:	Deprecated technic.register_power_tool use. Setting max_charge for oldminimal:powertool
W:	Deprecated technic.register_power_tool use. Setting max_charge for oldhalfway:powertool
W:	Unsupported registration overrides detected for core.registered_on_mods_loaded
W:	Deprecated technic.register_power_tool use. Ensuring fields for oldlegacy:powertool
W:	Using metadata charge values for oldlegacy:powertool
W:	Updated legacy Technic power tool definition for oldlegacy:powertool
W:	Deprecated technic.register_power_tool use. Ensuring fields for oldminimal:powertool
W:	Using metadata charge values for oldminimal:powertool
W:	Updated legacy Technic power tool definition for oldminimal:powertool
W:	Deprecated technic.register_power_tool use. Ensuring fields for oldhalfway:powertool
W:	Mod oldhalfway seems to be aware of technic.plus but oldhalfway:powertool is still using deprecated registration, skipping meta charge compatibility.
W:	Updated legacy Technic power tool definition for oldhalfway:powertool
🢆 Running tests from tools_compatibility_spec.lua:92 ▷ Technic power tool compatibility oldlegacy:powertool can be used (minimum charge)
W:	Use of deprecated function technic.set_RE_wear with stack: oldlegacy:powertool
🢆 Running tests from tools_compatibility_spec.lua:106 ▷ Technic power tool compatibility oldlegacy:powertool can be used (minimum charge + 1)
W:	Use of deprecated function technic.set_RE_wear with stack: oldlegacy:powertool
🢆 Running tests from tools_compatibility_spec.lua:92 ▷ Technic power tool compatibility oldminimal:powertool can be used (minimum charge)
W:	Use of deprecated function technic.set_RE_wear with stack: oldminimal:powertool
🢆 Running tests from tools_compatibility_spec.lua:106 ▷ Technic power tool compatibility oldminimal:powertool can be used (minimum charge + 1)
W:	Use of deprecated function technic.set_RE_wear with stack: oldminimal:powertool

++ Executing suite spec/tools_spec.lua
E:	Loading common/fs from 5.4.1 failed, trying to use builtin
⭆ Starting test set tools_spec.lua:13 ⯈ Technic power tool
W:	Unsupported registration overrides detected for core.registered_on_mods_loaded


219 successes / 0 failures / 0 errors / 4 pending : 99.004207 seconds

Pending → spec/api_spec.lua @ 133
Technic API Machine registration registers my_mod:my_battery
spec/api_spec.lua:133: Battery box registration does not include all fields

Pending → spec/api_spec.lua @ 190
Technic API Machine registration registers my_mod:machine_base
spec/api_spec.lua:190: Base machine registration does not include all fields

Pending → spec/api_spec.lua @ 285
Technic API internals technic.cables TBD, misleading name and should be updated
spec/api_spec.lua:285: TBD technic.cables naming and need, see technic networks data for possible options

Pending → spec/supply_converter_spec.lua @ 78
Supply converter building overloads network
spec/supply_converter_spec.lua:78: overload does not work with supply converter

@BuckarooBanzay BuckarooBanzay merged commit 814ebd7 into master Apr 26, 2024
6 checks passed
@BuckarooBanzay BuckarooBanzay deleted the mtt-action branch April 26, 2024 14:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@OgelGames OgelGames OgelGames left review comments

@S-S-X S-S-X S-S-X approved these changes

Assignees
No one assigned
Labels
Cleanup Cleanup of bad code or other redundant/unnecessary stuff Enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@BuckarooBanzay @OgelGames @S-S-X