Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Comstock Integration Tests #344

Draft
wants to merge 70 commits into
base: develop
Choose a base branch
from
Draft

Comstock Integration Tests #344

wants to merge 70 commits into from

Conversation

nmerket
Copy link
Member

@nmerket nmerket commented Feb 3, 2023

Fixes #327.

Pull Request Description

Adds integration testing with ComStock.

Checklist

Not all may apply

  • Code changes (must work)
  • Tests exercising your feature/bug fix (check coverage report on Checks -> BuildStockBatch Tests -> Artifacts)
  • Coverage has increased or at least not decreased. Update minimum_coverage in .github/workflows/ci.yml as necessary.
  • All other unit and integration tests passing
  • Update validation for project config yaml file changes
  • Update existing documentation
  • Run a small batch run on Eagle to make sure it all works if you made changes that will affect Eagle
  • Add to the changelog_dev.rst file and propose migration text in the pull request

@github-actions
Copy link

github-actions bot commented Feb 3, 2023 •
edited
Loading

File Coverage
All files 24% ❌
base.py 51% ✅
eagle.py 0% ❌
exc.py 57% ✅
local.py 61% ✅
postprocessing.py 48% ✅
utils.py 33% ✅
sampler/base.py 65% ✅
sampler/downselect.py 20% ❌
sampler/precomputed.py 90% ✅
sampler/residential_quota.py 33% ✅
test/conftest.py 28% ❌
test/test_base.py 0% ❌
test/test_docker.py 0% ❌
test/test_eagle.py 0% ❌
test/test_integration.py 49% ✅
test/test_postprocessing.py 0% ❌
test/test_utils.py 0% ❌
test/test_validation.py 0% ❌
workflow_generator/base.py 36% ✅
workflow_generator/commercial.py 28% ❌
workflow_generator/residential.py 10% ❌
workflow_generator/residential_hpxml.py 12% ❌
workflow_generator/test_workflow_generator.py 0% ❌

Minimum allowed coverage is 33%

Generated by 🐒 cobertura-action against 734a8c5

@nmerket nmerket mentioned this pull request Feb 16, 2023
Merged
15 tasks
nmerket and others added 24 commits February 23, 2023 13:43
@nmerket
removing schedules fix
526fe58
@nmerket
Merge branch 'kill-docker-singularity' into comstock-integration-tests
be5ff80
@nmerket
switching over custom gems to not use docker on local runs
845d31c
@nmerket
adding OS 3.4.0 for comstock tests to ci
218e618
@nmerket
trying to put in the absolute path
921916a
@nmerket
fixing typo in ci.yml
d15bede
@nmerket
fixing environment variable again
7255c71
@nmerket
adding setup ruby to ci
ee90c44
@nmerket
adding str() around path objects
e800691
@nmerket
saving gem list logs on failure
8d8a963
@nmerket
only doing python 3.11 while debugging ci
0e8c062
@nmerket
using pwd instead to get absolute path
cb73f29
@nmerket
installing a matching version of bundler
7752b8b
@nmerket
adding saving of simulation outputs for failed test
cc6d9f1
@nmerket
only doing the one test to save time
734a8c5
@nmerket
Merge remote-tracking branch 'origin/develop' into comstock-integrati…
96a369b 
…on-tests
@nmerket
Revert "only doing the one test to save time"
2ec2967 
This reverts commit 734a8c5.
@nmerket
adding ssh command to hopefully debug
c23fd8b
@nmerket
just downloading os 3.5.1 instead of installing it
cd59065
@nmerket
Merge remote-tracking branch 'origin/develop' into comstock-integrati…
e0f702d 
…on-tests
@nmerket
adding upterm back in for debugging
43ca36b
@nmerket
commenting out upterm and adding in artifacts for debugging
4ce1ede
@asparke2
Making CI rerun to test new comstock repo changes
52ff85d
@asparke2
Merge remote-tracking branch 'origin/develop' into comstock-integrati…
bb8af2b 
…on-tests
asparke2 and others added 30 commits June 7, 2023 12:18
@asparke2
Mock attempt 4
8bb2b13
@asparke2
Mock attempt 6 by moving openstudio_exe method
af43690
@asparke2
Mock attempt 7
6c8ef27
@asparke2
Revert "Mock attempt 7"
9885414 
This reverts commit 6c8ef27.
@asparke2
Revert "Mock attempt 6 by moving openstudio_exe method"
eca589d 
This reverts commit af43690.
@asparke2
Revert "Mock attempt 4"
fd47cce 
This reverts commit 8bb2b13.
@asparke2
Revert "Revert "Hard-code openstudio 3.4.0 to check comstock tests""
669a894 
This reverts commit 2a54371.
@asparke2
Revert "Hard-code openstudio 3.4.0 to check comstock tests"
452c4bf 
This reverts commit db3859c.
@asparke2
Revert "Mock attempt 3"
e7fac48 
This reverts commit 034d2d1.
@asparke2
Revert "Mock attempt 2"
366363a 
This reverts commit 6e22ffe.
@asparke2
Revert "Another mock attempt"
75f0a0c 
This reverts commit 06561d4.
@asparke2
Revert "New approach to patch openstudio exe for comstock tests"
50e54aa 
This reverts commit eefef6d.
@asparke2
Revert "More logging of openstudio exe version"
4af5d7d 
This reverts commit 21e959c.
@asparke2
Revert "Logging on openstudio_exe version used"
da7c94a 
This reverts commit 67c5e0a.
@asparke2
Revert "Temporarily report OpenStudio version used in comstock integr…
89ae631 
…ation test"

This reverts commit 2a08219.
@asparke2
Revert "Revert "Making CI rerun to test new comstock repo changes""
991c757 
This reverts commit 251a6d5.
@asparke2
Run ComStock integration tests with default OS version
2edee90
@asparke2
Merge remote-tracking branch 'origin/develop' into comstock-integrati…
9067d28 
…on-tests

# Conflicts:
#	.github/workflows/ci.yml
#	.gitignore
@asparke2
Soften buildstock.csv validation to throw warnings for ComStock compa…
5f9f1a4 
…tibility
@asparke2
Don't specify OPENSTUDIO_EXE in CI tests, use openstudio in path
054e828
@asparke2
Moves local custom gem install outside of resources dir and adds Wind…
e6caa6f 
…ows compatibility
@asparke2
Enable upterm session for debugging
547be98
@asparke2
Move upterm session after local test
df7a465
@asparke2
Adds debugging logging for local gem install
352fc62
@asparke2
More debugging for integration testing
65809a3
@asparke2
Disables most of test_comstock_local for debugging
a02c23c
@asparke2
Specifies Ruby 2.7.2 for CI
ca66921
@asparke2
Revert "Disables most of test_comstock_local for debugging"
7cf2338 
This reverts commit a02c23c.
@asparke2
Use ComStock develop branch
2a26db3
@eringold
Merge remote-tracking branch 'origin/develop' into comstock-integrati…
371f71a 
…on-tests
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Integration Tests for ComStock
3 participants
@nmerket @asparke2 @eringold