codecov.io is flaky

[yuvaljacoby]

Uploading coverage report fail (pretty often) and it seems to be a known problem (for example)

This cause the badge coverage to be wrong (it has only the python API section without the manualy created c++ report)

Two possible solutions that I can think of:

1. Use multiple retries when uploading
2. Use another service (CVC4 are using a local service, and not a public one)

Thoughts? comments?

[kjulian3]

Is it the curl command that fails? Right now it's set up to echo a failure message if curl fails so that the build doesn't completely fail. We could try putting curl in a for loop to try multiple times to upload before giving up and echoing a failure message.

[kjulian3]

I just merged in a pr that changed code comments (no actual code changes), and curl timed out:
https://travis-ci.com/github/NeuralNetworkVerification/Marabou/jobs/356659596

And you're right, now code coverage is at "97%" because it's only reporting the Python API. We can rerun the travis build to fix it (most likely), but it would be nice to have a better solution.

Some users have strung a bunch of upload attempts together:
codecov/codecov-python#158 (comment)

Other users have reported that coverage upload can still fail after multiple attempt in a row:
https://community.codecov.io/t/unreliable-coverage-report-uploads/322/8

This is disappointing. Seems like this has been an issue for a long time now, and for a lot of people, and they haven't fixed it. If there's no solution, then perhaps it's better to use another service.

[idan0610]

For now it seems like upload to codecov works well, so I close this issue