HTTPCameraAgent write file change

[davidvng]

Description

• Move the write to file into a try-except block and write in chunks
• Slight change to setting connected for each camera

Motivation and Context

To avoid ReadTimeoutError during writing to file, since the response data is streamed

How Has This Been Tested?

Tested on daq-dev

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)

Checklist:

• My code follows the code style of this project.
• My change requires a change to the documentation.
• I have updated the documentation accordingly.

[BrianJKoopman]

Looks good, though we should be more specific in the error handling.

[BrianJKoopman]

Catching BaseException like this isn't ideal. That catches exceptions that indicate the program should exit like SystemExit and KeyboardInterrupt, which if raised at the right time could make the agent difficult to stop.

With this code block:

shutil.copyfileobj(response.raw, out_file)
# Ensure all data is written to the disk before copying to latest
out_file.flush()
os.fsync(out_file.fileno())

We know the error we were running into is another urllib3.exceptions.ReadTimeoutError on the shutil.copyfileobj line. I don't know that you're going to see that with the new chunk iteration code.

Good practice is to catch and handle specific errors, so my preference here is to except ReadTimeoutError as e, like you do above, with the log indicating a timeout occurred. If needed, catching Exception is better than the broader BaseException, as that doesn't catch those exit exceptions.