Scraper failed to make a raw data dump on some occasions

[bepetersn]

This occurred on June 27th. Somehow, the completion (or creation) of the data dump did not happen, despite the fact that the scraper itself seemed to have completed its run. However, I judged this in a cursory albeit empirical manner based on there being no unusual errors that stood out in the log file for the day, and it may not be true. I will look into this some more.

In the near future, when #448 is done, we can recreate all the data that does exist in the v1 database (201 inmate records, ... an amount also not unusual).

[nwinklareth]

The June 11th scrape was also unsuccessful. The temp files still exist in
the website/scratch/scraper/raw_inmate_data directory. They look to be
smaller than then the normal ones. If you examine the log files for these
days, which by the way are June 12th and 28th respectively you will find
that there are index out of range errors exceptions thrown from
InmateDetails.

On Mon, Jun 30, 2014 at 6:55 PM, Brian Everett Peterson <
[email protected]> wrote:

This occurred on June 27th. Somehow, the completion (or creation) of the
data dump did not happen, despite the fact that the scraper itself seemed
to have completed its run. However, this was empirically judged based on
there being no unusual errors noted in the log file, and may not be true. I
will look into this some more.

In the near future, when #444
#444 is done, we can
recreate all the data that does exist in the v1 database (201 inmate
records, ... not an unusual amount).

—
Reply to this email directly or view it on GitHub
#452.

Regards

Norbert

Norbert Winklareth