RobotsAndSitemapTest fails at random times

[aecio]

Integration test RobotsAndSitemapTest is non-deternministic and fails at times in the continuous integration server. Example stacktraces:

focusedCrawler.integration.RobotsAndSitemapTest > test1ToNotToDownloadSitesDisallowedOnRobots FAILED
    java.lang.AssertionError: URL=http://127.0.0.1:1234/disallowed-link-1.html
    Expected: is not a value less than <0.0>
         but: was <-149.0>
        at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:20)
        at org.junit.Assert.assertThat(Assert.java:956)
        at focusedCrawler.integration.RobotsAndSitemapTest.assertWasNotCrawled(RobotsAndSitemapTest.java:173)
        at focusedCrawler.integration.RobotsAndSitemapTest.test1ToNotToDownloadSitesDisallowedOnRobots(RobotsAndSitemapTest.java:122)

focusedCrawler.integration.RobotsAndSitemapTest > test1ToNotToDownloadSitesDisallowedOnRobots FAILED
    java.lang.AssertionError: URL=http://127.0.0.1:1234/disallowed-link-1.html
    Expected: is not a value less than <0.0>
         but: was <-101.0>
        at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:20)
        at org.junit.Assert.assertThat(Assert.java:956)
        at focusedCrawler.integration.RobotsAndSitemapTest.assertWasNotCrawled(RobotsAndSitemapTest.java:173)
        at focusedCrawler.integration.RobotsAndSitemapTest.test1ToNotToDownloadSitesDisallowedOnRobots(RobotsAndSitemapTest.java:122)