Default timeout too large?

[antoine-levitt]

The current default timeout of five seconds is annoyingly large for simple benchmarking (if I want to benchmark four different versions of a simple loop, I have to wait 20s). How about reducing it to 1s? The rationale is that 1s is largely sufficient for a simple microkernel benchmark, and longer tests should probably not do a default @btime anyway.

[gdalle]

@antoine-levitt how breaking do you think this change would be?

[antoine-levitt]

This is a user-facing tool more than a library so I would imagine the notion of breaking is weaker here, but I guess in the end it depends on how strictly you believe in semver

[willow-ahrens]

I think it would be fine to lower this to two seconds. I have a benchmark suite for Finch that we can test on and see how we feel regarding the change in variance. We may want to add actual hypothesis testing as in ##74 first.