Fix doctests

[sol]

This addresses commercialhaskell/stackage#7493.

[sol]

@Bodigrim :set sets options for source files and ghci expressions, while :seti only affects ghci expressions.

1. With :set those three tests were failing, I think due to a reincarnation of https://gitlab.haskell.org/ghc/ghc/-/issues/20670.
2. For doctests you usually want :seti anyway.

[idontgetoutmuch]

Why do the generated random numbers change? I would have thought with a fixed seed we should always get the same random numbers i.e. why this: https://github.com/haskell/random/pull/165/files#diff-fe93ae678c398cfea72ff60186cc2cee27bd752aad99d8fd1ece059c912f0104L645

[sol]

@idontgetoutmuch I don't think the seed was fixed for those examples before this PR.

[lehins]

Why do the generated random numbers change? I would have thought with a fixed seed we should always get the same random numbers i.e. why this: https://github.com/haskell/random/pull/165/files#diff-fe93ae678c398cfea72ff60186cc2cee27bd752aad99d8fd1ece059c912f0104L645

@idontgetoutmuch That is because that example uses global StdGen, which as comment below example points out, is usually not a very good idea:

random/src/System/Random.hs

Lines 647 to 650 in 623cf51

	-- This is an outdated function and it is recommended to switch to its
	-- equivalent 'System.Random.Stateful.applyAtomicGen' instead, possibly with the
	-- 'System.Random.Stateful.globalStdGen' if relying on the global state is
	-- acceptable.

[lehins]

@idontgetoutmuch I am more curious to see floating point number generation be affected by the change of how doctests are being executed. I suspect this is coming from some ghc optimizations, so I don't think we'll be able to do anything about it, but it could be worth investigating where exactly it is coming from.

[lehins]

Thank you for fixing doctest for us @sol. I greatly appreciate it.
I could never figure out what was wrong with our current setup.
Could I ask you to change the CI in such a way that we run doctests for many ghc versions, instead of just one. This is quite important for making sure our examples also work the same for users of all supported ghc versions.

[Shimuuar]

In fact we do fix seed for doctests. In module System.Random.Stateful:

-- $setup
-- >>> import Control.Monad.Primitive
-- >>> import qualified System.Random.MWC as MWC
-- >>> writeIORef theStdGen $ mkStdGen 2021

Maybe cabal doctest executes modules in different order?

[lehins]

Maybe cabal doctest executes modules in different order?

I am sure that is the case. Which is also the reason why I always suggest avoid using the global generator whenever reproducability is important.

[juhp]

Are there any release plans, btw?
Asking, since Stackage is still stuck on an older doctest version because of this.

[lehins]

Are there any release plans, btw?

@juhp Yes, I am planning on making a release over Christmas break.

In any case, I suspect stackage will be stuck on this for a while, since last time major version of random was released it took stackage almost a year to adopt it (all due tons of speculative upper bounds in downstream packages: commercialhaskell/stackage#5474)

I can try making a patch release of random-1.2.1.3 that drops usage of doctest, in order to unblock doctes. Would that be helpful?

[juhp]

I can try making a patch release of random-1.2.1.3 that drops usage of doctest, in order to unblock doctes. Would that be helpful?

Yes that sounds good if you can, thank you!

[juhp]

For now I try to disable the random testsuite in Stackage Nightly as a workaround hopefully

[lehins]

Yes that sounds good if you can, thank you!

@juhp random-1.2.1.3 that does not depend on doctest is up on Hackage.