Scope of changes to random

[Shimuuar]

This issue is quite meta. But I think it's important to understand what is going on. Is goal just to make random not terribly inefficient and use better PRNG? Is it to create new generic API for working with random numbers which could be used by most of performant generators? I hope latter is on the table since ecosystem is terribly fragmented.

[idontgetoutmuch]

This issue is quite meta. But I think it's important to understand what is going on. Is goal just to make random not terribly inefficient and use better PRNG? Is it to create new generic API for working with random numbers which could be used by most of performant generators? I hope latter is on the table since ecosystem is terribly fragmented.

Both :) in my opinion

[curiousleo]

#3 (comment):

Portability is actually one of the reasons I think such API should be added. mwc-random has way to initialize generator using /dev/random on *nixes and windows' crypto API on windows. It took several contribution in order to gent working right.

It's not reasonable to expect that PRNG implementors will jump through flaming hoops in order to get good platform independent initialization. It's complicated thing and thus should be centralized. random seems to be a good place for that

#3 (comment):

If we can agree on reliable solution for this (I haven't looked at the one in mwc-random too closely yet), I agree that random package would be a really good home for it.

Related code is here: http://hackage.haskell.org/package/mwc-random-0.14.0.0/docs/src/System.Random.MWC.SeedSource.html#acquireSeedSystem

I'd like to voice an opinion about scope here:

• In scope: making sure that random can provide functions to seed any RandomGen easily using a system randomness source, e.g. by making sure that RandomGen and perhaps Seed typeclasses are defined to allow for this.
• Out of scope: actually implementing the system randomness source right now.

I agree with the comments linked above that this would be nice to have and would simplify PRNG implementors' lives, and I think it would be worth looking at as a follow-up to our main work here, but it is quite a separate concern from designing a performant and easy to use RandomGen and MonadRandom.

[lehins]

I think a good scope for this whole proposal would be:

• Unify interface of pure RNGs with stateful through a monadic interface (Explicit vs implicit state passing #3)
• Improve performance and quality of default number generation.
• Replace implementation of default StdGen with SMGen, by simple direct dependency on splitmix package Use SplitMix #2
• Possibility of unifying interface with CRNGs and generation of array of bytes, not just primitive values
• Export helper methods for creating instances (eg. defaultSplit, bitmaksWithRejection etc.) and using the library (eg, randomList)

One other important goal is to minimize breakage and for things that do break we should have a clear plan forward on how fixing them. So what sort of breakage is acceptable:

• Expected breakage:

  • Wrappers that provide a different interface (eg. MonadRandom and random-source). Current interface should completely replace those, so there is no need for a plan on how to fix them, but a short guide on how to migrate from them would be really good to have,

• Acceptable breakage:

  • Implementations of RNGs (eg, pcg-random, mwc-random, etc.) will need to have PRs submitted to in order to have instances fixed/implemented, so they are ok to break
  • Libraries that generate other distributions (eg. random-fu). These are advance users of random package therefore some breakage is expected, but again with clear plan on how to fix them (at least the most important ones).

• Reduced breakage:

  • Libraries that directly utilize random to generate other types of random data should preferably not break and continue utilize the same interface (eg. QuickCheck)

• No breakage.

  • User code
  • Packages that simply need some random values (eg. usages of random, randomR and split functions)

  Adding DEPRECATE pragma here is desired for all functions that are planned to be deprecated, so users can can act upon that deprecation over the course of a few subsequent version upgrades of random.

[lehins]

Here is a potential solution for drawing system randomness in a platform independent manner: https://hackage.haskell.org/package/entropy Haven't tried the package myself, but from a quick look at it gives me an impression that this is exactly what we need.

[Shimuuar]

AFAIR it have (had?) problem as source of entropy for cryptography. At least at some point it used hardware RNG which many presumed compromised

[curiousleo]

AFAIR it have (had?) problem as source of entropy for cryptography. At least at some point it used hardware RNG which many presumed compromised

Looks like there was a relevant change in v0.4:

Get a specific number of bytes of cryptographically secure random data using the system-specific sources. (As of 0.4. Verions <0.4 mixed system and hardware sources)

From https://hackage.haskell.org/package/entropy-0.4.1.5/docs/System-Entropy.html#v:getEntropy

[Shimuuar]

There's also inherent tension between MC purposes and crypto purposes. Former will gladly trade cryptographic security for speed. Latter not so much

[lehins]

@Shimuuar I mostly care about platform independence of drawing randomness from the OS. From the looks of it the entropy package has quite a bit of this figure out, much more than mwc-random has. Regardless if we gonna use it or not, I think both entropy and mwc-random can be used as a great source for ideas and bits and pieces of implementation for getting this issue tackled.