refactor(tools/rand): speed up random string generation, take 2

[bungle]

Summary

This PR optimizes the kong.tools.rand.random_string.

Results can be seen in PR that this PR replaces:
#13150 (comment)

KAG-4704

[fffonion]

left my comment in the old PR for archive purpose, but also posting here #13150 (comment)

Edit (by @bungle) copying it here:

I'm not referring the openssl rand_bytes vs. /dev/urandom is securer or weaker, but it depends on our usage.
There are three points makes read from /dev/urandom + base64 not a good design:

We have to use math.rand to map extra character range +, /, thus the output is a mix of cryptographic secure source and non cryptographic secure source.

In our code, we have places we just need random strings, thus it doesn't have to be CSPRNG. We can just use math.random to generate fast and non cryptographic secure random strings.
Use of /dev/urandom is not FIPS compliant (unless the OS itself is FIPS compliant), as it uses non approved algorithm, if the output is used as secret.

So I'm just suggesting to have a pure lua math.rand flavor and a openssl rand bytes variant (no encode_base64, just use to_hex) to random_string. Something like random_string(non_csprng) and default to generate CSPRNG, and can fallback to fast math.random if need high performance (for example, generate trace IDs).

[bungle]

We have to use math.rand to map extra character range +, /, thus the output is a mix of cryptographic secure source and non cryptographic secure source.

Yes, the + and / are really rare. They usually occur none to 2 times in most of the cases. I think this was a compromise. An original implementation was like this:

return v4_uuid():gsub("-", "")

I think original idea was to make it more secure because there was issues reported because of above (the same issue is with the proposed math.rand as is it almost the same).

In our code, we have places we just need random strings, thus it doesn't have to be CSPRNG.

Right but original commit definitely wanted to make it harder for reason, this is the commit:
c01752a

So I'm just suggesting to have a pure lua math.rand flavor and a openssl rand bytes variant (no encode_base64, just use to_hex) to random_string.

We originally had almost the pure math.rand flavor, which the commit referenced above wanted to change for reason.

I think this is out of scope of this work. We can have separate PR for that, and also discuss should we have variable length ones in separate PR. The code here does not change things. It speeds it up. The to_hex effectively cuts the random string entropy to half. With wider char space like base62 you don't do it as much (e.g. here we almost retain the 192 bits of entropy, not fully in all the cases, but almost - with hex you would need longer output string).

I suggest we put better ones in PDK. The better ones could take in account:

1. maximum entropy in shortest amount of text chars (baseXXX?)
2. copy-pasteability (does double click select the whole thing?)
3. readability (e.g. do not use chars that looks like each other)
4. CSPRNG vs non-CSPRNG
5. variable length outputs
6. ascii / utf-8 safety, perhaps specific use cases like idna etc.
   etc.

In general there are many things to consider.

[team-gateway-bot]

Successfully created cherry-pick PR for master:

• https://github.com/kong/kong-ee/pull/9436

[fffonion]

We originally had almost the pure math.rand flavor, which the commit referenced above wanted to change for reason.

I understand. But the commit reference doesn't actually make it any securer. The usage or base64 + rand mapping we
are trying to improve in this PR is not valid at first. So I was suggesting to just abandon the current design and invest
effort into properly design it, at least to solve the use cases in kong codebase. And it will make our FIPS mode more robust
too. I created a ticket to track the future work on this.