Atom generator exhausts the atom limit

[Maria-12648430]

The atom() generator produces atoms at random, with no limitations whatsoever. By extension, this is also true for union generators like term(). As atoms are never garbage collected, this means that over time the limit on the number of atoms in a runtime instance will get exhausted. How quickly that happens depends on how generators are used, how many properties there are in a suite, and how many suites there are.

We are working on property tests for OTP, currently for the lists module. We have 71 properties so far and are not done yet, but we already encounter failures because the atom space in exhausted. Yes, we can tune our generators to generate atoms more economically (cumbersome and makes tests more artificial), but still, this puts a tight lid on things in general. Going to extremes, it would never be possible to equip all of OTP with property tests, not even all of stdlib.

[kostis]

Atom garbage collection is something known to be missing from Erlang/OTP's implementation for many years now, and a limitation which has forced most Erlang developers to write ugly and/or unnatural code in many of their programs. Moreover, the implementation of an atom garbage collector is not rocket science. So, perhaps it's high time that it is implemented.

If property-based testing is something that is of use to Erlang/OTP and your particular way of using the atom()/term() generator causes trouble for OTP, perhaps this may give some extra incentive to the OTP folks to bite the bullet and finally implement atom garbage collection.

In my opinion, artificially limiting the atom() and term() generators to e.g. just the existing atoms is not the PropEr solution to the issue(s) you are facing. But I am willing to be convinced otherwise.

[Maria-12648430]

Well, I have no real idea how that works internally, I have my guesses and some cloudy memory. But I think it's not just a simple matter of not wanting to bother of cleaning up after yourself 😜 And I don't think my particular way of using atom() and term() will induce the OTP team to go down that road, despite them being in favor of property tests 😉

That said, personally I didn't feel any need for more or infinite atoms, and I would second-guess any production code that needs it.

On the other hand, I understand that limiting the generators is contrary to what property testing is really about: generating all sorts of possible input. But I don't see how the two will be brought together 🤷‍♀️

[Maria-12648430]

FWIW: https://erlangforums.com/t/on-garbage-collecting-atoms/1476

[Maria-12648430]

On a side note, an interesting point that came up in the discussion was that atoms generated purely at random alone are not all that useful. There are certain "interesting" atoms (well-known ones like ok, error, undefined etc, module and function names, etc) that are likely to trigger something but are rarely generated with just a random generator.