Naming conventions discussion

[CosmicToast]

There's general interest in improving the naming conventions and the consistency across the stdlib.
Let's discuss what they should be.

Factors to consider:

• Documentation locality: functions that do similar/related stuff should ideally have their names be adjacent in an alphabetically sorted list. For example, pretend we have sort (new copy) and sort! (in-place). They will be next to each other when sorted.
• Discoverability: a user should be able to think "I want to do X", and ^f to find the function by name. If not by name, then by description.
• Consistency: there should be as many paradigms in use as possible, so that users can predict the existence of other, related functions, where having them next to each other is impossible.

We should cover at least:

• Mutable/Immutable function types (sort / sorted / sort!).
• Boolean / return-the-value function types (any / any? vs or).
• Any theming (freeze/thaw vs (im)mutable).

Of additional note: in 2021 bakpakin explicitly said he doesn't want too many !s in the stdlib.
Whether this is still the case is up to him.

Most !-type functions can be converted into immutables fairly easily, however:

1. Make a (mutable) copy.
2. Run the ! function.
3. Return the frozen version.

Prior art: #626 #1248

Will edit any of the above if something new is brought to my attention.

[CosmicToast]

Initial Feelings

While this may be biased and likely won't make it in, for the purposes of discussion, I'll write down my thoughts.
I, personally, prefer the sort and sort! approach.
There are a few reasons for this.

The locality is roughly perfect, consider the following example:

(sorted ["sort" "sort-by" "sort-by!" "sort!"])
# => @["sort" "sort!" "sort-by" "sort-by!"]
(sorted ["sorted" "sorted-by" "sort-by" "sort"])
# => @["sort" "sort-by" "sorted" "sorted-by"]

With ! glyphs, the mutable version is always next to its immutable version.
On the other hand, with an "ed" suffix, they are separated.
Additionally, "sorted" makes it less clear what's going on, as it still returns a mutable data structure.

Furthermore, typically, "variants" will have a -something or a glyph added, rather than simple added characters.
It may also be desirable to enforce this.
Of particular note:
(sorted ["-" "*" "!"]) # => @["!" "*" "-"]
This means that the ! (in-place variant glyph) will be the first of the common suffixes, followed by "*", (functional variant for macros), followed by "-" (other functions, which in turns will have their own perfect locality in place).

Initial Proposal

For the entirety of this, when you see "function", this typically includes "macro", unless those are explicitly mentioned.

1. Every function in the stdlib that can accept multiple types will return the exact type it accepted.
   1.1 To achieve this, every such function will have a corresponding type/name function. These exclusively return type, unless it's performing some other clear operation (e.g "length").
   1.2 Those are then dispatched dynamically based on the type. So, for instance, (foo "string") will be called (string/foo "string"). Note that this is fairly easy to implement and extend with a macro. I can write an example macro just to demonstrate it. It would look something like (dispatch length [ds]) or (dispatch sort! [ds] :not [:tuple :buffer]).
2. Functions that have a mutable and unmutable variant will be called with a ! suffix for the in-place variants.
3. Macros that should have a functional variant will have the functional variant be called with a "*" suffix.
4. Functions that iterate over data structures will check the callback signature, allowing for arities of 1 and 2. An arity of 1 means iterate by value, while an artiy of 2 means iterate by key-value.
5. Functions that return a boolean value, and only those functions, will end in a "?".

These additional ideas I have in the back of my mind, and should not be considered part of the concrete proposal, but discussion items:

1. It is common to have functions that return an index. What if those were suffixed with a @? (Suffixes are good due to locality, see above). Notably, "@" sorts after "-".
2. It may be a good idea to delineate functions that return a value (e.g current or/any). I'm not sure what it should be, however. No particular glyph screams, "not a boolean" to me. A different potential way would be to have them return a default value, which would default to nil (thus no need to initialize it), which is false.

Additional Context

Based on the prior art, it definitely seems like Janet has a bit of an identity question.
On one hand, immutable data structures exist.
Tuples and strings especially are commonly used.
On the other hand, however, actually using them outside built-ins is awkward.

You cannot assoc (or similar) something into a "struct", and most of the stdlib will happily give you back mutable objects.
Similarly, prototypes are heavily limited on structs, being un-nestable.
This means that smalltalk-style message forwarding on static data is difficult, limiting the usefulness of the system.

I've previously tried to remedy this at least in the C API by making utilities that let you have a c-string out of both buffers and strings, but since buffers aren't guaranteed to be zero-terminated, that was suboptimal.
Such "gotchas" are fairly common because, as of currently, Janet is a mutability-first language.
This is doubled by the sentiment of not wanting to pepper !s everywhere, which comes from a practical observation: most of the stdlib is already doing almost exclusively mutable things.

One advantage of enforcing the above is to make it more of an equal playing-ground.
If every mutable function has an immutable variant, it becomes less of a pain.

Interesting Side-Effects

Since functions definitionally return their original type, immutable functions like put (would-be assoc) operate on mutable types.
So:

(def a @{:a :b})
(def b (put a :a :c))
# a is @{:a :b}, b is @{:a :c}: still mutable

It also means that mutable functions don't need to worry about many things, since they definitionally only accept mutable inputs, and therefore can never dynamic-dispatch to an immutable function type, and thus will never receive back an immutable type.
Furthermore, this type of dispatch can be a macro, since it's perfectly predictable with this scheme.
Since Janet can actually use macros as HOF, this is a really interesting advantage over a lot of other functional languages!

[primo-ppcg]

most of the stdlib will happily give you back mutable objects

Most functions that return mutables do so because the result was pieced together. I agree that it would be more elegant to return the same type if an immutable type were received.

Perhaps an interesting idea: strings and buffers, tuples and arrays, and structs and dictionaries could be actually the same data type, with an immutability flag set. With a fast analogue to freeze (that doesn't walk the entire ds), I think this could be a viable option. It would likely reduce the size of the executable significantly as well.

[CosmicToast]

Most functions that return mutables do so because the result was pieced together.

I'm aware ^^
This does though, as mentioned, strongly push people towards a mutable-only workflow.
Nobody wants to separate each step with a freeze.
In fact, an interesting experiment would be to make a secondary janet-only library that simply redefines most of the stdlib as (comp freeze func) (with whatever logic is needed to make them accept immutable types).
Not what the stdlib should be, mind you, just an experiment.

I agree that it would be more elegant to return the same type if an immutable type were received.

It's also what people expect, typically, which means it's at least positive.

Perhaps an interesting idea: strings and buffers, tuples and arrays, and structs and dictionaries could be actually the same data type, with an immutability flag set. With a fast analogue to freeze (that doesn't walk the entire ds), I think this could be a viable option. It would likely reduce the size of the executable significantly as well.

And then freeze can walk the entire ds but apply the fast analogue, yeah.
There are some details to iron out though.
For instance, buffers are not guaranteed to be null-terminated (strings are).
Which one will be the "backing" type? (would be nice if it was the string... but then what about other uses for the buffer?)

Regardless, that's probably a completely separate discussion.

[bakpakin]

Similarly, prototypes are heavily limited on structs, being un-nestable.

Prototypes on structs are nestable, not sure what you mean here.

The reason mutable and immutable data structures exist is really more about value vs reference semantics - immutable values can be used as keys in tables, not about functional programming in the sense of using HAMTs for everything. Unfortunately, we get a lot of comparisons to Clojure, which I think makes some overly prescriptive choices about how to program effectively. Most programs I write do indeed use mutation to build up a final result.

The freeze idea is interesting though, this is basically Clojure transient data types.

[nstgc]

Just throwing my 2-cents in, but I very much like how Julia adds ! after "unsafe" functions. I hadn't considered it from the perspective of "locality", but it kind of shifts my thinking from the default "math-mode" to whatever you want to call the mode I'm in while programming in C.

[primo-ppcg]

Common functions with uncommon names

The primary example for this is reduce, which in most functional programming languages would be called fold (this is at very least mentioned in the documentation, so it can be found ^F. There are certainly also worse choices). (EDIT: this appears to be Lisp* convention, so it's probably best as is.) It's inverse, unfold (repeatedly apply a function to a value or values, accumulating the results while a predicate holds) is nowhere to be found. I'm not even sure what it would be called; reduce doesn't have a clear antonym. unreduce maybe?

> (unfold |(< $ 100) + 0 1)
@[0 1 1 2 3 5 8 13 21 34 55 89]

Super useful, a shame it's not there.

Uncommon functions with common names

The primary example for this is chr (Convert a string of length 1 to its byte (ascii) value at compile time). I understand its utility, but it probably should have been called byte instead. chr is universally understood to perform the inverse operation, at runtime:

> (chr 65)
"A"

In a variadic form, chr could have been very useful as well. Converting a byte array to a buffer is a fairly common operation. The best we have at the moment is this:

(buffer/push @"" ;(range 65 91))
# vs.
(chr ;(range 65 91))

[primo-ppcg]

May be it's a lispy thing

I'm happy to concede the point. I would, however, like suggestions for a good name for unfold 😉

[sogaiu]

I don't have a good immediate answer to that.

There was the following train of thought though...

• accumulate and accumulate2 do something kind of related (to unfold) though termination appears to be expressed based on running out of things to process rather than a condition that is checked along the way
• @ianthehenry's remark below mentioned that accumulate is called scan or folding-map elsewhere

Some searching with the aforementioned terms turned this up. I noticed mention of iteratee in one of the answers. My stack is overflowing ATM though so I'm going to pause on this for now -- may be someone else will figure out something useful :)

---

Remembered reduced from Clojure. There's a SO question with some discussion that goes into it a bit. Mentioning it as something possibly of interest.

[primo-ppcg]

Some searching with the aforementioned terms turned this up.

That would be more of an accumulate-while or accumulate-until. In janet, seq already fills this use case sufficiently, I think.

Remembered reduced from Clojure.

Yet another reminder of how much better Janet is than Clojure 😉

---

These are three different functions:
reduce: array -> scalar
accumulate: array -> array
unfold: scalar -> array

A simple implementation might be:

(defn unfold
  [pred f & xs]
  (def len (length xs))
  (def res (array ;xs))
  (var val (f ;xs))
  (while (pred val)
    (array/push res val)
    (set val (f ;(take (- len) res))))
  res)

(pp (unfold |(< $ 100) + 0 1))

while (pred val) I suppose is closely related to :iterate, perhaps it could be called just iterate.

[sogaiu]

A bit more digging around turned up:

• "Unfolds" mentioned in a page on Fold at Wikipedia
• The Under-Appreciated Unfold paper
• SRFI 1's Fold, unfold & map section -- from the Scheme universe -- mentions an unfold

[pepe]

Yet another reminder of how much better Janet is than Clojure 😉

❤️

[ianthehenry]

The Janet name for function that I would expect to be called scan or folding-map is called accumulate. I'd written it a few times before I realized it was already in the standard library.

In OCaml's Base library, reduce means "fold with no initial element," and requires a collection with at one element. (reduce list f = fold (hd list) (drop 1 list) f basically). I haven't seen this terminology anywhere else; I'm used to reduce and fold being synonyms (though I prefer fold). But maybe this is a thing.

There's a recent naming choice in the current stdlib that is inconsistent: keep-syntax and keep-syntax!. keep-syntax creates a copy of an input tuple with a different sourcemap. And even though tuples are usually immutable, their sourcemaps are mutable, so you might expect that keep-syntax! is the mutable counterpart. But it's not: the ! means "coerce the types," which is not what ! means in the one other place it's used in the stdlib. There's no mutable counterpart in the stdlib.

[primo-ppcg]

In OCaml's Base library, reduce means "fold with no initial element," and requires a collection with at one element. (reduce list f = fold (hd list) (drop 1 list) f basically). I haven't seen this terminology anywhere else; I'm used to reduce and fold being synonyms (though I prefer fold). But maybe this is a thing.

Scheme apparently does the same. I'm not sure if it would be a good idea to rename reduce to fold (with current reduce2 as reduce), and accumlate to scan (with current accumulate2 as accumulate), although there is certainly precedent for these.

[CosmicToast]

That's a great point Ian, thanks!
I think keeping the lispy reduce but adding the argumentful fold and thus unfold may be advisable.
It does mean unfold is elsewhere on the page for generated docs, but at least you can ^f it.

[bakpakin]

Sorry to spoil the bike-shedding fun, but we are absolutely not renaming reduce to fold, or any of these other proposals, especially since these are functions that have been there since the beginning. These all strike me as arbitrary changes, copying arbitrary decisions from other languages that are no more consistent or correct.

Also yes, I still don't want ! littered everywhere in the standard library. While I'm not totally against it, it is still a form of hungarian notation (ad-hoc typing) that I dislike.

As far as strict rules as have been proposed here, they will not be maintained over time unless they can be mechanically enforced (or are trivial to enforce). Even in the Clojure/Lisp world, I have seen debate on whether or not a ? suffix is a predicate function or any boolean function.

Such "gotchas" are fairly common because, as of currently, Janet is a mutability-first language.

Yup. I don't plan on adding assoc to the language because people will try to write horribly inefficient code with it. Even things like take and drop I don't particularly like but I suppose are fairly useful for simple scripts, and usually are done only once at the end of a calculation. assoc is often used to build up data in a loop (or recursively if we are being fancy) resulting in quadratic time and memory usage.

I do think the standard library could be improved, but it would be more related to things like redoing the module system to be simpler, removing functionality like juxt and juxt* that I rarely use and rarely see used, and generally undoing a lot of Clojure-isms rather than leaning harder into them.

[sogaiu]

Just having fun below using the existing ability to use a struct as metadata:

(defn my-fn
  {:args [Number Number Number]
   :ret Number}
  [x y z]
  (+ x y z))
  
# would not type check: String != Number
(defn my-fn-2
  {:args [String String]
   :ret Number}
  [a b]
  (+ 10 (my-fn a b 20)))

[primo-ppcg]

Don't forget the docstring:

(defn my-fn
  {:doc "Returns the sum of three numbers."
   :args [:number :number :number]
   :ret :number}
  [x y z]
  (+ x y z))

[sogaiu]

Ah, may be that's nice for one-liners.

Keywords for signature things seem nice.

[sogaiu]

really any tool that can help verify and test Janet programs automatically (generative testing is also cool).

I like property-based testing [1] and I'd prefer to have something that has good automated shrinking.

I'd looked into Hypothesis sometime back and although I was able to apply it to a project [2], I failed to understand it well enough to want to attempt an implementation in Janet.

Interestingly, I came across this article today:

https://getcode.substack.com/p/property-based-testing-6-random-all

It describes another allegedly simpler approach which has been implemented here.

May be that's worth considering...

---

[1] Haven't knowingly done a lot of other kinds of generative testing...

[2] Unfortunately not Janet code -- though interestingly enough, not Python either :)

[sogaiu]

To follow up on this, here is a repository with a translation attempt into Janet of code from the article.