Semantics of toSignal

[cubuspl42]

What's the semantics of toSignal?

I guess that it's an operator that takes an initial values, and creates a signal that, whenever an event occurs in the source event stream, switches to that event.

Like FRP stepper.

(Image from to Reactive Banana documentation)

But how does toSignal achieve this effect, taking into consideration that both EventStream and Signal are "lazy"? How does it ensure that it doesn't "miss" any event (which would be reflected in its state)?

[raquo]

Correct. The resulting signal can miss events if the events are fired while the signal is stopped. Such a situation is possible if the signal has no observers but the stream has observers other than this signal.

I don't like this, but I'm not aware of how this can be implemented any differently while keeping things lazy since the basic tenet of our memory management is that the parent observable must not have a reference to dependent observables that are stopped.

I also don't know how to make things strict (not lazy) without memory management footguns. Originally we used to have a State type which was like Signal but strict, in simple terms it was its own observer, activated immediately upon creation. But that was basically impossible to work with safely, e.g.

val state: State[S] = ??? // When does this state get released?
val models: List[Model] = ???
val views: List[View] = models.map { model =>
  makeView(model, state.map(st => st.filterByModel(model))) // When does THIS state get released?
}

Hope that makes sense, but I can elaborate if not. I'd be curious to know if there are better solutions to this conundrum.

PS I've just enabled discussions on this github project to this kind of talk separate from actionable bugs and feature requests.

[raquo]

I'm not against typed effects in principle, just on the frontend I don't feel like the tradeoffs are worth it, at least for me.

I can certainly respect wanting to use the same streaming / frp library across your stack, on the frontend and backend. Some people use ZIO for this with Laminar, although they lose some of the safety in doing that (need to be more careful to manually clean up any subscriptions they didn't tell Laminar about).

I don't have the resources to make Airstream into a cross platform lib, and I'm not sure that if I did, that Airstream would be able to maintain its simplicity that I like about it. I made Airstream because I felt the pain of alternative solutions on the frontend. For some reason I didn't really feel that much pain on the backend. Not sure why.

In Scala specifically the problem is also compounded by the JVM being a different execution environment than JS. Airstream is designed for single threaded JS, it doesn't have a notion of parallelism. I honestly don't even want to think about what it would take to make it work on the JVM, which of its contracts become impossible or excessively complicated in such an environment. I will never recuperate my time trying to accomplish that.

[cubuspl42]

I believe I'm lacking most of your practical experience with typed effects, as I wasn't aware that there are differences at applying them at frontend/backend.

I feel obliged to say that my experience with practical applications of purely functional (reactive) programming is very limited. At my work, we develop a commercial mobile application (in Dart programming language and Flutter framework) that uses an in-house quasi-FRP solution to manage state and event propagation. By quasi-FRP I mean a solution that's semantically very close to FRP (Sodium FRP semantics, let's say), but it's full of approximations, to say the least. It's "glitchful" (transaction-less) and uses untyped effects. To be specific, these untyped effects include sampling (reading cell's value at time t) and imperative actions. We don't (currently) change that, because currently it's less costly for us to work-around problems related with these limitations that to fix them.

Personally, I became very interested in Functional Reactive Programming and I'm researching on possible solutions to problems I'm facing at work with out quasi-FRP implementation (out of personal curiosity; just because I find it interesting). So far, my observation is that typed effects would be a possible solution to a few problems problems in FRP:

• Non-semantic (glitchful, one might say) behavior of pure transformational operators when functions passed to them unexpectedly do sampling (reading current state at some time t)
  • Possible solution: typed effect for sampling, i.e. read only state access; let's call it Behavior
• The same goes for imperative actions (what Airstream calls Observers)
  • Possible solution: typed effect for imperative action, i.e. read/write mutable state access; let's call it Action
• The third, possibly the least obvious, is the resource management (conceptually similar to Airstream's ownership system, but putting it into the typed effect; also extended to a few other operatots than just addObserver/listen)
  • Possible solution: typed effect for creating a limited-lifetime entity; let's call it Until

So, to sum up, for me the typed effects are a (possible) solution to the practical problems I find in a real life practical (or "too practical") quasi-FRP system. Aas you probably know better than me, applying typed effects has its own challenges, but so far the only one I know are mostly syntactic. As, in this context, I start from the mostly conceptual point, the programming language wasn't fixed. Haskell has the most powerful syntactic solutions available here (do / mdo), but I was disappointed by its tooling (including the JS-ecosystem related ones, like GHCJS). Now I'm trying Scala and it's "for comprehension" mechanism seems kind-of promising, although I've already learned that there's no way of expressing recursive bindings (i.e., in FRP, creating cyclic definitions).

After taking Scala under investigation, I've searched for existing FRP Scala solutions, and that's how I found Airstream. I was interested in how you solved various FRP implementation challenges, that's why I'm asking these various questions. =)

1. It's possible that we mean something slightly different thing by "typed effects". It's even possible that I'm using the term incorrectly. I meant: any kind of stuff, that one desires to happen "behind the scenes", but with preservation of proper typing. For me, in practice, it's closely related to monads and monad transformers.

2. I don't feel like the tradeoffs are worth it

What are the tradeoffs of typed effects you had in mind?

[raquo]

Warning: personal opinion / rant follows :)

It's possible that we mean something slightly different thing by "typed effects".

Oh for sure. I've no idea what's a "typed effect". Is Cats Effect IO a typed effect? One would think so, given the name, but Eff is also a system of typed effects, yet they are so different, Eff is so much more elaborate. And despite IO being a typed effect, neither Future nor Observable are generally considered typed effects.

So I honestly don't really know what people are talking about when they say typed effects. It's a type-safe program-as-value... as long as the program is not a () => Future[A] or a graph of Observables. Or I dunno, you're not doing typed effects as long as you need more knowledge of your business domain than of category theory. I'm only half joking...

Like, currently in Airstream you can't actually access a Signal's current value without proving that the signal is started. So... is this a "typed effect for sampling", or an alternative solution to the same problem? I don't even know, and I don't really care: it solves my problem of not wanting to accidentally access stale values, and end users don't need to know any academic theories to use this safely.

What are the tradeoffs of typed effects you had in mind?

Overhead, especially conceptual. On a small scale, there is nothing simpler than imperative code. A simple list of instructions executed one after another. But then, we need to deal with asynchrony, and that's where abstractions like Futures and Observables are imo superior to blocking calls. Then when using these abstractions to manage asynchronous pipelines, you naturally start wanting to use them to manage synchronous ones as well. Well, with observables, anyway. That's where Future-s fall off, for me. And this is my sweet spot.

Working in Scala, I hear about how typed effects will solve my problems all the time, but I don't seem to ever have problems that typed effects are supposed to be good at. No, I never accidentally logged my server to death. No, I never accidentally accessed a database when I thought I didn't need to. Or blew up something by writing to the DOM when I didn't intend to, or making a network request when I didn't intend to. I caused orders of magnitudes more problems by not understanding business requirements or how to translate them into code, passing the wrong type of data around (in dynamically typed languages), forgetting parens around an (A OR B) condition in a longer WHERE clause in an SQL query, or even just writing CSS that does the wrong thing. Scala's type system helps me with some of those real problems now, with or without typed effects.

At the same time, advanced typed effect tools like Eff and stuff like category theory are hard. People who are fluent in those don't appreciate just how hard these concepts are, and how disconnected they are from existing mental models and day-to-day business objectives. I can easily explain mutable state or Cat extends Animal to any child, and futures / observables to any developer. I could go as far as explain Cats Effect based on that: "Think about IO[A] as () => Future[A] but with nice operators to compose such values". But more complex stuff beyond that loses meaning and relevance very quickly.

I wasn't aware that there are differences at applying them [typed effects] at frontend/backend.

The main difference is, most of the backend is pipelines: request comes in, map flatMap, access database, get result, encode, return. Most mutable state is hidden away in a database behind a data service with a nice, limited, API that you designed for other services' consumption. Modifying that database state is a big deal.

I guess the reason this feels ok to me is because I use Cats Effect IO, which is like, not very typed. I basically treat it as not much more than Future[A], and then it's fine.

On the other hand, the frontend app is quite literally a single global mutable mega-object: the DOM. Everything about it is imperative. You didn't design it that way, it's just how it is, and you need to work with it. So you pick up some library like Laminar that provides you a more manageable API than calling element.append/setProperty/etc. manually. I've used React.js a lot before. It was ok without any typed effects, but it was annoying because it didn't have any built-in tools to manage time and asynchrony. Observables in Laminar provided that just fine to me, without any further complications. I don't need to think about category theory, and I don't need to care about purity. If I need to set an attribute on an element, I just do that. I don't need to wrap it into an effect type or anything.

Outwatch is supposed to be the Scala.js UI library of choice for those who love typed effects and purity and stuff. Is using it noticeably safer than Laminar? I don't think so, although I haven't used it recently. But I do know that Laminar has safety features that Outwatch doesn't, thanks to Airstream. And they have nothing to do with purity or typed effects or category theory or any of that. It's things like glitch resistance, mandatory ownership, and separation of streams and signals.

All this is of course highly personal, not sure how much that will help you if at all. Just wanted to vent I guess, and to lay this out there to make it clear just how much I can't give you any insight about all the hardcore / academic FP problems. Very much not my forte...

[cubuspl42]

Oh for sure. I've no idea what's a "typed effect".

It's an effect described by pure functions, I believe. In a language without "impure functions" (procedures), it's the only way to write programs (like in Haskell; let's not mention unsafe IO, unless you believe it brings something to the dispute =P). In languages with procedures, it means using them as pure functions by choice.

Is Cats Effect IO a typed effect?

Looks like one.

One would think so, given the name, but Eff is also a system of typed effects, yet they are so different, Eff is so much more elaborate.

The fact that there are many ways to approach typed effects doesn't make them disappear, I think.

And despite IO being a typed effect, neither Future nor Observable are generally considered typed effects.

I think it's because they aren't semantically pure, i.e. they are designed to work with procedural side effects. No catches here, they are just totally impure-by-design solutions. Semantically pure alternatives are described by FRP (or more like references, as FRP predates all popular toolkits and inspired most of them).

So I honestly don't really know what people are talking about when they say typed effects. It's a type-safe program-as-value... as long as the program is not a () => Future[A] or a graph of Observables.

More or less, I guess? The moment your "typed effects" start to semantically rely on impureness of procedures, they aren't typed effects in my vocabulary anymore.

Or I dunno, you're not doing typed effects as long as you need more knowledge of your business domain than of category theory. I'm only half joking...

If someone told you that one needs to know category theory to understand typed effects, they possibly mislead you.

Like, currently in Airstream you can't actually access a Signal's current value without proving that the signal is started.

I was really surprised by this statement. In other words, Airstream Signals can't be sampled at arbitrary time?

So... is this a "typed effect for sampling"

Airstream, having strong FRP roots, is not far from being close to a semantically pure solution. But various small things shift it further, including the influence of the reference count to the signal content over time (what we discussed in another thread) and procedural observers.

...or an alternative solution to the same problem?

This one, I'd say.

users don't need to know any academic theories to use this safely.

They don't need to know academic theories to use typed effects neither. Theories help with designing such systems, though.

Working in Scala, I hear about how typed effects will solve my problems all the time

I don't think they solve all problems, to be honest. I have an intuition that they can solve some problems in context of FRP, the ones I mentioned in my post above.

At the same time, advanced typed effect tools like Eff and stuff like category theory are hard.

So is quantum physics, but neither is required to use typed effects AFAIK. Note that I haven't used Eff specifically, and I'm also not sure that overly "detailed" effects systems make sense, but not because of them being typed.

and how disconnected they are from existing mental models

I very much agree with this one. The same could be said about basic functional programming a few years ago, though.

and day-to-day business objectives.

I disagree about this one, it may depend on business of course.

I can easily explain mutable state or Cat extends Animal to any child, and futures / observables to any developer.

I also explain any programming concept to a child, as the essence of explaining programming concepts to children is that you only have to make them think they understood them, not make them able to apply them. Explaining programming to developers (younger or older, but people that already have programming experience) is a totally different challange and the ease of explaining a concept will in 90% depend on the fact if they already know similar concepts. Where we come back to the "existing mental models" argument, where I agree with you.

On the other hand, the frontend app is quite literally a single global mutable mega-object: the DOM. Everything about it is imperative. You didn't design it that way, it's just how it is, and you need to work with it.

RAM cells are also a mutable mega-object, that's why we have abstractions. These whole FP, FRP and typed effects are all happy to run on top of mega mutable environements like RAM ord DOM, through some implementation, be it a compiler or a helper library. "On top" is crucial, though.

I don't need to think about category theory,

Come on ;)

All this is of course highly personal, not sure how much that will help you if at all. Just wanted to vent I guess, and to lay this out there to make it clear just how much I can't give you any insight about all the hardcore / academic FP problems.

I'm very grateful for your thoughts. It helped me a lot, as I'm trying my best to connect theoretical foundations with practical reality. I agree with you on a lot of points, which may be not obvious because always the most "interesting" parts to reply to or quote are the one where we have some different perspective. Also, to be honest, while here I'm "defending" type effects a bit, and I actually do believe that a) you don't have to have PhD or know category theory to use them and b) that they could solve some FRP challenges elegantly, I also don't think that they're "perfect". But which solution is, on the other hand?...

[raquo]

I was really surprised by this statement. In other words, Airstream Signals can't be sampled at arbitrary time?

You can get a signal's value in three ways:

1. Using observable.sample(signal) or observable.withCurrentValueOf(signal) operators. But the resulting observable will only run if it is started, i.e. has listeners, in which case signal is also guaranteed to have listeners as the resulting observable depends on it.

2. Using signal.observe(owner) and then calling .now() on the resulting StrictSignal, but then you need to provide an owner, and this will start a signal if it wasn't started already. Technically you could access the signal's value this way even after it was stopped, but there's only so much I can do to prevent users from shooting themselves in the foot.

3. Sending the signal's output into a Var, and then using the Var's .now() method, although (2) is a better way to do this unless you actually need the functionality of a Var.

The reason you can't just access signal.now() is to prevent users from sampling signals this way: observable.map(foo => withBar(barSignal.now())). Due to observables being lazy, this runs a high risk of accessing a stale value from a stopped signal, which is usually not what you want. In typical Airstream usage stopped observables are usually on their way out of existence, with few exceptions. If you want to read something, be it current value or all updates, you need to listen to it, aka make sure it's started.