Make a decision about the dynamics module

[nwlandry]

Right now we have functions related to synchronization but there are far more dynamics we could implement. Right now there are external packages (hypercontagion, simplicial-kuramoto) which are packages for higher-order dynamics. I don't think it makes sense to have a dynamics module unless we commit to filling it out more. If not, I would like to remove it. That being said, if we decide to fill out the dynamics module, there could be some cool opportunities for extending node and edge stats to dynamical states.

[leotrs]

How much more would you like to see in the dynamics module and how much effort do you think that would take? Is it more or less effort than implementing export/import to formats accepted by these other libraries (thinking that maybe interoperability might be a replacement)?

…

On Thu, Oct 24, 2024, 1:45 PM Nicholas Landry ***@***.***> wrote: Right now we have functions related to synchronization but there are far more dynamics we could implement. Right now there are external packages (hypercontagion, simplicial-kuramoto) which are packages for higher-order dynamics. I don't think it makes sense to have a dynamics module unless we commit to filling it out more. If not, I would like to remove it. That being said, if we decide to fill out the dynamics module, there could be some cool opportunities for extending node and edge stats to dynamical states. — Reply to this email directly, view it on GitHub <#611>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAILYAFZ5HWB5ZLNVP4TGUDZ5DMXDAVCNFSM6AAAAABQQ4T6WCVHI2DSMVQWIX3LMV43ASLTON2WKOZSGYYTCMZRG42DINQ> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[tlarock]

I think it would be great to have a dynamics module, even if it only has some basic models. I also think it will be difficult to do well without a lot of development time and energy, not least because what counts as a "basic model" and how it should be implemented is already unclear given the variability in the literature. If someone were interested in taking this up, they could potentially take cues from the structure of netrd dynamics.

I'm not as sold on removing what already exists, not because I don't agree in principle, but because it was useful for me at one point in a project recently to be able to play with what we have there now. Could be convinced in the long run it is better to just remove it if we aren't going to develop actively.

Agree with @leotrs that interoperability might be a good compromise solution if possible, I'm not familiar with the other packages mentioned.

[nwlandry]

How much more would you like to see in the dynamics module and how much effort do you think that would take? Is it more or less effort than implementing export/import to formats accepted by these other libraries (thinking that maybe interoperability might be a replacement)?
…
On Thu, Oct 24, 2024, 1:45 PM Nicholas Landry @.> wrote: Right now we have functions related to synchronization but there are far more dynamics we could implement. Right now there are external packages (hypercontagion, simplicial-kuramoto) which are packages for higher-order dynamics. I don't think it makes sense to have a dynamics module unless we commit to filling it out more. If not, I would like to remove it. That being said, if we decide to fill out the dynamics module, there could be some cool opportunities for extending node and edge stats to dynamical states. — Reply to this email directly, view it on GitHub <#611>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAILYAFZ5HWB5ZLNVP4TGUDZ5DMXDAVCNFSM6AAAAABQQ4T6WCVHI2DSMVQWIX3LMV43ASLTON2WKOZSGYYTCMZRG42DINQ . You are receiving this because you are subscribed to this thread.Message ID: @.>

I think that there are two potential goals here: (1) add dynamical models to xgi. This seems fairly easy. We can copy-paste from hypercontagion to form an epidemic sub-module and opinion formation sub-module. We can add to this piecemeal. At some point, I would worry about the scope of this because, for instance, hypercontagion contains code to generate animations which requires some niche dependencies. (2) integrate the stats module with dynamics. Right now stats are static, but we would need to incorporate temporality to capture dynamical states. This could also be useful for temporal hypergraphs which would be great to support in the future. This second item seems like a longer term project, but it's what excites me most about potentially having a dynamics module.

[maximelucas]

In a sense, I do agree that having dynamics inside XGI is a bit "out of scope". XGI is mainly structure, and we even have a data repository which arguably could even be a separated (but linked) thing. For comparison, say, NetworkX does not have dynamics. If one want to do it well, dynamics is a whole different thing from structure, from simulation to visualisation, and finding a way to implement different dynamical processes in a similar way. I'm personally working on a "hypersync" package I hope to make public soon.

This being said, there aren't many libraries to simulate dynamics on hypergraphs out there yet. As pointed out by Tim, it can be useful to have a few basic dynamical systems implemented, as a centralised starting point for people to just test something.

I see 2 shorter-term options:

1. Remove all dynamics from XGI, but advertise explicitly the existing outside libraries as XGI "extensions", and make them visible somewhere (for example, a "Dynamics" page on the website that explains this and has links?)
2. Have a very minimal but self-consistent dynamics module inside XGI. And still advertise outside libraries "to go further". But it should not create problems for XGI (weird dependencies) and stay very general. I also don't think I would add any specific functions to visualise time series and such (or super minimal).

I think (2) can be nice, and it's what's been happening by default. Maybe we could make a list of a minimal set of dynamical processes that would make sense to have? See if their implementation can have a common base and not require weird dependencies?

I like the idea of having dynamic stats, and temporal hypergraphs of course, but this seems like a much longer-term goal?

[leotrs]

Temporal stats were in the designer's mind from the beginning 😉 Yes it would take some leg work, but it's meant to be possible.

Agree with Max here, the way forward seems for one of us to put together a notebook where some simple dynamical system is implemented using the XGI Hypergraph class. But the implementation would be done from outside XGI, from the perspective of a potential user, answering the question "what if I wanted to use XGI structures to implement dynamics". And then we can work out how big of a deal it would be to implement it inside the library.

One more thing that's not clear to me right now is whether we are talking about dynamics Of or dynamics ON hypergraphs? Dynamics OF hypergraphs can essentially be a list/dict of Hypergraphs....

[tlarock]

I would rank explicitly supporting temporal hypergraphs/stats above offering a wider range of dynamics. A solid foundation for temporal/evolving hypergraphs could make dynamics much simpler to implement in the future.

This is not necesarily inconsistent with offering something along the lines of @maximelucas's point (2), subject to developer capacity. However, implementing them separately would almost certainly mean substantial breaking changes down the line once we have non-static stats implemented.

A (potentially worst of all worlds) compromise could be adding an "xgi.experimental" module with the express purpose of placing submodules that are not actively developed or gauranteed to be supported in the future, but that still contain useful functionality. At start it would include only the xgi.experimental.dynamics submodule. The problem with this route is deciding at what point we fully stop supporting something in xgi.experimental, as well as how something gets promoted out. (Edited to add:) Graphs.jl offers an Experimental module, for example

I also think @leotrs's suggestion of a dynamics notebook/tutorial showing how one might implement themselves may be the most workable solution that allows us to remove the module without losing what is most helpful about its existence.

[maximelucas]

Side note: implementation-wise, I see mostly two categories of dynamical processes (ON hypergraphs @leotrs). The deterministic ones, and the stochastic ones. The deterministic ones, as long as they can be written as ODEs, can be implemented in a similar way by wrapping some RK4 or scipy's ODE solver. Then it's mostly a matter of writing the function for the rhs of the ODE. That includes sync, I guess also consensus and others, and mean-field versions of contagion processes etc. The stochastic ones, including SIS etc require a different (heavier?) apparatus.

[leotrs]

I suggest we don't move things to a experimental sub-package. Instead, we can keep the dynamics stuff outside of the main package by default so that import xgi doesn't reveal any dynamics-related code, but import xgi.dynamics does. At the same time, we can issue a warning when importing the dynamics module specifically in this way, something that says "this module is experimental" or something along those lines.

@tlarock the purpose of the notebook I mentioned was to show US how difficult it would be to implement things. But I agree that it could be polished and made available as part of the new gallery for example.

I believe @tlarock is speaking about dynamics OF and @maximelucas is talking about dynamics ON. (No?) So maybe we also have to prioritize these. And agree with @maximelucas that stochastic and deterministic processes seem to require different approaches. I'm out of touch, how useful is one vs the other?

[tlarock]

Got it re: notebook, that makes sense!

I guess I am talking about both, responding to point (2) in @nwlandry's last comment. In my mind, integrating dynamics ON with stats would be made easier by first adding features to stats for explictily dealing with dynamics OF (i.e., temporal/dynamic hypergraphs). Once we have "temporal node stats", many numerical dynamical process simulations should boil down to dropping in the right update equation (famous last words).

[nwlandry]

Side note, I'm currently writing a grant that will hopefully fund (1) the development of temporal hypergraph data structures and (2) more tightly integrate hypercontagion with xgi, which could provide some guidance on how to implement some of these things.

[leotrs]

So... who makes that notebook?

[tlarock]

How does the notebook you have in mind compare to the two in tutorials/case_studies @leotrs ?

[leotrs]

One of them uses what's already implemented in the library. The other one seems to be computing the distribution of eigenvalues of some random structures.

I guess my idea is closer to the second one. I would like to see what code is necessary on top of what's in the library, not code that uses what's currently in the library.

FWIW, you can think of an eigenvalue as a Hypergraph-level stat. Right now we only have node and edge stats (right?) but extending the stats framework to Hypergraphs wouldn't be an issue. Then it would also not be an issue to put together a class, say HypergraphCollection, with methods such as HypergraphCollection([H1, H2, H3, ...]).some_stat.mean() which computes the mean of some_stat of the underlying list of hypergraphs.

[maximelucas]

Before seeing who's gonna do what, let's make a list of what we would want.
Minimal set of dynamical processes :

1. sync (node-based)
2. sync (simplex-based)
3. contagion

Those we kinda already have somewhere, and could adapt (and put in a notebook if needed).

What else would we really need? Options would be random walk, consensus, ... ?