RFC 36 (async/await testing)

[tilk]

This PR refactors the tests for Amaranth's RFC 36. It is pretty massive, so thorough reviews are not expected. Let's merge this as quickly as possible and then deal with the possible aftermath later.

Important changes:

• In case when synchronization between different processes is actually needed, uses of Settle were replaced with sim.delay().
• Other instances of Settle were removed.
• Signal sampling (sim.tick().sample()) is used instead of signal value getting where it was reasonably simple to do so. In other cases, sim.get() is used (but we should try to avoid it).
• Method mocks work differently:
  • They consist of multiple processes: a sequential process, which works like before, plus combinational processes for updating the circuit after combinational changes. This helps avoid races in case of combinational dependencies between mocked methods.
  • Because the method mock can now be called multiple times per cycle, side effects must be guarded by MethodMock.effect. The effects are run once per cycle.
• For testbench processes which manipulate multiple methods in synchrony, CallTrigger was added, which allows to call multiple methods in a single clock cycle, together with sampling signals and the state of other methods.
• Number of ticks from simulation start can now be accessed by using a counter signal under TicksKey().

• Update testing abstractions
• Refactor tests
• Fix documentation
• Cleanup

[lekcyjna123]

Only some bikeshedding.

[Arusekk]

This equals to .sample(records[0].trigger, records[0].fields[0], ..., records[0].fields[-1], records[1].trigger, records[1].fields[0], ..., records[1].fields[-1], ...), do I understand right? It fits with logic inside handle_logs.

Looks a bit contrived, maybe some conversion could be extracted to a helper function, so that the records are grouped somehow maybe. Can handle_logs sample the records on its own?

[tilk]

Had the exact same thought. To do it better, you would probably need to represent the records using amaranth.lib.data or create custom ShapeCastables, ValueCastables, etc. Each record is different, so each would need a different shape. I decided to use this solution because it seemed to need less work than the alternative. I can leave a comment if you think it's warranted.

[whitequark]

@tilk How was your experience with async/await processes/testbenches? Any pain points I should know about?

[tilk]

@whitequark Pretty good actually. The sampling mechanism helps with races quite well. However, porting some tests required more work, because sampling requires you to take samples on a tick; you cannot decide later that you wanted to have a sample of some signal. A variant of get which returns the value of the signal that was there on the last tick would increase flexibility, but I imagine this might be hard to implement without losing some of the simulation performance.

It would be nice to have some synchronization mechanism which could allow removing races on Python state shared between processes. Currently I solve this by (ab)using sim.delay() - this is a good example.

[whitequark]

Very happy that you like it! The plan is to treat the new simulation mechanism as stable and not ever have as much churn as this migration caused, so I'm glad that it works well. A lot of careful work went into it.

A variant of get which returns the value of the signal that was there on the last tick would increase flexibility, but I imagine this might be hard to implement without losing some of the simulation performance.

Yes. Sampling a value essentially sets up a "tripwire" which then calls .get and remembers the result. Copying the entire simulator state is of course technically feasible but seems very wasteful. We considered it and decided not to add it at the time. Of course an RFC could change that.

It would be nice to have some synchronization mechanism which could allow removing races on Python state shared between processes.

I agree! At the time, my "solution" to it is to use Amaranth signals to share state, but this of course does not cover everything. It comes to mind that you could probably make a "mutex" (or other sync primitives) out of an Amaranth signal and random Python data. Would this solve anything?

Ideally I'd like to expose an easier to use concept, but I don't have any thoughts on the design.

[whitequark]

(When building a mutex there comes the issue of two processes locking it at exactly the same time. I'm thinking that each lock() call site could allocate a bit while spinning and the lowest set bit wins. It's kind of a weird way to construct a primitive, but you probably have parallels to this in the CPU itself, don't you?)