Qi Meeting Aug 30 2024

Choose Your Words Carefully

Qi Meeting Aug 30 2024

Adjacent meetings: Previous | Up | Next [None]

Summary

We continued discussing next steps on deforestation.

Background

A few weeks ago, we started an integration branch based on Dominik's groundwork on deforesting different types of list-oriented interfaces and handling them in a uniform way, reorganizing the compiler to separate and encapsulate concerns more effectively. Sid started a PR against the integration branch introducing the #%deforestable core form to serve as the focal point for deforestation in the compiler. Together, these changes affect the design of list-oriented APIs in Qi, formalize the relationship between Qi and the host language, and open up questions on handling certain common patterns (such as passing arguments in arbitrary positions via templates) in a uniform way, which were discussed with Michael and Ben recently.

Merging the PR

We agreed that merging the PR would be a good next step as it would unblock other steps that could be undertaken in parallel. The coverage CI job fails on the PR due to a known issue, but after merging it into the branch on the upstream fork, we saw that coverage was down by about 6%, to 93%. That's OK, we will have a lot of time to work on that on the integration branch!

Continuing on Deforestation

We identified these next steps:

1. How to provide a runtime for a deforested API via the macro definition, and how can we enrich the syntax of #%deforestable so that it encodes which are floe vs expr positions in the provided runtime? This runtime must be available to the compiler so that it can generate Racket code for it if it reaches the end of compilation without being deforested (currently, this is encoded directly in the code generation step instead of being specified on the definition side).
2. Supporting fine and blanket templates in list-oriented forms uniformly without duplication (as these are now syntactic forms, we cannot use the existing templating infrastructure in the Qi core which are for host language functions).
3. Implementing continuation-passing based fused runtimes for more list-oriented APIs, like append, drop, and member.
4. Updating our benchmarking infrastructure. This will help us make informed choices as we work on optimizations.
5. Means of extending deforestation to custom list APIs (e.g. define-qi-stream-producer, ...).

1-3 are mostly parallelizable, while at least (5) is for down the road when things settle a bit and the interfaces become more clear.

The Dangers of Working on Qi While Living in Prague

Many of us around the world are governed by cat overlords, but in Prague if you say "Qi," the cats believe you are summoning them to perform humble duties in their service, such as feeding or petting them. As Dominik lives with five cats, this means that his desk can sometimes be overrun by cats in the course of our discussions! But luckily, it sounds like he has their permission to work on deforestation, so we are in the clear 😸

Next Steps

(Some of these are carried over from last time)

• Implement more fusable stream components like append, drop, and member.
• Provide a runtime and indicate floe positions by enriching the syntax of #%deforestable
• Implement the generic scheme for promoting position-oriented Qi macro definitions to handle templates
• Update the benchmarking infrastructure (vlibench and also the basic ones in qi-sdk)
• Define an extension scheme for deforesting custom list APIs.
• Fix the bug in using bindings in deforestable forms like range
• Finalize and document guiding principles for Git commits in the developer wiki
• Revisit extending deforestation to multi-valued settings.
• Write a proof-of-concept compiling Qi to another backend (such as threads or futures), and document the recipe for doing this.
• Ask Ben to review tests from the perspective of Frosthaven Manager.
• Review Cover's methodology for checking coverage.
• Document the release model in the user docs and announce the performance regression (and the remedy) to users.
• Improve unit testing infrastructure for deforestation.
• Discuss and work out Qi's theory of effects and merge the corresponding PR.
• Decide on appropriate reference implementations to use for comparison in the new benchmarks report and add them.
• Deforest other racket/list APIs via qi/list
• Decide on whether there will be any deforestation in the Qi core, upon (require qi) (without (require qi/list))
• Continue investigating options to preserve or synthesize the appropriate source syntax through expansion for blame purposes.

Attendees

Dominik, Sid