v0.3.1: Image models, TypeScript rewrite, Bedrock support

[ianarawjo]

This change has been in the works for over a month. The most significant change comprises a rewrite of the entire front-end of ChainForge, which is tens of thousands of lines of code, into TypeScript. More details below.

Rewrite of the front-end into TypeScript

The entire front-end has been converted to tsx files, with the appropriate typing. Other nice refactorings.

Among identifying bugs or never reached code, we have worked to simplify and standardize the ways LLMResponses are stored on the backend and front-end. It is not perfect, but now LLMResponse and LLMResponseData make the format of stored responses transparent to developers.

This change makes it easier for developers to extend ChainForge with confidence, chiefly because TypeScript flags side-effects of changes to core code. For instance, TypeScript enabled us to add image models in only 2 hours, since all that was required was changing the type LLMResponseData from a string to a string | object-with-image-data. This change would not have been easy to perform with confidence without the visibility TypeScript provides into the downstream effects of changing a core datatype. In the future, it will help us add support for multi-modal vision models.

Support for image models (with Dall-E OpenAI models being the first)

Images are compressed by default using compressorjs, with no visible impact on quality and average compression around 60% of the original. Users can turn off compression in the Advanced tab of the Settings window. It is recommended to keep it on, however.

Warning

Be warned that images eat up browser storage space fast and will quickly disable autosaving.

Custom Right-click Actions on Nodes

Right-clicking on a node can now present more options.

• TextFields can be converted into Items Nodes
• Items Nodes can be converted into TextFields Nodes
• Prompt and Chat Turn node LLM response cache can be cleared

Amazon Bedrock Support

Thanks to @massi-ang , Amazon Bedrock hosted models have been added to ChainForge. We've just added these endpoints, and I wasn't able to test them directly (I don't have access), so if you encounter problems please open an Issue and poke @massi-ang to let him know.

Nested Add Model Menu on Prompt nodes

Thanks also to @massi-ang , clicking Add+ to add a model now brings up a nested menu. The list is still limited (use the Settings to access more specific models), but we were facing a growing problem as the number of providers increased.

Better Rate Limiting

Rate limiting has been improved to use Bottleneck. Rate limiting is now performed on a rolling timing window rather than the naive "block and wait" approach we used before. Note that this doesn't take into account what "tier" of access you have for OpenAI and Anthropic models, as there's no way for us to know that, so it's based on Tier 2 access. If you encounter a rate limit just re-run the node.

Problems? Let us know!

This change is comprehensive, and while I have tested it extensively, it is possible I have missed something. If you encounter an error, please open an Issue.

---

This discussion was created from the release v0.3.1: Image models, TypeScript rewrite, Bedrock support.