v0.2.1

What's new in 0.2.1 (2023-08-23)

These are the changes in inference v0.2.1.

New features

• FEAT: Adding support for stream returning completion results by @takatost in #368

Enhancements

• ENH: including default context length into model family by @Bojun-Feng in #374

Bug fixes

• BUG: PyTorch generate config max_new_tokens not compatible with RESTful API by @Bojun-Feng in #373
• BUG: llm class match by @UranusSeven in #383
• BUG: return chat model handle by @UranusSeven in #382
• BUG: xinference cache dir doesn't exist by @UranusSeven in #380

New Contributors

• @takatost made their first contribution in #368

Full Changelog: v0.2.0...v0.2.1