general improvement ideas

[pszemraj]

Ideas courtesy of chatGPT. Things we have already thought of have been struck out.

1. Add support for other providers, such as Hugging Face, GPT-3 Sandbox, and T5.
2. Implement a web interface that allows users to run queries easily and view results.
3. Add support for batch processing, allowing users to run multiple queries at once.
4. Implement a caching system that stores previous queries and their results, allowing users to easily retrieve them without having to run the query again.
5. Improve the CLI by adding tab-completion and more user-friendly error messages.
6. Add support for different output formats, such as JSON, CSV, in addition to markdown.
7. Create a Python package that can be easily imported and used in other projects. (NOTE: we do need to build on this, though, add more functionality if running as a python package)
8. Add support for query customization options, such as controlling the number of tokens generated, temperature and other advanced settings. (review whether we provide the ability to change ALL of the options)
9. Implement a logging system that tracks usage of the API, such as the number of queries run and the providers used. (I read this as a "overall stats" log and not a "here is what happened in the last call log")
10. Add more test coverage to ensure the software works as expected.