Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature Request] Temperature Last #14

Open
alicat22 opened this issue Jan 30, 2024 · 0 comments
Open

[Feature Request] Temperature Last #14

alicat22 opened this issue Jan 30, 2024 · 0 comments
Labels
enhancement New feature or request

Comments

@alicat22
Copy link
Contributor

Is your feature request related to a problem? Please describe.
Temperature Last is important, since temperature behaves differently before the sampler (like Min-P) vs after. When Temperature Last is enabled, it becomes much easier to find reasonable values and allows a much higher temperature since the tokens have already been trimmed.

Describe the solution you'd like
A clear and concise description of what you want to happen.

Additional context
For API's like tabby, this parameter is labeled temperature_last and is a bool. Aphrodite (either added or added soon) and Ooba also have temperature last with their OAI endpoint API. KCPP, I believe, still uses sampler order, so we can technically have temp last already with KCPP.

@alicat22 alicat22 added the enhancement New feature or request label Feb 1, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant