VS Code: LLM request cancellations are not transmitted reliably to Cody Gateway

[valerybugakov]

• LLM request cancellations are not transmitted reliably to Cody Gateway, leading to token inference continuing until the maximum limit is reached. It significantly increases the load on our inference providers, increasing latency and spending. By the latest estimate, about 2/3 of inferenced tokens are "overhead" tokens.
• The Google Sheet with pricing breakdown and approximate potential savings.
• Slack thread with more context.

[github-actions]

This issue is marked as stale because it has been open 60 days with no activity. Remove stale label or comment or this will be closed automatically in 5 days.

[github-actions]

This issue is marked as stale because it has been open 60 days with no activity. Remove stale label or comment or this will be closed automatically in 5 days.