Verified Solution

[microsoft/vscode] Limited context size (33k) for Ollama Models

Sponsored Content
### ROOT CAUSE: The context window limitation occurs because the VS Code Copilot extension enforces a default context size of approximately 33k tokens for Ollama models, overriding their actual larger capacities. This restriction is likely implemented within the Copilot extension's code, preventing models from utilizing their full context window unless explicitly configured. ### CODE FIX: To resolve this issue, we need to modify the VS Code Copilot extension to respect the model's default context size or allow users to configure it. Here's how the fix can be implemented: 1. **Modify the model configuration to use the model's specified context size:** - **File:** `packages/copilot-chat/src/model.ts` - **Change:** Adjust the context size retrieval to fetch the model's default context value from the Ollama client or allow an override. 2. **Add a user-configurable setting for the context window:** - **File:** `packages/copilot-chat/src/settings.ts` - **Change:** Introduce a new setting, such as `github.copilot.ollama.contextWindow`, to allow users to specify the context size for Ollama models. 3. **Update the Ollama client to handle dynamic context sizes:** - **File:** `packages/copilot-chat/src/ollama.ts` - **Change:** Ensure the Ollama client uses the specified context size when making API calls, either from model defaults or user settings. These modifications will enable Ollama models to utilize their full context window capacity or allow users to set a custom size, enhancing the functionality of VS Code's Copilot feature. ```json // Example configuration to add user settings { "github.copilot.ollama.contextWindow": 128000 } ``` By implementing these changes, the context window limitation is lifted, allowing Ollama models to function as intended with their larger context sizes.
Deploy on DigitalOcean ($200 Credit)

Related Fixes

[microsoft/vscode] bugged out response
[docker/cli] Increase memory for MicroVM in docker sandbox
[gitlab-org/gitlab] Auto-merge (MWCP) does not respect "Skipped pipelines are considered successful" project setting