Please remember that Copilot Workspace is a technical preview and is an area of active development. This document lists some known issues and known areas where we'd like to make future improvements of the product.
The following are core areas where we are actively working to improve Copilot Workspace.
When Copilot Workspace implements a plan that involves changes in a large file, it can take a long time to complete. Copilot Workspace currently uses "whole file rewriting" as we have found this achieves a high level of thoroughness on the very heterogeneous range of tasks Copilot Workspace can be used for.
We are working on partial-file rewriting techniques, both automatic and under user-guidance, to improve the performance of this operation.
The quality of the code generated by Copilot Workspace is not always perfect. It is highly related to the quality of the underlying AI-models used. We are working on improving the quality of the code generated by Copilot Workspace on many levels.
For example, the quality of code generation is affected by the quality of the planning and specification of the task, and the overall user-experience of assessing and clarifying these. We are working to improve these too.
The achieved quality is also related to the experience of iterating on the generated code. We are actively looking at more fine-grained iteration techniques.
The content selection in Copilot Workspace can sometimes be suboptimal, leading to the generation of code that is not relevant to the task. We are working on improving the content selection in Copilot Workspace.
Tasks can include direct links to web resources such as documentation. Further, some web retrieval can also be deduced from the task. Copilot Workspace does not currently perform web retrieval and we are working on adding this feature.
After code is generated, both AI and traditional tooling can be used to "repair" the code based on diagnostics generated from building, testing and running the code. We already have some support for this in Copilot Workspace, and we are working on improving this.
Some tasks are very small: updating a few lines of a file. Some tasks are very large: implementing an entirely new repository feature by feature.
Copilot Workspace is currently designed for the mid-range of tasks typical of GitHub issues. We are interested in delivering variations on the core concepts of Copilot Workspace in arrangements more suitable to tasks both small and large. For example, for small tasks we may offer a "lite" version of Copilot Workspace where there is only the task. For large tasks we may offer a way to decompose the task into sub-tasks.
Copilot Workspace uses a GitHub OAuth App for authentication. Some organizations can have policies which restrict OAuth applications from interacting with their repositories. You will not be able to perform tasks in private repositories with Copilot Workspace, or create pull requests in public repositories unless the organization admin approves the Copilot Workspace OAuth application.
We are working to add a second authorization option for Copilot Workspace based on a GitHub App, and will update this document when this is available.
Copilot Workspace can be used with many existing large repositories. However, some repositories are so large that even listing their files can be a challenging exercise. Further, some repositories contain highly unusual content: files with enormously long lines, or binary files with unusual extensions, and so on. Large repositories are also challenging to content selection and planning.
Copilot Workspace places a limit on the GitHub-reported size of repositories that may be analyzed. If a repository is above this limit you may see Copilot Workspace may not be used to analyze <owner>/<repo> due to size limitations.
We are working on improving the scalability of Copilot Workspace to handle these cases.
Copilot Workspace provides an integrated terminal to assist with validating code. We are continuously working on a range of techniques to improve the validation capabilities of Copilot Workspace.
Copilot Workspace is currently delivered as a web application with an opinionated UX modality. A core design principle is to make this experience available pervasively, and we will be assessing enabling alternative delivery mechanisms in IDEs or at the command-line.
We are also looking at augmenting the range of developer tooling to integrate with Copilot Workspace web experience.
The following are more specific known issues related to the current release of Copilot Workspace.
The ambiguity detection in Copilot Workspace can sometimes be too sensitive, and can kick in even when the task is clear. We are working on improving the accuracy of the ambiguity detection in Copilot Workspace.
When choosing an existing issue to work on, the user experience is not as smooth as it could be, and it can be easier just to adjust the deep-link URL for Copilot Workspace or return to the issue page on GitHub and choose "Open in Workspace".
Commits from Workspaces PRs are not currently signed. We will be working on adding this feature.
There is no 'Stop' button when generating pull request descriptions. We are working on adding this feature.
On mobile, the auto-completion list is positioned wrong when adding a file to the plan. A fix is in progress for this.
We are actively working to improve the terminal start-up time in Copilot Workspace.
The repair flow from the terminal to the plan sometimes times out, causing the terminal help button to say it can't assist you. A workaround for this is to click the send button (up arrow icon) again can regenerate the suggestion. We understand that this is not ideal, and are working on a solution to this issue.