Sort environments by assumed usefulness

This is a design document for a new environment sorting logic for the quickpick that appears when selecting an interpreter (#16520).

Motivation

Currently, environments are sorted by Python version > architecture (if any) > company name (if any). This hampers discoverability, and we believe that sorting them with the most useful ones at the top will improve the environment selection experience.

"Usefulness" would be determined by environment type:

• Local environments (same path as the workspace root) will be closest to the top of the quickpick;
• Globally-installed environments (e.g. $WORK_ON or conda, but basically anything not local) will follow;
• Global installs (e.g. /usr/bin/python3.9) will be last.

Within each category, sort by Python versions in descending order (most recent first), with some Conda-related special cases:

• If there are several types of globally-installed environment available, Conda ones should have lowest priority within their Python version subgroup;
• When having multiple Conda environments with the same Python version, the base environment should be last within its Python version subgroup.

Scope

The work done here covers the quickpick that appears when the Python: Select Interpreter command is triggered. This work will also impact the auto-selection process, which will be discussed once the sorting logic gets merged.

Alternative Considered

An option could be to still sort by Python version only, but by descending order instead of the current ascending logic. Even though it would simpler than what we decided on, it doesn't differentiate between environment types, and doesn't suggest that local environments would be more preferable to the user.

Architecture

The quickpick gets populated by IInterpreterSelector.getSuggestions:

    public async getSuggestions(resource: Resource, ignoreCache?: boolean): Promise<IInterpreterQuickPickItem[]> {
        const interpreters = await this.interpreterManager.getInterpreters(resource, {
            onSuggestion: true,
            ignoreCache,
        });
        interpreters.sort(this.interpreterComparer.compare.bind(this.interpreterComparer));
        return Promise.all(interpreters.map((item) => this.suggestionToQuickPickItem(item, resource)));
    }

Environments are retrieved from interpreterManager.getInterpreters, but the real sorting magic happens in interpreters.sort(this.interpreterComparer.compare.bind(this.interpreterComparer)), which is an instance of the InterpreterComparer class implementing the IInterpreterComparer interface:

export interface IInterpreterComparer {
    compare(a: PythonEnvironment, b: PythonEnvironment): number;
}

The comparison logic would follow this algorithm:

1. Check the environment type: local > global environment > global install;
2. Then, compare by descending Python version;
3. Apply Conda-specific rules if necessary;
4. Finally, compare with the rest of the environment info: architecture, company name, and environment name.

Environment types will be labeled as follow:

Environment Type	Sorted Type
venv	Local/Globally-installed
Conda	Globally-installed
virtualenv	Globally-installed
virtualenvwrapper	Globally-installed
Pipenv	Globally-installed
Poetry	Globally-installed
pyenv	Global
Windows Store	Global
System	Global
Global	Global
Unknown	Global

We currently use a singleton of the IInterpreterHelper helper class to retrieve the display name of an interpreter, which we still need necessary for the last sorting rule (compare env info). As such, we will need to expose the new comparison logic in a class that implements the IInterpreterComparer interface.

This helper will also be used to retrieve the path of the current workspace to determine whether a venv environment is local or globally installed.

APIs

No externally-facing APIs will be exposed.

Telemetry and Experimentation

This sorting algorithm will be gated behind an experiment, so we can see how it impacts our current metrics. Since we know that we want to roll out this change to all users (instead of just testing hypotheses), the experiment will have a quick progression.

No new telemetry will be added as part of this work.

Testing Strategy

Unit tests will be written for the comparison function. Tests should also be added to interpreterSelector.getSuggestions to make sure that the sorting logic works for both experiment groups.

Appendix: Interpreter auto-selection

Autoselection happens when InterpreterAutoSelectionService.autoSelectInterpreter (in src/client/interpreter/autoSelection/index.ts) is called.

As part of the sorting update, we are also going to change the way auto-selection works, so as to use a similar logic. This would only be visible in the first session ever with a workspace though, since afterwards we would have cached interpreters and a value stored in persistent state with the interpreter that was auto-selected for this workspace.

What does InterpreterAutoSelectionService do?

The code:

    public async autoSelectInterpreter(resource: Resource): Promise<void> {
        const key = this.getWorkspacePathKey(resource);
        if (!this.autoSelectedWorkspacePromises.has(key)) {
            const deferred = createDeferred<void>();
            this.autoSelectedWorkspacePromises.set(key, deferred);
            await this.initializeStore(resource);
            await this.clearWorkspaceStoreIfInvalid(resource);
            await this.userDefinedInterpreter.autoSelectInterpreter(resource, this);
            this.didAutoSelectedInterpreterEmitter.fire();
            Promise.all(this.rules.map((item) => item.autoSelectInterpreter(resource))).ignoreErrors();
            deferred.resolve();
        }
        return this.autoSelectedWorkspacePromises.get(key)!.promise;
    }

1. Check if the auto-selection mechanism for this workspace has been triggered before;
2. If yes -> return the deferred promise attached to it;
3. If not, create a deferred promise and start the auto-selection mechanism:
   1. Remove any (in-session) cached information about the workspace;
   2. If there is a globally preferred interpreter and it has been fetched ("global" interpreter set as a default for no workspace), exit early. If not, fetch it from the persistent store, and if the file path from this value doesn't exist update the persistent store to undefined;
   3. Kick off auto-selection rules with the settings rule first: If in the DeprecatePythonPath experiment retrieve the value of python.defaultInterpreterPath, otherwise python.pythonPath. If the value returned is not python, exit early;
   4. Fire a didAutoSelectedInterpreterEmitter event
   5. Wait for all rules to resolve in the following order:
      1. Settings rule in 3.iii.;
      2. Check workspace interpreters: If there are workspace values for python.defaultInterpreterPath/python.pythonPath (in experiment or not), exit early. Else, find all workspace interpreters, sort them by version and pick the most recent one;
      3. Check cached interpreters: Check if the system, current path and windows registry rules have cached interpreters, and if that's the case pick the most recent one if it's more recent than the globally preferred interpreter from 3.ii.;
      4. Check current path: Find Python interpreters on known paths (either interpreters of type PythonEnvSource.PathEnvVar or the CURRENT_PATH_SERVICE locator if not in the discovery experiment), and pick the most recent one if it's more recent than the globally preferred interpreter from 3.ii.;
      5. Check Windows registry: If on Windows, get the interpreters in the Windows registry, and pick the most recent one if it's more recent than the globally preferred interpreter from 3.ii.;
      6. Check system: Go all in and retrieve all interpreters, filter out virtualenv, venv and pipenv environments, and pick the most recent one if it's more recent than the globally preferred interpreter from 3.ii.;
   6. Resolve the deferred promise

Where is it called?

• ExtensionActivationManager.activate:

    public async activate(): Promise<void> {
        await this.initialize();
        // Activate all activation services together.
        await Promise.all([
            Promise.all(this.singleActivationServices.map((item) => item.activate())),
            this.activateWorkspace(this.activeResourceService.getActiveResource()),
        ]);
        await this.autoSelection.autoSelectInterpreter(undefined);
    }

• ExtensionActivationManager.activateWorkspace, which is also called inside ExtensionActivationManager.activate above:

    public async activateWorkspace(resource: Resource): Promise<void> {
        const key = this.getWorkspaceKey(resource);
        if (this.activatedWorkspaces.has(key)) {
            return;
        }
        this.activatedWorkspaces.add(key);

        if (this.experiments.inExperimentSync(DeprecatePythonPath.experiment)) {
            await this.interpreterPathService.copyOldInterpreterStorageValuesToNew(resource);
        }

        // Get latest interpreter list in the background.
        this.interpreterService.getInterpreters(resource).ignoreErrors();

        await sendActivationTelemetry(this.fileSystem, this.workspaceService, resource);

        await this.autoSelection.autoSelectInterpreter(resource);
        await Promise.all(this.activationServices.map((item) => item.activate(resource)));
        await this.appDiagnostics.performPreStartupHealthCheck(resource);
    }

• InterpreterDisplay.updateDisplay:

    private async updateDisplay(workspaceFolder?: Uri) {
        const interpreterPath = this.configService.getSettings(workspaceFolder)?.pythonPath;
        if (!interpreterPath || interpreterPath === 'python') {
            await this.autoSelection.autoSelectInterpreter(workspaceFolder); // Block on this only if no interpreter selected.
        }
        const interpreter = await this.interpreterService.getActiveInterpreter(workspaceFolder);
        this.currentlySelectedWorkspaceFolder = workspaceFolder;
        if (interpreter) {
            this.statusBar.color = '';
            this.statusBar.tooltip = this.pathUtils.getDisplayName(interpreter.path, workspaceFolder?.fsPath);

Other public auto-selection methods, events and interfaces

• IInterpreterAutoSelectionProxyService and its implementation InterpreterAutoSelectionProxyService: Proxy class set up so there are no circular dependencies between get autoselected interpreter <-> config settings, until the auto-selection service gets instantiated
  • registerInstance?(instance: IInterpreterAutoSelectionProxyService): Register auto-selection service instance so that calls are made instead of short-circuiting to undefined in the proxy
  • setWorkspaceInterpreter(resource: Uri, interpreter: PythonEnvironment | undefined): Update persistent state value for a given workspace path
• onDidChangeAutoSelectedInterpreter event: Self-explanatory
• getAutoSelectedInterpreter(resource: Resource): Self-explanatory
• setGlobalInterpreter(interpreter: PythonEnvironment | undefined): Save interpreter data with no workspace path

How do we implement the new auto-selection algorithm?

What is the new logic?

New auto-selection logic:

• If there's a cached auto-selected interpreter -> use it and exit
• If there are cached interpreters (not the first session in this workspace) -> sort them, pick the first one and exit
• If not, we already fire all the locators, so wait for their response, sort the interpreters, pick the first one and exit

Architecture

In the InterpreterAutoSelectionService class, the content of the autoSelectInterpreter method needs to change depending on whether we are in the sorting experiment or not:

• Move current logic in its own private method
• Add separate private method for new logic

Remember to add tests.