slow workspace/project loading and Open File By Name (Ctrl + P)

[Waqar144]

Describe the bug

Performance is poor when loading a project of around ~50k files. Loading the workspace takes more than 10 seconds. When the project has loaded, Ctrl+P file opening is slow and laggy as in, it takes a while before I can type in the next char. Initially the project had 200K files and everything was unusable, but then I ignored a lot of stuff so now its better but still the performance is very poor in these features.

Compared to this, fzf can handle around a million files with 0 lag and its written in go

Expected behavior

Be fast. At least for file opening, really its just 50K strings.

• OS: Linux, X11
• Focus Version: 0.3.7

[focus-editor]

The open file fuzzy finder will be improved, I'll prioritize it for the next release after 0.3.8.

Workspace scanning and search are not such low hanging fruits, so speed improvements there aren't likely to be dramatic, but we haven't done a proper optimisation pass, so some improvement is to be expected.

However we don't intend to redesign the editor to support millions of files in the workspace, or very large files. This is the tradeoff to keep the internals simple and still cover 95% of use cases.

[Waqar144]

Millions of files dont need to be supported, I agree. That is not a realworld usecase even. But upto 200K files should be supported. E.g., big repos like the chromium, llvm or the linux kernel have that many files. LLVM 19 on my system has 146329 committed files (checked using git ls-files | wc -l).

Just for testing how fast it would be to recurse the same repository using std c++:

#include <chrono>
#include <filesystem>
#include <iostream>
#include <string>
#include <vector>

int main()
{
    auto start = std::chrono::high_resolution_clock::now();
    std::vector<std::string> files;
    for (const auto &dirEntry : std::filesystem::recursive_directory_iterator("."))
    {
        files.push_back(dirEntry.path());
    }
    auto end = std::chrono::high_resolution_clock::now();
    std::cout << "Collected " << files.size() << " files, Took "
              << std::chrono::duration_cast<std::chrono::milliseconds>(end - start).count() << " ms\n";
}

cold run:

$ g++ -O2 -std=c++17 test.cpp -o test
$ ./test
Collected 311220 files, Took 1604 ms

---

I suggest using git ls-files --recurse-submodules for file collection optionally. Git will do things much faster on linux at least and will handle the ignores and all other stuff as well. This can allow opening directories without having a project file. All vcs support file collection so its easily extendible. This is what kate does for creating projects on the fly https://github.com/KDE/kate/blob/master/addons/project/kateprojectworker.cpp

[focus-editor]

You think when scanning the workspace we're just getting the file list?

No, we're opening each file, loading it into memory, scanning for newlines, tokenizing etc. That's how the editor works

[Waqar144]

No, we're opening each file, loading it into memory, scanning for newlines, tokenizing etc. That's how the editor works

Is there a reason for doing this? Sounds incredibly wasteful.

[focus-editor]

The main reason is to support fast real-time search.

[Waqar144]

Ok thank you for answering.