Provide local project filter in west config

[aescolar]

As an enhancement to west,
provide a new feature in which users can, thru local west workspace configuration, filter projects out of a manifest.
It could either be thru a deny-list or an allow-list (or both).

Motivation/Use case:

A user wants to track a project main manifest (e.g. Zephyr), but only needs for his work a subset of the projects that project manifest has.
The users wants/needs to save disk space, "west update" time and/or bandwidth. Yet he wants to jump back and forth in branches of that project. The user would prefer to avoid needing to create another project or super-manifest that includes the main project of interest as an import with an allowlist, as that requires extra steps (as he would like to jump between several branches, which are changing over time, he would need to keep updating/branching that super-manifest).
The user does not have any means to affect the project of interest to require it to add group filters, or some of the projects cannot be added to group filters (for ex. due to imports)

The workflow would be something like:
A user does a west init, after which (probably before a west update) he configures the project to filter in/out what he does want/not want (west config manifest.project-filter "-.*, +project_x, +project_y") . And then he does subsequent west updates, which just ignore the filtered out projects/all but the filtered-in projects.

---

As an extension to this idea, so we would also cover the needs of #652 . In manifests projects definitions, allow a new key "default-inactive".
When a project has this key, the project is filtered out unless the local configuration project-filter enables it.
All other projects (without 'default-inactive') would be enabled by default unless the local configuration project-filter disables them.
There is no "default-active" key (it would be redundant).
Note: The project being filtered out means also a possible import done by that project is also ignored.
(Also, an imported manifest cannot define the same project and set "default-inactive : false", as the imported project with the same name is just ignored by west (this is the behavior already today)).

---

As a possible implementation (just to provide an understanding of a desired workflow and expected west behavior, not necessarily guide the solution):
The project-filter would be a list of projects names regexes, each preceded with "+" or "-".
When evaluating if a project is enabled or not, west would go thru this project-filter list from right to left. As soon as it finds the project name with a "-" it considers the project filtered out (and stops searching thru the list). As soon as it finds the project with a "+" , it considers it not filtered out (and stops searching thru the list).
The "default-inactive" key for <project_x> would be functionally equivalent to the list having as its left-most element -<project_x>

For extra users convenience, regex matches of the project name could be allowed. In this way, users could define a project filter like "-.*, +project_x, +project_y" (to disable everything but, project_x and y);
or conversely "+.*, -project_y" to disable only project_y, but enable all other projects (even if they had a "default-inactive" key;
or just " -project_y" to only disable "project_y" but let the rest as they would otherwise.

[marc-hb]

Déjà vu? https://github.com/zephyrproject-rtos/west/issues?q=+label%3A%22Partial+imports%22+

[mbolivar-nordic]

Déjà vu? https://github.com/zephyrproject-rtos/west/issues?q=+label%3A%22Partial+imports%22+

Agree this is part of that general theme -- are you suggesting it's a dupe of any one of these in particular? This doesn't seem to quite fit any open issue.

[marc-hb]

I thought it could have been, thanks for reviewing all existing issues and confirming it's not.

[aescolar]

Something that can be done today with a super-manifest, which achieves a relatively similar outcome, though each west update will detach the zephyr project into its current commit, can be done for example with:

mkdir my_workspace && cd my_workspace
git clone [email protected]:zephyrproject-rtos/zephyr.git
mkdir manifest
#Add a local manifest file
---------
$ cat manifest/mani.yml
manifest:
  projects:
    - name: zephyr
      url: https://github.com/zephyrproject-rtos/zephyr
      revision: HEAD~0
      import:
        name-allowlist:
          - cmsis
-------
cd manifest
west init -l --mf mani.yml
west update

which results in:
.
├── manifest
│   └── mani.yml
├── modules/hal/cmsis
└── zephyr

Note this example is with a local manifest file, a similar solution can be achieved getting the manifest from a git repo as described by @mbolivar-nordic https://discord.com/channels/720317445772017664/733037524498382881/1100819756904366151 :

you make an internal.git.server/meta-manifest.git, with a meta-manifest/west.yml file that imports zephyr from internal.git.server/zephyr.git, [..]

$ cat meta-manifest/west.yml 
manifest:
  projects:
    - name: zephyr
      url: https://internal.git.server/zephyr.git
      west-commands: scripts/west-commands.yml
then
west init -m https://internal.git.server/meta-manifest meta-workspace
cd meta-workspace
west update

[marc-hb]

Something that can be done today with a super-manifest

It's really strange and awkward to ask everyone to add a manifest layer on top of the default manifest to remove an opt-in that doesn't have to be stuffed in the default manifest in the first place.

It would be so much simpler and more scalable to ask users of babblesim (or of any other opt-in) to drop their opt-in manifest in some predefined submanifests/ location.

Compose by addition, not by stuffing+disable+re-add. Longer comment in the west update only what I need regression:

• bsim module must be cloned for west to work zephyr#57198 (comment)

[aescolar]

@marc-hb: You misunderstood my comment. What I tried to describe in it is a workaround for somebody who would otherwise want the feature requested in this enhancement request. It is not optimal, but it seemed worthwhile spelling it out in case it would help somebody.