Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add more versatile replication filters #8614

Open
cedvan opened this issue Aug 8, 2019 · 23 comments
Open

Add more versatile replication filters #8614

cedvan opened this issue Aug 8, 2019 · 23 comments
Labels
area/replication kind/requirement New feature or idea on top of harbor

Comments

@cedvan
Copy link

cedvan commented Aug 8, 2019

Hi all,

Need

I want replicate images from a namespace. But only need replicate tags "stable" to save my storage

On docker hub :

  • 1.10.0-alpha
  • 1.9.01
  • 1.9.0
  • 1.9.0-rc2
  • 1.9.0-rc1
  • 1.8.0

Need get on my harbor :

  • 1.9.01
  • 1.9.0
  • 1.8.0

So ignore pre release version and alpha version to save storage.

Problem

Try configure my replication rule with existing filters (cf. https://github.com/goharbor/harbor/blob/master/docs/user_guide.md#resource-filter)

  • *.*.* => but 1.9.0-rc1 match
  • ?.?.? => but 1.9.01 not match
  • {?.?.?,?.??.??} => Ok, but 1.10.0 soon available and so don't match

Problem : It is necessary to update the filter regularly so that my replication rule works in the future

Proposal solutions

1 - Add a pattern to match only number, like \d with regex
2 - Add more versatile replication filter like composer *, ~, ^, <, =, > (cf. https://getcomposer.org/doc/articles/versions.md#versions-and-constraints)
3 - Simply use regex as pattern to configure filter

@jcmartins
Copy link

+1

@ywk253100 ywk253100 added kind/requirement New feature or idea on top of harbor area/replication labels Aug 9, 2019
@xaleeks xaleeks self-assigned this Aug 9, 2019
@xaleeks
Copy link
Contributor

xaleeks commented Sep 13, 2019

we need a common consistent regex support for all features, not just limited to replication. Adding this to next release planning

@dkulchinsky
Copy link
Contributor

@xaleeks was wondering if there's any tentative timeline for this work (this is still tagged for 1.11.0)? I totally agree that there need to be a consistent regex support throughout the different features, our immediate use case is with immutability rules.

@HaveFun83
Copy link

as workaround we use this filter {?,??}.{?,??}.{?,??} for some kind of semver matching

@cedvan
Copy link
Author

cedvan commented Feb 5, 2021

@HaveFun83 This solution not match versions "X" and "X.X"

@HaveFun83
Copy link

@HaveFun83 This solution not match versions "X" and "X.X"

sure "X" and "X.X" is not a semver annotation so far as i know.
https://semver.org/#spec-item-2

@dkulchinsky
Copy link
Contributor

Hey @xaleeks, is there any update you can provide on this? would love to this get slotted for a future version 🙏🏼

@badsmoke
Copy link

this topic has also just come up in our company, it would be fabulous if this feature were to be implimented soon.

@veyselsahin
Copy link

We are waiting for this feature, too.

@badsmoke
Copy link

I have the feeling that this thread receives no attention :-(

@cedvan
Copy link
Author

cedvan commented Jun 16, 2021

1.11.0 candidate, last version is 2.2.2...

@niklasweimann
Copy link

I would love to see this feature in the next release. It's super annoying without this feature

@exight
Copy link

exight commented Jun 14, 2022

This feature would be absolut awesome!

@UrsusLvovich
Copy link

My company would also love to have regex support in the tag filtering

@wy65701436
Copy link
Contributor

ping @qnetter for awareness

@Aadeesh11
Copy link

I have applied for this project on LFX website. I have started learning GO, so can I get some pointers on how to get started with this project.

@WilfredAlmeida
Copy link
Contributor

This issue seems like a great starting point for getting started with a contribution to Harbor. Landed here from the LFX Mentorship projects.

I'd like to get started with the issue regardless.

Can someone point out the file where the regex is written for the filters for me to take a look?

@chlins
Copy link
Member

chlins commented Feb 21, 2023

@Vad1mo
Copy link
Member

Vad1mo commented Feb 25, 2023

I would opt for option 3.
Paired with a fast regex lib like re2, it should be possible to filter a big chunk of repos at once

@WilfredAlmeida
Copy link
Contributor

I would opt for option 3. Paired with a fast regex lib like re2, it should be possible to filter a big chunk of repos at once

So will the function IsSemanticVersion() be similar to the other functions in pattern.go do? It'll have regex-matching semantic versioning numbers.

@Vad1mo
Copy link
Member

Vad1mo commented Feb 26, 2023

We should also, make sure that the new feature is backwards-compatible.

I see two options.

  1. migrate/convert existing rules into regex
  2. add an option for the user to select a simple option or regex.
    ( I, personally, think this many options would make the understanding too complex)

I would prefer option 1)

@WilfredAlmeida
Copy link
Contributor

migrate/convert existing rules into regex

Agreed. This can be done. There are 2 approaches to doing this

  1. Migrate existing rules to regex, then add for this issue
  2. Implement current issue filters in regex and then migrate existing rules

I'd prefer option 2 since it'll give us more clarity into regex.

@WilfredAlmeida
Copy link
Contributor

I've drafted a proposal for this issue formulating multiple approaches to solve the issue, please review and share your thoughts.

goharbor/community#221

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/replication kind/requirement New feature or idea on top of harbor
Projects
None yet
Development

No branches or pull requests