Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

any CVMFS mirrors #463

Closed
stdweird opened this issue Dec 2, 2024 · 11 comments
Closed

any CVMFS mirrors #463

stdweird opened this issue Dec 2, 2024 · 11 comments

Comments

@stdweird
Copy link

stdweird commented Dec 2, 2024

If there are any cvmfs mirrors we can add to the configuration, please list them in the docs. If there are none, please describe the high availibility of the single server_url.

@stebo85
Copy link
Contributor

stebo85 commented Dec 2, 2024

Dear @stdweird - yes, we will document this in full details in the next days on our website! Thank you for raising it.

@stebo85
Copy link
Contributor

stebo85 commented Dec 6, 2024

Dear @stdweird ,

We now documented everything in detail: https://www.neurodesk.org/docs/getting-started/neurocontainers/cvmfs/

Can you have a look through and let us know if you need anything else?

@stdweird
Copy link
Author

stdweird commented Dec 6, 2024

@stebo85 thanks for the list. i will use the list that you set in the variable SERVER_URL.
one remark wrt the docs. i would move the description of the servers in a separate section, after the troubleshooting. this is pretty advanced, and changing that variable is probably only needed for setups running in eg aws. putting that whole description where it is now, forces people to read it, and i am quite sure it will confuse people ;)

@stebo85
Copy link
Contributor

stebo85 commented Dec 6, 2024

You don't need to add all servers - pick a few that are close to you only. Also depending on your setup you might want to use the direct servers and not the cdn aliases - especially when you are closer to one of the servers than what the cdn can provide you and if you are running a local proxy.

@stdweird
Copy link
Author

stdweird commented Dec 6, 2024

@stebo85 ugh. you better document that as well ;)
i assumed that cvmfs would figure it out, eg based on the geoip data. i assume there is not cvmfs tool to give a list of urls, and the get it reordered with closest first?

anyway, in our case, i'll use the nikhef one first, and then the aws frankfurt one (so i have 2 at least). any chance you can convince the aws maintainers to register a dns alias? the current hostname looks like it might change any time now ;)

thanks again

@stebo85
Copy link
Contributor

stebo85 commented Dec 6, 2024

Yes, the cvmfs client figures out where the servers are based on their ip address. That's what the USE_GEOAPI parameter controls.

It's also possible to probe the servers and measure the latency but the CDNs will always give a short response time which then doesn't translate to the real use case because it's rarely all in cache. I can add this to the documentation.

Yes, we can create another dns record for the Frankfurt server.

@stdweird
Copy link
Author

stdweird commented Dec 6, 2024

@stebo85 also, s1nikhef-cvmfs.openhtc.io -> cvmfs01.nikhef.nl:8000 it's without port 8000

@stebo85
Copy link
Contributor

stebo85 commented Dec 6, 2024

Yes, i probably missed that

@stebo85
Copy link
Contributor

stebo85 commented Dec 13, 2024

I had a look at this again to complete this issue:

  • update s1nikhef-cvmfs.openhtc.io:8080 -> cvmfs01.nikhef.nl:8000
  • cvmfs-frankfurt.neurodesk.org is now a DNS entry to ec2-3-72-92-91.eu-central-1.compute.amazonaws.com

Let us know if you see anything else.

@stebo85 stebo85 closed this as completed Dec 13, 2024
@stdweird
Copy link
Author

@stebo85 hmm, why is the :8000 needed for nikhef .nl? it seems to work on default port as well

@stebo85
Copy link
Contributor

stebo85 commented Dec 13, 2024

I don't think they are needed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants