Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add preBuild hook to configuration standard #1343

Open
ed-randall-blk opened this issue Apr 3, 2024 · 4 comments
Open

Add preBuild hook to configuration standard #1343

ed-randall-blk opened this issue Apr 3, 2024 · 4 comments

Comments

@ed-randall-blk
Copy link

ed-randall-blk commented Apr 3, 2024

Proposed change

Builds which follow the repo2docker configuration standard can be difficult to configure behind certain firewalled environments where a proxy must be configured to gain "outside" Internet access.

For example, (ref. pangeo-docker-images issue-517) the pangeo/base-image/Dockerfile contains a number of ONBUILD instructions (following repo2docker configuration standard) which are intended to help the developer with installs of additional packages etc, but none of these steps works; We see errors from the ONBUILD steps like:

    RuntimeError: Download error (60) SSL peer certificate or SSH remote key was not OK [https://conda.anaconda.org/conda-forge/noarch/repodata.json]
    SSL certificate problem: self-signed certificate in certificate chain

A preBuild step similar to the existing [postBuild](https://repo2docker.readthedocs.io/en/latest/config_files.html#postbuild-run-code-after-installing-the-environment) would be provide a useful point where we could configure the additional certs, proxy environment config, and anything else required before apt-get package installation can succeed.

Possible code for the preBuild hook (untested):
(Insert at line 117 of https://github.com/pangeo-data/pangeo-docker-images/blob/master/base-image/Dockerfile between existing ONBUILD Checking for binder and Checking for apt.txt )

# If a preBuild file exists, run it
ONBUILD RUN echo "Checking for 'preBuild'..." \
        ; [ -d binder ] && cd binder \
        ; [ -d .binder ] && cd .binder \
        ; if test -f "preBuild" ; then \
        chmod +x preBuild \
        && ./preBuild \
        ; fi

Alternative options

As a workaround we have to forego the convenience of the provided ONBUILD steps and repeat the same logic in our own Dockerfile which builds FROM pangeo/base-image. This seems repetitive and should not be necessary.

We've considered forking pangeo/base-image into our own repo to make these changes possible but that's a highly undesirable option.

Who would use this feature?

Anyone who wishes to make a build following repo2docker configuration standards but is stuck behind a proxy firewall.

How much effort will adding it take?

Patch is above - I would estimate less than half a day for documentation updates and code integration.

Who can do this work?

Basic shell skills, ability to read and write documentation.

Copy link

welcome bot commented Apr 3, 2024

Thank you for opening your first issue in this project! Engagement like this is essential for open source projects! 🤗

If you haven't done so already, check out Jupyter's Code of Conduct. Also, please try to follow the issue template as it helps other other community members to contribute more effectively.
welcome
You can meet the other Jovyans by joining our Discourse forum. There is also an intro thread there where you can stop by and say Hi! 👋

Welcome to the Jupyter community! 🎉

@ed-randall-blk ed-randall-blk changed the title Add preBuild hook to configuration Add preBuild hook to configuration standard Apr 3, 2024
@manics
Copy link
Member

manics commented Apr 6, 2024

The r2d configuration files are designed to enabled reproducible repositories, but having a deployment specific pre-build breaks that, since it's only applicable to a single deployment and may actively break other deployments.

It sounds like you really want a deployment specific override separate from the repositories, e.g. in a global configuration file.
repo2docker already has an appendix option

appendix = Unicode(
config=True,
help="""
Appendix of Dockerfile commands to run at the end of the build.
Can be used to customize the resulting image after all
standard build steps finish.
""",
)

so maybe the equivalent for that, but at the start of the Dockerfile, could be useful, and you give all your users a standard repo2docker configuration file?

@ed-randall-blk
Copy link
Author

ed-randall-blk commented Apr 8, 2024

Our intended use for the proposed step is simply setting 3 environment variables (probably in a profile file, since following steps should be able to pick them up) and installing an SSL cert.
The problem we have is that, without setting these first, the subsequent steps cannot succeed.
If pre-appendix (for want of a name) could do that, we're good.

@MartinK84
Copy link

An equivalent for appendix but for the start of the Dockerfile would be perfect and should be sufficient to set up environment variables or other configs to make rep2docker work behind corporate firewalls. I'm also facing similar problems as @ed-randall-blk, using repo2docker behind a corporate firewall is a complete nightmare.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants