Error when building in Docker due to layered filesystem

[daniellockyer]

Hey! 👋🏻 I'm working on getting Linux ARM builds for https://github.com/TryGhost/node-sqlite3 but I'm running into an issue when using node-pre-gyp in Docker.

#10 [6/8] RUN npx node-pre-gyp build
#10 0.458 node-pre-gyp info it worked if it ends with ok
#10 0.460 node-pre-gyp info using [email protected]
#10 0.460 node-pre-gyp info using [email protected] | linux | arm64
#10 0.485 node-pre-gyp ERR! UNCAUGHT EXCEPTION
#10 0.485 node-pre-gyp ERR! stack Error: EXDEV: cross-device link not permitted, rename 'build-tmp-napi-v3' -> 'build'
#10 0.485 node-pre-gyp ERR! stack     at Object.renameSync (node:fs:980:3)
#10 0.485 node-pre-gyp ERR! stack     at Object.module.exports.swap_build_dir_in (/usr/src/build/node_modules/@mapbox/node-pre-gyp/lib/util/napi.js:179:8)
#10 0.485 node-pre-gyp ERR! stack     at /usr/src/build/node_modules/@mapbox/node-pre-gyp/lib/build.js:21:12
#10 0.485 node-pre-gyp ERR! stack     at handle_gyp_opts (/usr/src/build/node_modules/@mapbox/node-pre-gyp/lib/util/handle_gyp_opts.js:101:10)
#10 0.486 node-pre-gyp ERR! stack     at do_build (/usr/src/build/node_modules/@mapbox/node-pre-gyp/lib/build.js:13:3)
#10 0.486 node-pre-gyp ERR! stack     at build (/usr/src/build/node_modules/@mapbox/node-pre-gyp/lib/build.js:49:12)
#10 0.486 node-pre-gyp ERR! stack     at Object.self.commands.<computed> [as build] (/usr/src/build/node_modules/@mapbox/node-pre-gyp/lib/node-pre-gyp.js:86:37)
#10 0.486 node-pre-gyp ERR! stack     at run (/usr/src/build/node_modules/@mapbox/node-pre-gyp/lib/main.js:81:30)
#10 0.486 node-pre-gyp ERR! stack     at Object.<anonymous> (/usr/src/build/node_modules/@mapbox/node-pre-gyp/lib/main.js:125:1)
#10 0.486 node-pre-gyp ERR! stack     at Module._compile (node:internal/modules/cjs/loader:1103:14)
#10 0.487 node-pre-gyp ERR! System Linux 5.10.104-linuxkit
#10 0.487 node-pre-gyp ERR! command "/usr/local/bin/node" "/usr/src/build/node_modules/.bin/node-pre-gyp" "build"
#10 0.487 node-pre-gyp ERR! cwd /usr/src/build
#10 0.488 node-pre-gyp ERR! node -v v16.14.2
#10 0.488 node-pre-gyp ERR! node-pre-gyp -v v1.0.9
#10 ERROR: executor failed running [/bin/sh -c npx node-pre-gyp build]: exit code: 7

According to https://docs.docker.com/storage/storagedriver/overlayfs-driver/#modifying-files-or-directories:

Renaming directories: Calling rename(2) for a directory is allowed only when both the source and the destination path are on the top layer. Otherwise, it returns EXDEV error (“cross-device link not permitted”). Your application needs to be designed to handle EXDEV and fall back to a “copy and unlink” strategy.

You should be able to reproduce the issue by cloning the node-sqlite3 repo, dropping the following into tools/BinaryBuilder.Dockerfile:

ARG NODE_VERSION=16
ARG VARIANT=bullseye

FROM node:$NODE_VERSION-$VARIANT

WORKDIR /usr/src/build

COPY . .
RUN npm install --ignore-scripts
RUN npx node-pre-gyp configure
RUN npx node-pre-gyp build
RUN npm run test
RUN npx node-pre-gyp package

CMD ["sh"]

and then running

docker buildx build --file ./tools/BinaryBuilder.Dockerfile --progress plain --no-cache .

[daniellockyer]

My #dirtyhack to workaround this is the following:

RUN cd node_modules/\@mapbox/node-pre-gyp \
  && npm install [email protected] \
  && sed -i -e s/'fs/'fs-extra/ -e s/fs.renameSync/fs.moveSync/ ./lib/util/napi.js