release-3.1.0: internal/server/server.go:19:12: pattern out/*: no matching files found

[jlsalvador]

Describe the bug
You forget the "out" directory for the release 3.1.0.

The makefile "build" does not compiles the web, the "out" directory. What do you think about add a dependency to a new target? Maybe a web?

The new target could execute:

• cd web (change workdir)
• npm ci
• npm run build
• mv out ../internal/server/

Another alternative could be to write an embed.go in the project root, in order to include the folder "web/out".

Thoughts?

The frequency of this issue

• This issue is easy to reproduce.
• This issue is not easy to reproduce.

Steps to Reproduce
Download latest release, try to compile it.

Your GoDNS configuration

Expected behavior
A successful 3.1.0 binary.

Screenshots

Environment (please complete the following information):

• Compiling the AUR package in a clean working directory.

Additional context

• Time for a 3.1.1 release. 👍

[TimothyYe]

Thanks for the quick feedback, I've added one line of annotation at the beginning of the file internal/server/server.go to copy the out directory generated by the frontend project. So theoretically, you need to compile & build the frontend project first, and run go generate ./..., it copies the out directory to internal/server.

Before doing that, you need to set up the frontend development environment first, please refer to: https://github.com/TimothyYe/godns?tab=readme-ov-file#setup-the-frontend-development-environment

I'll update the Makefile in 3.1.1 release.

[diizzyy]

I'm also seeing this in 3.1.2 or is the npm stuff a hard requirement now (please don't make it)?

===>  Building for godns-3.1.2
(cd /usr/ports/dns/godns/work/godns-3.1.2;  for t in ./cmd/godns; do  out=$(/usr/bin/basename $(echo ${t} |  /usr/bin/sed -Ee 's/^[^:]*:([^:]+).*$/\1/' -e 's/^\.$/godns/'));  pkg=$(echo ${t} |  /usr/bin/sed -Ee 's/^([^:]*).*$/\1/' -e 's/^godns$/./');  echo "===>  Building ${out} from ${pkg}";  /usr/bin/env XDG_DATA_HOME=/usr/ports/dns/godns/work  XDG_CONFIG_HOME=/usr/ports/dns/godns/work  XDG_CACHE_HOME=/usr/ports/dns/godns/work/.cache  HOME=/usr/ports/dns/godns/work PATH=/usr/ports/dns/godns/work/.bin:/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin:/root/bin PKG_CONFIG_LIBDIR=/usr/ports/dns/godns/work/.pkgconfig:/usr/local/libdata/pkgconfig:/usr/local/share/pkgconfig:/usr/libdata/pkgconfig MK_DEBUG_FILES=no MK_KERNEL_SYMBOLS=no SHELL=/bin/sh NO_LINT=YES PREFIX=/usr/local  LOCALBASE=/usr/local  CC="cc" CFLAGS="-O2 -pipe -march=tigerlake  -fstack-protector-strong -fno-strict-aliasing "  CPP="cpp" CPPFLAGS=""  LDFLAGS=" -fstack-protector-strong " LIBS=""  CXX="c++" CXXFLAGS="-O2 -pipe -march=tigerlake -fstack-protector-strong -fno-strict-aliasing  " BSD_INSTALL_PROGRAM="install  -s -m 555"  BSD_INSTALL_LIB="install  -s -m 0644"  BSD_INSTALL_SCRIPT="install  -m 555"  BSD_INSTALL_DATA="install  -m 0644"  BSD_INSTALL_MAN="install  -m 444" CGO_ENABLED=1  CGO_CFLAGS="-I/usr/local/include"  CGO_LDFLAGS="-L/usr/local/lib"  GOAMD64=  GOARM=  GOTMPDIR="/usr/ports/dns/godns/work" GOPATH="/usr/ports/distfiles/go/dns_godns"  GOBIN="/usr/ports/dns/godns/work/bin"  GO111MODULE=on  GOFLAGS=-modcacherw  GOSUMDB=sum.golang.org GO_NO_VENDOR_CHECKS=1 GOMAXPROCS=6 GOPROXY=off /usr/local/bin/go120 build -buildmode=exe -v -trimpath -ldflags=-s -buildvcs=false -mod=vendor  -o /usr/ports/dns/godns/work/bin/${out}  ${pkg};  done)
===>  Building godns from ./cmd/godns
internal/server/server.go:19:12: pattern out/*: no matching files found
*** Error code 1

[TimothyYe]

@jlsalvador @diizzyy

I'll update the Makefile with two available building commands:

make all #builds all the projects, including the frontend project & the backend project
make build # only build the backend project

[jlsalvador]

@jlsalvador @diizzyy

I'll update the Makefile with two available building commands:

make all #builds all the projects, including the frontend project & the backend project
make build # only build the backend project

I was working on it a bit: jlsalvador@0765d10
WIP. Take a look for improvements/ideas. 👍

A merge request will be submitted when it's completed.

[TimothyYe]

@diizzyy If you'd like to build it with your scripts, here is the workaround:

mkdir -p ./internal/server/out
touch ./internal/server/out/index.html
GO111MODULE=on go build cmd/godns/godns.go

[diizzyy]

@TimothyYe
Thanks, it's actually for the FreeBSD port ( https://repology.org/project/godns/versions ) and that workaround seems to do the trick.

[TimothyYe]

@diizzyy Yes, it depends on whether you want to include the frontend project at compile time.

[diizzyy]

There's a strict rule that no internet access is allowed during builds (which makes sense) however that makes npm more or less a no go (or a major pain I'm not willing to spend time on) so unless we can get a precompiled frontend there's not going to be packaged version with the frontend.

[TimothyYe]

@diizzyy Thanks for the explanation, however, I still have a bit of confusion. If there is no internet connection during builds, when compiling this Go project, if new modules are introduced in the code, they will also be downloaded from the internet during the compilation period. How is this achieved?

[diizzyy]

By build I guess I should say compilation as packaging is divided into different stages ( https://docs.freebsd.org/en/books/porters-handbook/book/#options-targets ).

See also:
https://docs.freebsd.org/en/books/porters-handbook/book/#using-go
https://cgit.freebsd.org/ports/tree/Mk/Uses/go.mk

[TimothyYe]

@diizzyy I asked ChatGPT for an example of a Makefile to build the port, which includes npm and Go as dependencies. I'm not sure if it will be helpful:

# Ports metadata
PORTNAME=       myproject
DISTVERSION=    1.0.0
CATEGORIES=     www

MAINTAINER=     [email protected]
COMMENT=        An example Go project with embedded Next.js static files

LICENSE=        MIT

BUILD_DEPENDS=  npm:www/npm \
                go:lang/go

USES=           go:modules

# Use the post-extract target to install Node.js dependencies and build the Next.js project
post-extract:
    @cd ${WRKSRC}/web && ${SETENV} ${MAKE_ENV} npm install
    @cd ${WRKSRC}/web && ${SETENV} ${MAKE_ENV} npm run build

# Use the pre-build target to move the Next.js build output to the location expected by the Go code
pre-build:
    @${MKDIR} ${WRKSRC}/internal/server
    @${MV} ${WRKSRC}/web/out/* ${WRKSRC}/internal/server

# Use the do-build target to build the Go project, embedding the Next.js static files
do-build:
    @cd ${WRKSRC} && ${SETENV} ${MAKE_ENV} go build -o ${PORTNAME} cmd/${PORTNAME}/main.go

# Use the do-install target to install the built Go binary
do-install:
    ${INSTALL_PROGRAM} ${WRKSRC}/${PORTNAME} ${STAGEDIR}${PREFIX}/bin/${PORTNAME}

.include <bsd.port.mk>

Key Points:

• BUILD_DEPENDS: This line declares build-time dependencies for your port, including npm for building the Next.js project and go for compiling the Go project.

• USES: This directive specifies that the port uses Go modules for managing Go dependencies.

• post-extract: This target is run after extracting your port's distfiles but before building. It's used here to install Node.js dependencies and build the Next.js project. Adjust the paths according to your project's structure.

[diizzyy]

npm is more complicated than that (which is why you rarely see it packaged in any repo).

See below for example
https://cgit.freebsd.org/ports/tree/textproc/write-good/Makefile
https://cgit.freebsd.org/ports/tree/www/py-django-hijack/Makefile