Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Earthly-based builds #2

Open
wants to merge 8 commits into
base: rv
Choose a base branch
from
Open

Add Earthly-based builds #2

wants to merge 8 commits into from

Conversation

alistairking
Copy link

@alistairking alistairking commented Oct 10, 2022

Currently builds for Ubuntu 20.04, 22.04, Centos 7/8.

  1. Install Earthly (https://earthly.dev/get-earthly)
  2. Clone this repo (this branch for now; the rv branch eventually)
  3. earthly +all
  4. Go get a coffee
  5. All packages are in ./dist/

Results:

$ find dist/
dist
dist/linux
dist/linux/amd64
dist/linux/amd64/ubuntu
dist/linux/amd64/ubuntu/20.04
dist/linux/amd64/ubuntu/20.04/frr-doc_8.5~dev-1_all.deb
dist/linux/amd64/ubuntu/20.04/frr_8.5~dev-1_amd64.deb
dist/linux/amd64/ubuntu/20.04/frr-pythontools_8.5~dev-1_all.deb
dist/linux/amd64/ubuntu/20.04/frr-rpki-rtrlib_8.5~dev-1_amd64.deb
dist/linux/amd64/ubuntu/20.04/frr-snmp_8.5~dev-1_amd64.deb
dist/linux/amd64/ubuntu/22.04
dist/linux/amd64/ubuntu/22.04/frr-doc_8.5~dev-1_all.deb
dist/linux/amd64/ubuntu/22.04/frr_8.5~dev-1_amd64.deb
dist/linux/amd64/ubuntu/22.04/frr-pythontools_8.5~dev-1_all.deb
dist/linux/amd64/ubuntu/22.04/frr-rpki-rtrlib_8.5~dev-1_amd64.deb
dist/linux/amd64/ubuntu/22.04/frr-snmp_8.5~dev-1_amd64.deb
dist/linux/amd64/centos
dist/linux/amd64/centos/8
dist/linux/amd64/centos/8/frr-snmp-8.5_dev_rv-01.el8.x86_64.rpm
dist/linux/amd64/centos/8/frr-8.5_dev_rv-01.el8.x86_64.rpm
dist/linux/amd64/centos/8/frr-devel-8.5_dev_rv-01.el8.x86_64.rpm
dist/linux/amd64/centos/8/frr-rpki-rtrlib-debuginfo-8.5_dev_rv-01.el8.x86_64.rpm
dist/linux/amd64/centos/8/frr-contrib-8.5_dev_rv-01.el8.x86_64.rpm
dist/linux/amd64/centos/8/frr-rpki-rtrlib-8.5_dev_rv-01.el8.x86_64.rpm
dist/linux/amd64/centos/8/frr-debugsource-8.5_dev_rv-01.el8.x86_64.rpm
dist/linux/amd64/centos/8/frr-snmp-debuginfo-8.5_dev_rv-01.el8.x86_64.rpm
dist/linux/amd64/centos/8/frr-debuginfo-8.5_dev_rv-01.el8.x86_64.rpm
dist/linux/amd64/centos/8/frr-pythontools-8.5_dev_rv-01.el8.x86_64.rpm
dist/linux/amd64/centos/7
dist/linux/amd64/centos/7/frr-8.5_dev_rv-01.el7.x86_64.rpm
dist/linux/amd64/centos/7/frr-snmp-8.5_dev_rv-01.el7.x86_64.rpm
dist/linux/amd64/centos/7/frr-debuginfo-8.5_dev_rv-01.el7.x86_64.rpm
dist/linux/amd64/centos/7/frr-devel-8.5_dev_rv-01.el7.x86_64.rpm
dist/linux/amd64/centos/7/frr-pythontools-8.5_dev_rv-01.el7.x86_64.rpm
dist/linux/amd64/centos/7/frr-rpki-rtrlib-8.5_dev_rv-01.el7.x86_64.rpm
dist/linux/amd64/centos/7/frr-contrib-8.5_dev_rv-01.el7.x86_64.rpm

@alistairking alistairking marked this pull request as ready for review October 12, 2022 01:31
@ryeleo
Copy link

ryeleo commented Oct 12, 2022

Very nice! For testing the built package, see "Test Steps" in our building_frr doc:

  • CentOS: we can deploy to our test collector -- frr.routeviews.org
  • Ubuntu: we don't actually have an Ubuntu test collector yet. We can deploy it to route-views7 -- alternately, it might be wise to ramp up another test collector running Ubuntu 22.04

@ryeleo ryeleo linked an issue Oct 12, 2022 that may be closed by this pull request
2 tasks
@ryeleo
Copy link

ryeleo commented Nov 3, 2022

Thanks again for this PR @alistairking! I'm happy to be getting around to testing earthly on my Windows Desktop on my home network -- first time ran through I hit the following error: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/p/po-debconf/po-debconf_1.0.21_all.deb Undetermined Error [IP: 185.125.190.39 80]. More context around the error copied below.

As a workaround, I'm tempted to try Earthly's "GitHub Actions integration paired with the upload-artifact GitHub Action.

I'll come back to dig into this later!

Earthly build failure

           +deps-one | E: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/p/po-debconf/po-debconf_1.0.21_all.deb  Undetermined Error [IP: 185.125.190.36 80]
           +deps-one | E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?
           +deps-one | ERROR Earthfile line 79:4
           +deps-one |       The command
           +deps-one |           RUN DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends autoconf automake libtool make build-essential devscripts equivs libreadline-dev texinfo pkg-config libpam0g-dev libjson-c-dev bison flex libc-ares-dev python3-dev python3-sphinx install-info libsnmp-dev perl libcap-dev libelf-dev libunwind-dev wget
           +deps-one |       did not complete successfully. Exit code 100
            +pkg-one | WARN: Canceled

================ ❌ FAILURE [2. Build 🔧] ================

@alistairking
Copy link
Author

Any chance you're behind some kind of proxy?
prakhar1989/docker-curriculum#186

@ryeleo
Copy link

ryeleo commented Nov 4, 2022

Any chance you're behind some kind of proxy? prakhar1989/docker-curriculum#186

Not intentionally behind a proxy on my home network... 🤔 I wonder if 'UO Split VPN' solution that I have running is effecting this build. I'll try to kick off another build when I'm not using that VPN later today.

@ryeleo
Copy link

ryeleo commented Nov 9, 2022

Running again from my home network without VPN -- running into a similar network issue but with a different package this time:

          +deps-one | Get:235 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjson-c-dev amd64 0.13.1+dfsg-7ubuntu0.3 [46.9 kB]
           +deps-one | Fetched 105 MB in 1min 39s (1054 kB/s)
           +deps-one | E: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/t/tzdata/tzdata_2022e-0ubuntu0.20.04.0_all.deb  404  Not Found [IP: 91.189.91.38 80]
           +deps-one | E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?
           +deps-one | ERROR Earthfile line 79:4
           +deps-one |       The command
           +deps-one |           RUN DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends autoconf automake libtool make build-essential devscripts equivs libreadline-dev texinfo pkg-config libpam0g-dev libjson-c-dev bison flex libc-ares-dev python3-dev python3-sphinx install-info libsnmp-dev perl libcap-dev libelf-dev libunwind-dev wget
           +deps-one |       did not complete successfully. Exit code 100

============================ ❌ FAILURE [2. Build 🔧] ============================

Now I'm even more tempted to try Earthly's "GitHub Actions integration paired with the upload-artifact GitHub Action... I'm going to want to look into Limits for GitHub Actions probably to make sure I don't drain all of Route Views' freebie minutes 😅

ryeleo pushed a commit that referenced this pull request Nov 10, 2022
Fixing the crash:

> #0  0x0000560aa80f8e30 in lspdb_const_find (h=<error reading variable: Cannot access memory at address 0x7fff5e95efe8>, item=<error reading variable: Cannot access memory at address 0x7fff5e95efe0>) at ./isisd/isis_lsp.h:64
> #1  0x0000560aa80f8e9d in lspdb_find (h=0x560aaa1ed3b8, item=0x7fff5e95f050) at ./isisd/isis_lsp.h:64
> #2  0x0000560aa80f92f9 in lsp_search (head=0x560aaa1ed3b8, id=0x7fff5e95f200 "") at isisd/isis_lsp.c:100
> FRRouting#3  0x0000560aa8113d69 in spf_adj_list_parse_tlv (spftree=0x560aaa1f09d0, adj_list=0x560aaa214480, id=0x560aad331a78 "", desig_is_id=0x0, pseudo_metric=0, metric=3, oldmetric=false, subtlvs=0x0) at isisd/isis_spf.c:1330
> FRRouting#4  0x0000560aa811419d in spf_adj_list_parse_lsp (spftree=0x560aaa1f09d0, adj_list=0x560aaa214480, lsp=0x560aaa1f4e50, pseudo_nodeid=0x0, pseudo_metric=0) at isisd/isis_spf.c:1429
> FRRouting#5  0x0000560aa81141fe in spf_adj_list_parse_lsp (spftree=0x560aaa1f09d0, adj_list=0x560aaa214480, lsp=0x560aaa1ff8e0, pseudo_nodeid=0x0, pseudo_metric=0) at isisd/isis_spf.c:1442
> FRRouting#6  0x0000560aa81141fe in spf_adj_list_parse_lsp (spftree=0x560aaa1f09d0, adj_list=0x560aaa214480, lsp=0x560aaa1f4e50, pseudo_nodeid=0x0, pseudo_metric=0) at isisd/isis_spf.c:1442
> (...)
> #65507 0x0000560aa81141fe in spf_adj_list_parse_lsp (spftree=0x560aaa1f09d0, adj_list=0x560aaa214480, lsp=0x560aaa1ff8e0, pseudo_nodeid=0x0, pseudo_metric=0) at isisd/isis_spf.c:1442
> #65508 0x0000560aa81141fe in spf_adj_list_parse_lsp (spftree=0x560aaa1f09d0, adj_list=0x560aaa214480, lsp=0x560aaa1f4e50, pseudo_nodeid=0x0, pseudo_metric=0) at isisd/isis_spf.c:1442
> #65509 0x0000560aa81141fe in spf_adj_list_parse_lsp (spftree=0x560aaa1f09d0, adj_list=0x560aaa214480, lsp=0x560aaa1ff8e0, pseudo_nodeid=0x0, pseudo_metric=0) at isisd/isis_spf.c:1442
> #65510 0x0000560aa81141fe in spf_adj_list_parse_lsp (spftree=0x560aaa1f09d0, adj_list=0x560aaa214480, lsp=0x560aaa1f4e50, pseudo_nodeid=0x0, pseudo_metric=0) at isisd/isis_spf.c:1442
> #65511 0x0000560aa8114313 in isis_spf_build_adj_list (spftree=0x560aaa1f09d0, lsp=0x560aaa1f4e50) at isisd/isis_spf.c:1455
> #65512 0x0000560aa8114f09 in isis_run_spf (spftree=0x560aaa1f09d0) at isisd/isis_spf.c:1775
> #65513 0x0000560aa8115057 in isis_run_spf_with_protection (area=0x560aaa1ed3b0, spftree=0x560aaa1f09d0) at isisd/isis_spf.c:1801
> #65514 0x0000560aa8115311 in isis_run_spf_cb (thread=0x7fff5f15e5a0) at isisd/isis_spf.c:1859
> #65515 0x00007f90bac66dcc in thread_call (thread=0x7fff5f15e5a0) at lib/thread.c:2002
> #65516 0x00007f90bac013ee in frr_run (master=0x560aa9f5cb40) at lib/libfrr.c:1196
> #65517 0x0000560aa80e7da2 in main (argc=2, argv=0x7fff5f15e7b8, envp=0x7fff5f15e7d0) at isisd/isis_main.c:273

The fix is similar to the crash fix included in d9884a7
("isisd: Prepare IS-IS for Link State support"). The fix was:

> diff --git a/isisd/isis_lsp.c b/isisd/isis_lsp.c
> index 94353a5bc8..92d329f035 100644
> --- a/isisd/isis_lsp.c
> +++ b/isisd/isis_lsp.c
> @@ -2166,7 +2178,7 @@ int isis_lsp_iterate_ip_reach(struct isis_lsp *lsp, int family, uint16_t mtid,
>  	if (lsp->hdr.seqno == 0 || lsp->hdr.rem_lifetime == 0)
>  		return LSP_ITER_CONTINUE;
>
> -	/* Parse main LSP. */
> +	/* Parse LSP */
>  	if (lsp->tlvs) {
>  		if (!fabricd && !pseudo_lsp && family == AF_INET
>  		    && mtid == ISIS_MT_IPV4_UNICAST) {
> @@ -2236,13 +2248,17 @@ int isis_lsp_iterate_ip_reach(struct isis_lsp *lsp, int family, uint16_t mtid,
>  		}
>  	}
>
> -	/* Parse LSP fragments. */
> -	for (ALL_LIST_ELEMENTS_RO(lsp->lspu.frags, node, frag)) {
> -		if (!frag->tlvs)
> -			continue;
> +	/* Parse LSP fragments if it is not a fragment itself */
> +	if (!LSP_FRAGMENT(lsp->hdr.lsp_id))
> +		for (ALL_LIST_ELEMENTS_RO(lsp->lspu.frags, node, frag)) {
> +			if (!frag->tlvs)
> +				continue;
>
> -		isis_lsp_iterate_ip_reach(frag, family, mtid, cb, arg);
> -	}
> +			if (isis_lsp_iterate_ip_reach(frag, family, mtid, cb,
> +						      arg)
> +			    == LSP_ITER_STOP)
> +				return LSP_ITER_STOP;
> +		}
>
>  	return LSP_ITER_CONTINUE;
>  }

Fixes: 7b36d36 ("isisd: make the SPF code more modular")
Fixes: 5e56a50 ("isisd: fix infinite loop when parsing LSPs")
Signed-off-by: Louis Scalbert <[email protected]>
(cherry picked from commit 8c8a5a0)
@ryeleo
Copy link

ryeleo commented Nov 20, 2022

@alistairking, I've been working on a GitHub Action workflow for "Build FRR" in the rv-CICD branch. The build itself appears to work well!!! Earthly is returning a happy exit code after building for around 5 minutes 😁👍

However, the SAVE ARTIFACTS ... LOCAL earthly task seems to just not run, and I'm otherwise unable to find the packages after the build success.

I wonder if you have thoughts on why I can't find packages from the Earthly build?

(Maybe I do just need the "--push" flag, like they suggest in their template GitHub Action -- I'll read more and give that a try).

@alistairking
Copy link
Author

You might want to take a look at: https://github.com/alistairking/scamper/blob/master/.github/workflows/build.yml
which uses https://github.com/alistairking/scamper/blob/master/Earthfile

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Build FRR 8.3.1 RPMs (for Route Views)
2 participants