POSIX conformance: run an HTTP file server (#215)

This simplifies a lot of other code that was spinning up busybox as an http server. This has failed a number of times in CI. The simpler deployment of this approach has a lot going for it. The one downside is that it shows non-canonical usage of the POSIX log. But as there's already an HTTP server running, it seems wasteful to not take advantage in this way.

.github/workflows/benchmark.yml

@@ -29,7 +29,7 @@ jobs:
  - name: Start Tessera
  run: docker compose -f ./cmd/conformance/posix/docker/compose.yaml up --build --detach
  - name: Run benchmark
- run: go run ./hammer --log_public_key=example.com/log/testdata+33d7b496+AeHTu4Q3hEIMHNqc6fASMsq3rKNx280NI+oO5xCFkkSx --log_url=http://localhost:2024 --write_log_url=http://localhost:2025 --max_read_ops=0 --num_writers=512 --max_write_ops=512 --max_runtime=1m --leaf_write_goal=2500 --show_ui=false
+ run: go run ./hammer --log_public_key=example.com/log/testdata+33d7b496+AeHTu4Q3hEIMHNqc6fASMsq3rKNx280NI+oO5xCFkkSx --log_url=http://localhost:2025 --max_read_ops=0 --num_writers=512 --max_write_ops=512 --max_runtime=1m --leaf_write_goal=2500 --show_ui=false
  - name: Stop Tessera
  if: ${{ always() }}
  run: docker compose -f ./cmd/conformance/posix/docker/compose.yaml down

.github/workflows/integration_test.yml

@@ -26,10 +26,10 @@ jobs:
  steps:
  - name: Checkout code
  uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
- - name: Start Docker services (tessera-conformance-posix-read-proxy and tessera-conformance-posix)
+ - name: Start Docker services (tessera-conformance-posix)
  run: docker compose -f ./cmd/conformance/posix/docker/compose.yaml up --build --detach
  - name: Run integration test 
- run: go test -v -race ./integration/... --run_integration_test=true --log_url="http://localhost:2024" --write_log_url="http://localhost:2025" --log_public_key="example.com/log/testdata+33d7b496+AeHTu4Q3hEIMHNqc6fASMsq3rKNx280NI+oO5xCFkkSx"
- - name: Stop Docker services (tessera-conformance-posix-read-proxy and tessera-conformance-posix)
+ run: go test -v -race ./integration/... --run_integration_test=true --log_url="http://localhost:2025" --write_log_url="http://localhost:2025" --log_public_key="example.com/log/testdata+33d7b496+AeHTu4Q3hEIMHNqc6fASMsq3rKNx280NI+oO5xCFkkSx"
+ - name: Stop Docker services (tessera-conformance-posix)
  if: ${{ always() }}
  run: docker compose -f ./cmd/conformance/posix/docker/compose.yaml down

cmd/conformance/posix/README.md

@@ -22,13 +22,10 @@ go run ./cmd/conformance/posix \
 In another terminal, run the hammer against the log.
 In this example, we're running 32 writers against the log to add 128 new leaves within 1 minute.
 
-Note that the writes are sent to the HTTP server we brought up in the previous step, but reads are sent directly to the file system.
-
 ```shell
 go run ./hammer \
  --log_public_key=example.com/log/testdata+33d7b496+AeHTu4Q3hEIMHNqc6fASMsq3rKNx280NI+oO5xCFkkSx \
  --write_log_url=http://localhost:2025 \
- --log_url=file:///tmp/mylog2/ \
  --max_read_ops=0 \
  --num_writers=32 \
  --max_write_ops=64 \
@@ -37,7 +34,8 @@ go run ./hammer \
  --show_ui=false
 ```
 
-Optionally, inspect the log using the woodpecker tool to see the contents:
+Optionally, inspect the log on the filesystem using the woodpecker tool to see the contents.
+Note that this reads only from the files on disk, so none of the commands above need to be running for this to work.
 
 ```shell
 go run github.com/mhutchinson/woodpecker@main --custom_log_type=tiles --custom_log_url=file:///${LOG_DIR}/ --custom_log_origin=example.com/log/testdata --custom_log_vkey=${LOG_PUBLIC_KEY}

cmd/conformance/posix/docker/compose.yaml

@@ -1,23 +1,5 @@
 services:
 
- tessera-conformance-posix-read-proxy:
- container_name: tessera-conformance-posix-read-proxy
- image: "busybox:stable"
- ports:
- - "2024:2024"
- command: [
- "/bin/busybox", 
- "httpd", 
- "-f", 
- "-p", 
- "2024",
- "-h",
- "/tmp/tessera-posix-log"
- ]
- volumes:
- - tiles:/tmp/tessera-posix-log
- restart: always 
-
  tessera-conformance-posix:
  container_name: tessera-conformance-posix
  build:

cmd/conformance/posix/main.go

@@ -75,6 +75,9 @@ func main() {
  return
  }
  })
+ // Proxy all GET requests to the filesystem as a lightweight file server.
+ // This makes it easier to test this implementation from another machine.
+ http.Handle("GET /", http.FileServer(http.Dir(*storageDir)))
 
  // Run the HTTP server with the single handler and block until this is terminated
  if err := http.ListenAndServe(*listen, http.DefaultServeMux); err != nil {