refactor: migrate to gitea actions

This commit is contained in:
Nils Andreas Svee committed 2026-10-11 10:43:26 +07:00
1 parent 36e69fdd67
commit 45be25b2be
11 files changed
+504 -7

No files matched your search

+111
View File
@@ -0,0 +1,111 @@
# Migration validation — 2026-10-10
Inspected the original Jenkinsfile, Dockerfile, downstream `sqm-autorate-rust`
workflows and package Makefile, Terraform runner definition, and live TheBeast
runner configuration before changing the repository.
## Passed
- YAML parsing; TOML parsing; `sh -n`; ShellCheck; `git diff --check`.
- actionlint 1.7.12, excluding only its unsupported `builtin:checkout` action
syntax. The workflow uses the existing `ubuntu-latest` label. Gitea's documentation and the
existing downstream workflow confirm built-in checkout support.
- Default matrix resolves six entries for stable 25.12.5 and oldstable 24.10.8.
`targets=all` ran upstream target discovery inside the actual job image and
resolved 194 entries. Empty/invalid targets fail. Every default base-image
tag was read successfully from GHCR.
- Docker Compose configuration accepted; Redis healthy with its named volume,
AOF enabled and successful AOF writes. Service and bounded builder were
provisioned on TheBeast without changing the runner configuration.
- Builder inspected: eight-CPU quota, 20 GiB memory/swap limit, one BuildKit
worker step, 50 GiB layer GC budget. A second disposable job container with
fresh Docker client configuration reused the existing daemon and builder.
- Dockerfile build checks completed. Two expected advisory warnings: mandatory
base-image arguments have no defaults, and Docker's secret-name heuristic
mistakes the public `SCCACHE_REDIS_KEY_PREFIX` for a secret.
- `configured` Dockerfile stage built for x86/64 on both OpenWrt releases.
Pinned feeds select Rust 1.94.0 on 25.12.5 and 1.90.0 on 24.10.8. Both retain
`CONFIG_RUST_SCCACHE=y` through defconfig and contain the explicit bootstrap
`build.ccache=sccache` setting with LLVM CI downloads disabled.
- Actual OpenWrt make-variable expansion injects sccache into host and package
compilation, despite unexporting an inherited wrapper.
- Real Cargo library compilation in Docker BuildKit: one Redis miss. A fresh
runtime container using the Docker-network endpoint: one hit, zero misses.
- Recreated the newly provisioned Redis container using the same named volume;
a fresh compiler container still recorded one hit, zero misses. No existing
infrastructure containers were restarted.
- Same source/cache namespace with GNU versus musl target: first musl build
missed; repetition hit. Rust 1.90.0 versus 1.91.0: first 1.91 build missed;
repetition hit. All probes reported zero cache read/write errors.
- A disposable OpenWrt package used the real `Build/Compile/Cargo` helper,
SDK GCC/musl linker, OpenWrt LTO/optimization flags and a local Rust library
dependency. It built successfully in two separate containers: one dependency
miss, then one hit. The linked executable correctly remained non-cacheable.
This used a prebuilt Rust 1.90 toolchain for the probe, not a freshly compiled
SDK toolchain; it is not a full SDK/source-bootstrap acceptance result.
- Retained `ci/verify-cache.sh` passed its separate-container miss/hit checks
against that prebuilt validation toolchain. Its original hardcoded SDK rustc
path was later corrected after the live mediatek/filogic build exposed the
target-specific staging layout; see the follow-up below.
- Publication configuration checked: original Docker Hub namespace/tag format,
verification before push, explicit publish input, temporary credential
directory, no automatic publication trigger. No registry login or push was
attempted.
## Remaining acceptance
1. Enable repository Actions and configure the two Docker Hub secrets. No
runner configuration changes are required.
2. Complete a non-publishing Gitea dispatch for the full default matrix.
The mediatek/filogic 24.10.8 job completed Rust source bootstrap and image
export; its corrected verification passed on the built image. Acceptance
for the other combinations remains pending.
3. With explicit publication approval, run the publishing dispatch, pull the
resulting tags and build the downstream OpenWrt package with the new image.
Actual Docker Hub authorization and publication remain unverified.
4. After acceptance, disable the old Jenkins job and remove `Jenkinsfile`.
It is deliberately retained until the replacement passes these checks.
Remote test logs and disposable probe sources are under
`/tmp/openwrt-sdk-rust-migration` in TheBeast's `gitea-runner` instance. The
Redis volume/service and BuildKit builder remain available for the replacement
pipeline; no cache volume or existing infrastructure was deleted.
## Follow-up: mediatek/filogic 24.10.8 verification
The supplied job 1602 log shows full SDK compilation and image export completed
successfully. Verification failed with exit 127 because it assumed
`staging_dir/host/bin/rustc`. Inspection of that exact built image reproduced
the failure and located Rust/Cargo in
`staging_dir/target-aarch64_cortex-a53_musl/host/bin` instead. Both the workflow
and retained cache probe now resolve the unique executable compiler from the
SDK staging tree, rather than assuming a shared host directory.
Reran the corrected verification on that image: Rust 1.90.0 and Cargo 1.90.0
execute successfully. The retained cache probe reports one Redis miss in its
first container and one hit in a fresh second container, with zero cache
read/write errors. ShellCheck, actionlint (excluding only Gitea's built-in
checkout syntax), and `git diff --check` pass. No rebuild or publication was
performed for this fix.
## Follow-up: mediatek/filogic 25.12.5 bootstrap concurrency
Job 1650 failed inside LLVM compilation with
`c++: fatal error: Killed signal terminated program cc1plus`. Its diagnostic
Make retry still configured `LLVM_PARALLEL_COMPILE_JOBS=32`. The persistent
builder's cgroup reports three OOM kills and its memory limit remains 20 GiB;
the counter is cumulative, while the supplied log identifies the killed
compiler in this job. Outer Make parallelism had not bounded Rust bootstrap's
independent job count.
The SDK Rust Makefile now passes `RUST_BUILD_JOBS` to `x.py --jobs`, defaulting
to eight for subsequent SDK use. `build-rust` supplies eight (or the requested
`JOBS`) on the first attempt and one on the serial diagnostic retry. Rust's
bootstrap uses this job count for LLVM's compile pool and CMake build command.
Built the corrected `configured` image for mediatek/filogic 25.12.5, and
expanded the actual OpenWrt `Host/Compile` recipe at both job counts: it invokes
`x.py --jobs=8` and `x.py --jobs=1` respectively. The same recipe checks pass
against the 24.10.8 feed in a disposable container. ShellCheck, shell syntax,
actionlint and diff checks pass. Full source compilation after this change has
not been rerun, and no images were published or infrastructure limits changed.
+14
View File
@@ -0,0 +1,14 @@
#!/bin/sh
set -eu
: "${SCCACHE_REDIS_ENDPOINT:?Set the persistent sccache Redis endpoint}"
: "${JOBS:=8}"
case "$JOBS" in ''|*[!0-9]*|0) echo 'JOBS must be a positive integer' >&2; exit 1 ;; esac
sccache --start-server
trap 'sccache --show-stats; sccache --stop-server' EXIT
sccache --zero-stats
# Bootstrap/Ninja does not inherit Make's job count. Bound both explicitly.
make package/rust/host/compile -j"$JOBS" RUST_BUILD_JOBS="$JOBS" || \
make package/rust/host/compile -j1 RUST_BUILD_JOBS=1 V=sc
rm -rf build_dir/host/rustc-*/rustc-*/build dl/rustc-*.tar.*
+6
View File
@@ -0,0 +1,6 @@
# Shared by all SDK jobs on this daemon, including simultaneous dispatches.
[worker.oci]
max-parallelism = 1
gc = true
reservedSpace = "10GB"
maxUsedSpace = "50GB"
+40
View File
@@ -0,0 +1,40 @@
services:
cache:
image: redis:7.4.6-alpine
container_name: openwrt-rust-cache
restart: unless-stopped
command:
- redis-server
- --appendonly
- 'yes'
- --appendfsync
- everysec
- --maxmemory
- 6gb
- --maxmemory-policy
- allkeys-lru
- --save
- ''
mem_limit: 8g
memswap_limit: 8g
cpus: 1
ports:
- '127.0.0.1:6379:6379'
volumes:
- cache:/data
networks:
- jobs
healthcheck:
test: ['CMD', 'redis-cli', 'ping']
interval: 5s
timeout: 3s
retries: 12
volumes:
cache:
name: openwrt-rust-sccache
networks:
jobs:
external: true
name: gitea-runner-jobs
+40
View File
@@ -0,0 +1,40 @@
#!/usr/bin/env python3
"""Resolve the same stable/oldstable x target matrix as Jenkins."""
import json
import os
import re
import subprocess
import tempfile
import urllib.request
def resolve(versions, targets):
versions = [versions[k] for k in ("stable_version", "oldstable_version") if versions.get(k)]
if not versions:
raise ValueError("OpenWrt returned no stable or oldstable version")
targets = [t.strip() for t in targets.split(",") if t.strip()]
if not targets:
raise ValueError("No targets selected")
for target in targets:
if not re.fullmatch(r"[a-z0-9_+-]+/[a-z0-9_+-]+", target):
raise ValueError(f"Invalid target: {target!r}")
for version in versions:
if not re.fullmatch(r"[a-zA-Z0-9_.-]+", version):
raise ValueError(f"Invalid version: {version!r}")
return {"include": [{"target": t.replace("/", "-"), "version": v}
for v in versions for t in dict.fromkeys(targets)]}
if __name__ == "__main__":
with urllib.request.urlopen("https://downloads.openwrt.org/.versions.json", timeout=60) as response:
versions = json.load(response)
targets = os.environ.get("TARGETS", "x86/64,mediatek/filogic,octeon/generic")
if targets.strip() == "all":
with tempfile.TemporaryDirectory(prefix="openwrt-targets-") as source:
subprocess.run(["git", "clone", "--depth", "1", "https://github.com/openwrt/openwrt.git", source], check=True)
output = subprocess.check_output(["perl", "scripts/dump-target-info.pl", "targets"], cwd=source, text=True)
targets = ",".join(line.split()[0] for line in output.splitlines() if line.strip())
matrix = json.dumps(resolve(versions, targets), separators=(",", ":"))
print(matrix)
with open(os.environ["GITHUB_OUTPUT"], "a") as output:
output.write(f"matrix={matrix}\n")
+11
View File
@@ -0,0 +1,11 @@
#!/bin/sh
set -eu
docker compose -p openwrt-rust -f ci/cache.compose.yml up -d --wait
if ! docker buildx inspect openwrt-sdk-rust >/dev/null 2>&1; then
docker buildx create --name openwrt-sdk-rust --driver docker-container \
--driver-opt network=host,memory=20g,memory-swap=20g,cpu-period=100000,cpu-quota=800000 \
--buildkitd-config ci/buildkitd.toml \
--buildkitd-flags '--allow-insecure-entitlement network.host' || \
docker buildx inspect openwrt-sdk-rust
fi
docker buildx inspect openwrt-sdk-rust --bootstrap
+39
View File
@@ -0,0 +1,39 @@
#!/bin/sh
# Compile in two disposable containers: the second must read the first's cache.
set -eu
image=${1:?Usage: verify-cache.sh IMAGE}
nonce=$(mktemp -d)
trap 'rmdir "$nonce"' EXIT
prefix="validation:${image##*:}:$(basename "$nonce"):"
for expected in miss hit; do
docker run --rm -i --network gitea-runner-jobs --cpus 2 \
--memory 2g --memory-swap 2g \
-e SCCACHE_REDIS_ENDPOINT=redis://openwrt-rust-cache:6379 \
-e SCCACHE_REDIS_KEY_PREFIX="$prefix" -e EXPECTED="$expected" \
"$image" sh -es <<'PROBE'
rustc=$(find /builder/staging_dir -type f -path '*/host/bin/rustc')
test -x "$rustc"
mkdir /tmp/sccache-probe
cd /tmp/sccache-probe
printf 'pub fn probe(n: u64) -> u64 { n.wrapping_mul(17) }\n' > probe.rs
sccache --start-server
trap 'sccache --stop-server' EXIT
sccache --zero-stats
sccache "$rustc" --crate-name ci_sccache_probe \
--crate-type rlib --emit link,dep-info --out-dir . probe.rs
sccache --show-stats
sccache --show-stats --stats-format json > stats.json
python3 - <<'PY'
import json
import os
data = json.load(open("stats.json"))
assert data["cache_location"].startswith("redis,"), data
stats = data["stats"]
field = "cache_hits" if os.environ["EXPECTED"] == "hit" else "cache_misses"
assert sum(stats[field]["counts"].values()) == 1, data
assert stats["cache_read_errors"] == stats["cache_write_errors"] == 0, data
print("Verified persistent cache", os.environ["EXPECTED"])
PY
PROBE
done