refactor: migrate to gitea actions
This commit is contained in:
11 files changed
+476
-7
No files matched your search
@@ -0,0 +1,89 @@
|
||||
# Migration validation — 2026-10-10
|
||||
|
||||
Inspected the original Jenkinsfile, Dockerfile, downstream `sqm-autorate-rust`
|
||||
workflows and package Makefile, Terraform runner definition, and live TheBeast
|
||||
runner configuration before changing the repository.
|
||||
|
||||
## Passed
|
||||
|
||||
- YAML parsing; TOML parsing; `sh -n`; ShellCheck; `git diff --check`.
|
||||
- actionlint 1.7.12, excluding only its unsupported `builtin:checkout` action
|
||||
syntax. The workflow uses the existing `ubuntu-latest` label. Gitea's documentation and the
|
||||
existing downstream workflow confirm built-in checkout support.
|
||||
- Default matrix resolves six entries for stable 25.12.5 and oldstable 24.10.8.
|
||||
`targets=all` ran upstream target discovery inside the actual job image and
|
||||
resolved 194 entries. Empty/invalid targets fail. Every default base-image
|
||||
tag was read successfully from GHCR.
|
||||
- Docker Compose configuration accepted; Redis healthy with its named volume,
|
||||
AOF enabled and successful AOF writes. Service and bounded builder were
|
||||
provisioned on TheBeast without changing the runner configuration.
|
||||
- Builder inspected: eight-CPU quota, 20 GiB memory/swap limit, one BuildKit
|
||||
worker step, 50 GiB layer GC budget. A second disposable job container with
|
||||
fresh Docker client configuration reused the existing daemon and builder.
|
||||
- Dockerfile build checks completed. Two expected advisory warnings: mandatory
|
||||
base-image arguments have no defaults, and Docker's secret-name heuristic
|
||||
mistakes the public `SCCACHE_REDIS_KEY_PREFIX` for a secret.
|
||||
- `configured` Dockerfile stage built for x86/64 on both OpenWrt releases.
|
||||
Pinned feeds select Rust 1.94.0 on 25.12.5 and 1.90.0 on 24.10.8. Both retain
|
||||
`CONFIG_RUST_SCCACHE=y` through defconfig and contain the explicit bootstrap
|
||||
`build.ccache=sccache` setting with LLVM CI downloads disabled.
|
||||
- Actual OpenWrt make-variable expansion injects sccache into host and package
|
||||
compilation, despite unexporting an inherited wrapper.
|
||||
- Real Cargo library compilation in Docker BuildKit: one Redis miss. A fresh
|
||||
runtime container using the Docker-network endpoint: one hit, zero misses.
|
||||
- Recreated the newly provisioned Redis container using the same named volume;
|
||||
a fresh compiler container still recorded one hit, zero misses. No existing
|
||||
infrastructure containers were restarted.
|
||||
- Same source/cache namespace with GNU versus musl target: first musl build
|
||||
missed; repetition hit. Rust 1.90.0 versus 1.91.0: first 1.91 build missed;
|
||||
repetition hit. All probes reported zero cache read/write errors.
|
||||
- A disposable OpenWrt package used the real `Build/Compile/Cargo` helper,
|
||||
SDK GCC/musl linker, OpenWrt LTO/optimization flags and a local Rust library
|
||||
dependency. It built successfully in two separate containers: one dependency
|
||||
miss, then one hit. The linked executable correctly remained non-cacheable.
|
||||
This used a prebuilt Rust 1.90 toolchain for the probe, not a freshly compiled
|
||||
SDK toolchain; it is not a full SDK/source-bootstrap acceptance result.
|
||||
- Retained `ci/verify-cache.sh` passed its separate-container miss/hit checks
|
||||
against that prebuilt validation toolchain. Its original hardcoded SDK rustc
|
||||
path was later corrected after the live mediatek/filogic build exposed the
|
||||
target-specific staging layout; see the follow-up below.
|
||||
- Publication configuration checked: original Docker Hub namespace/tag format,
|
||||
verification before push, explicit publish input, temporary credential
|
||||
directory, no automatic publication trigger. No registry login or push was
|
||||
attempted.
|
||||
|
||||
## Remaining acceptance
|
||||
|
||||
1. Enable repository Actions and configure the two Docker Hub secrets. No
|
||||
runner configuration changes are required.
|
||||
2. Complete a non-publishing Gitea dispatch for the full default matrix.
|
||||
The mediatek/filogic 24.10.8 job completed Rust source bootstrap and image
|
||||
export; its corrected verification passed on the built image. Acceptance
|
||||
for the other combinations remains pending.
|
||||
3. With explicit publication approval, run the publishing dispatch, pull the
|
||||
resulting tags and build the downstream OpenWrt package with the new image.
|
||||
Actual Docker Hub authorization and publication remain unverified.
|
||||
4. After acceptance, disable the old Jenkins job and remove `Jenkinsfile`.
|
||||
It is deliberately retained until the replacement passes these checks.
|
||||
|
||||
Remote test logs and disposable probe sources are under
|
||||
`/tmp/openwrt-sdk-rust-migration` in TheBeast's `gitea-runner` instance. The
|
||||
Redis volume/service and BuildKit builder remain available for the replacement
|
||||
pipeline; no cache volume or existing infrastructure was deleted.
|
||||
|
||||
## Follow-up: mediatek/filogic 24.10.8 verification
|
||||
|
||||
The supplied job 1602 log shows full SDK compilation and image export completed
|
||||
successfully. Verification failed with exit 127 because it assumed
|
||||
`staging_dir/host/bin/rustc`. Inspection of that exact built image reproduced
|
||||
the failure and located Rust/Cargo in
|
||||
`staging_dir/target-aarch64_cortex-a53_musl/host/bin` instead. Both the workflow
|
||||
and retained cache probe now resolve the unique executable compiler from the
|
||||
SDK staging tree, rather than assuming a shared host directory.
|
||||
|
||||
Reran the corrected verification on that image: Rust 1.90.0 and Cargo 1.90.0
|
||||
execute successfully. The retained cache probe reports one Redis miss in its
|
||||
first container and one hit in a fresh second container, with zero cache
|
||||
read/write errors. ShellCheck, actionlint (excluding only Gitea's built-in
|
||||
checkout syntax), and `git diff --check` pass. No rebuild or publication was
|
||||
performed for this fix.
|
||||
@@ -0,0 +1,13 @@
|
||||
#!/bin/sh
|
||||
set -eu
|
||||
|
||||
: "${SCCACHE_REDIS_ENDPOINT:?Set the persistent sccache Redis endpoint}"
|
||||
: "${JOBS:=8}"
|
||||
case "$JOBS" in ''|*[!0-9]*|0) echo 'JOBS must be a positive integer' >&2; exit 1 ;; esac
|
||||
|
||||
sccache --start-server
|
||||
trap 'sccache --show-stats; sccache --stop-server' EXIT
|
||||
sccache --zero-stats
|
||||
# Preserve Jenkins' serial diagnostic retry. Both attempts use the same cache.
|
||||
make package/rust/host/compile -j"$JOBS" || make package/rust/host/compile -j1 V=sc
|
||||
rm -rf build_dir/host/rustc-*/rustc-*/build dl/rustc-*.tar.*
|
||||
@@ -0,0 +1,6 @@
|
||||
# Shared by all SDK jobs on this daemon, including simultaneous dispatches.
|
||||
[worker.oci]
|
||||
max-parallelism = 1
|
||||
gc = true
|
||||
reservedSpace = "10GB"
|
||||
maxUsedSpace = "50GB"
|
||||
@@ -0,0 +1,40 @@
|
||||
services:
|
||||
cache:
|
||||
image: redis:7.4.6-alpine
|
||||
container_name: openwrt-rust-cache
|
||||
restart: unless-stopped
|
||||
command:
|
||||
- redis-server
|
||||
- --appendonly
|
||||
- 'yes'
|
||||
- --appendfsync
|
||||
- everysec
|
||||
- --maxmemory
|
||||
- 6gb
|
||||
- --maxmemory-policy
|
||||
- allkeys-lru
|
||||
- --save
|
||||
- ''
|
||||
mem_limit: 8g
|
||||
memswap_limit: 8g
|
||||
cpus: 1
|
||||
ports:
|
||||
- '127.0.0.1:6379:6379'
|
||||
volumes:
|
||||
- cache:/data
|
||||
networks:
|
||||
- jobs
|
||||
healthcheck:
|
||||
test: ['CMD', 'redis-cli', 'ping']
|
||||
interval: 5s
|
||||
timeout: 3s
|
||||
retries: 12
|
||||
|
||||
volumes:
|
||||
cache:
|
||||
name: openwrt-rust-sccache
|
||||
|
||||
networks:
|
||||
jobs:
|
||||
external: true
|
||||
name: gitea-runner-jobs
|
||||
@@ -0,0 +1,40 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Resolve the same stable/oldstable x target matrix as Jenkins."""
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import subprocess
|
||||
import tempfile
|
||||
import urllib.request
|
||||
|
||||
|
||||
def resolve(versions, targets):
|
||||
versions = [versions[k] for k in ("stable_version", "oldstable_version") if versions.get(k)]
|
||||
if not versions:
|
||||
raise ValueError("OpenWrt returned no stable or oldstable version")
|
||||
targets = [t.strip() for t in targets.split(",") if t.strip()]
|
||||
if not targets:
|
||||
raise ValueError("No targets selected")
|
||||
for target in targets:
|
||||
if not re.fullmatch(r"[a-z0-9_+-]+/[a-z0-9_+-]+", target):
|
||||
raise ValueError(f"Invalid target: {target!r}")
|
||||
for version in versions:
|
||||
if not re.fullmatch(r"[a-zA-Z0-9_.-]+", version):
|
||||
raise ValueError(f"Invalid version: {version!r}")
|
||||
return {"include": [{"target": t.replace("/", "-"), "version": v}
|
||||
for v in versions for t in dict.fromkeys(targets)]}
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
with urllib.request.urlopen("https://downloads.openwrt.org/.versions.json", timeout=60) as response:
|
||||
versions = json.load(response)
|
||||
targets = os.environ.get("TARGETS", "x86/64,mediatek/filogic,octeon/generic")
|
||||
if targets.strip() == "all":
|
||||
with tempfile.TemporaryDirectory(prefix="openwrt-targets-") as source:
|
||||
subprocess.run(["git", "clone", "--depth", "1", "https://github.com/openwrt/openwrt.git", source], check=True)
|
||||
output = subprocess.check_output(["perl", "scripts/dump-target-info.pl", "targets"], cwd=source, text=True)
|
||||
targets = ",".join(line.split()[0] for line in output.splitlines() if line.strip())
|
||||
matrix = json.dumps(resolve(versions, targets), separators=(",", ":"))
|
||||
print(matrix)
|
||||
with open(os.environ["GITHUB_OUTPUT"], "a") as output:
|
||||
output.write(f"matrix={matrix}\n")
|
||||
@@ -0,0 +1,11 @@
|
||||
#!/bin/sh
|
||||
set -eu
|
||||
docker compose -p openwrt-rust -f ci/cache.compose.yml up -d --wait
|
||||
if ! docker buildx inspect openwrt-sdk-rust >/dev/null 2>&1; then
|
||||
docker buildx create --name openwrt-sdk-rust --driver docker-container \
|
||||
--driver-opt network=host,memory=20g,memory-swap=20g,cpu-period=100000,cpu-quota=800000 \
|
||||
--buildkitd-config ci/buildkitd.toml \
|
||||
--buildkitd-flags '--allow-insecure-entitlement network.host' || \
|
||||
docker buildx inspect openwrt-sdk-rust
|
||||
fi
|
||||
docker buildx inspect openwrt-sdk-rust --bootstrap
|
||||
@@ -0,0 +1,39 @@
|
||||
#!/bin/sh
|
||||
# Compile in two disposable containers: the second must read the first's cache.
|
||||
set -eu
|
||||
image=${1:?Usage: verify-cache.sh IMAGE}
|
||||
nonce=$(mktemp -d)
|
||||
trap 'rmdir "$nonce"' EXIT
|
||||
prefix="validation:${image##*:}:$(basename "$nonce"):"
|
||||
for expected in miss hit; do
|
||||
docker run --rm -i --network gitea-runner-jobs --cpus 2 \
|
||||
--memory 2g --memory-swap 2g \
|
||||
-e SCCACHE_REDIS_ENDPOINT=redis://openwrt-rust-cache:6379 \
|
||||
-e SCCACHE_REDIS_KEY_PREFIX="$prefix" -e EXPECTED="$expected" \
|
||||
"$image" sh -es <<'PROBE'
|
||||
rustc=$(find /builder/staging_dir -type f -path '*/host/bin/rustc')
|
||||
test -x "$rustc"
|
||||
mkdir /tmp/sccache-probe
|
||||
cd /tmp/sccache-probe
|
||||
printf 'pub fn probe(n: u64) -> u64 { n.wrapping_mul(17) }\n' > probe.rs
|
||||
sccache --start-server
|
||||
trap 'sccache --stop-server' EXIT
|
||||
sccache --zero-stats
|
||||
sccache "$rustc" --crate-name ci_sccache_probe \
|
||||
--crate-type rlib --emit link,dep-info --out-dir . probe.rs
|
||||
sccache --show-stats
|
||||
sccache --show-stats --stats-format json > stats.json
|
||||
python3 - <<'PY'
|
||||
import json
|
||||
import os
|
||||
|
||||
data = json.load(open("stats.json"))
|
||||
assert data["cache_location"].startswith("redis,"), data
|
||||
stats = data["stats"]
|
||||
field = "cache_hits" if os.environ["EXPECTED"] == "hit" else "cache_misses"
|
||||
assert sum(stats[field]["counts"].values()) == 1, data
|
||||
assert stats["cache_read_errors"] == stats["cache_write_errors"] == 0, data
|
||||
print("Verified persistent cache", os.environ["EXPECTED"])
|
||||
PY
|
||||
PROBE
|
||||
done
|
||||
Reference in new issue
Block a user