feat: Bump opendal to 0.52 to support ghac v2 - #2339

Refs: https://github.blog/changelog/2025-03-20-notification-of-upcoming-breaking-changes-in-github-actions/
Refs: mozilla/sccache#2339

Refs: https://github.blog/changelog/2025-03-20-notification-of-upcoming-breaking-changes-in-github-actions/
Refs: mozilla/sccache#2339
PR-URL: #57573
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>

Refs: https://github.blog/changelog/2025-03-20-notification-of-upcoming-breaking-changes-in-github-actions/
Refs: mozilla/sccache#2339
PR-URL: #57573
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>

Refs: https://github.blog/changelog/2025-03-20-notification-of-upcoming-breaking-changes-in-github-actions/
Refs: mozilla/sccache#2339
PR-URL: #57573
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>

Refs: https://github.blog/changelog/2025-03-20-notification-of-upcoming-breaking-changes-in-github-actions/
Refs: mozilla/sccache#2339
PR-URL: nodejs#57573
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>

Refs: https://github.blog/changelog/2025-03-20-notification-of-upcoming-breaking-changes-in-github-actions/
Refs: mozilla/sccache#2339
PR-URL: #57573
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>

Refs: https://github.blog/changelog/2025-03-20-notification-of-upcoming-breaking-changes-in-github-actions/
Refs: mozilla/sccache#2339
PR-URL: #57573
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>

Refs: https://github.blog/changelog/2025-03-20-notification-of-upcoming-breaking-changes-in-github-actions/
Refs: mozilla/sccache#2339
PR-URL: #57573
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>

Refs: https://github.blog/changelog/2025-03-20-notification-of-upcoming-breaking-changes-in-github-actions/
Refs: mozilla/sccache#2339
PR-URL: #57573
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>

Refs: https://github.blog/changelog/2025-03-20-notification-of-upcoming-breaking-changes-in-github-actions/
Refs: mozilla/sccache#2339
PR-URL: #57573
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>

Refs: https://github.blog/changelog/2025-03-20-notification-of-upcoming-breaking-changes-in-github-actions/
Refs: mozilla/sccache#2339
PR-URL: #57573
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>

Refs: https://github.blog/changelog/2025-03-20-notification-of-upcoming-breaking-changes-in-github-actions/
Refs: mozilla/sccache#2339
PR-URL: nodejs/node#57573
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>

Refs: https://github.blog/changelog/2025-03-20-notification-of-upcoming-breaking-changes-in-github-actions/
Refs: mozilla/sccache#2339
PR-URL: nodejs/node#57573
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>

Refs: https://github.blog/changelog/2025-03-20-notification-of-upcoming-breaking-changes-in-github-actions/
Refs: mozilla/sccache#2339
PR-URL: #57573
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>

Refs: https://github.blog/changelog/2025-03-20-notification-of-upcoming-breaking-changes-in-github-actions/
Refs: mozilla/sccache#2339
PR-URL: #57573
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>

CI run 25811044608 failed in the project's cpptrace dependency compile
with:

  sccache: error: Server startup failed: create gha cache failed:
  ConfigInvalid (permanent) at Builder::build => ACTIONS_CACHE_URL
  not found, maybe not in github action environment?

The mozilla-actions/sccache-action runner step now exports the v2 cache
URL (ACTIONS_RESULTS_URL=https://results-receiver.actions.github..., set
by GitHub's rolling 2026 cache backend switch) and no longer sets the
v1 URL (ACTIONS_CACHE_URL). The 0.8.2 sccache binary we ship in the
container only knows about the v1 URL, so it errors at startup even
though docker/run.sh is forwarding the v2 URL into the container.

sccache 0.10.0 ships the opendal 0.52 bump (PR mozilla/sccache#2339)
that adds ghac v2 support. Pin to that version. Earlier sccache 0.9.x
also supports v2 but 0.10.0 is the first stable release that bundles
it, so we don't have to chase pre-releases.

Side-effect: docker layer cache will invalidate the sccache layer in
docker/Dockerfile (Layer 4 — only this layer rebuilds, the apt LLVM 22
install above stays cached). docker/run.sh build picks up the new
image automatically via cmd_image.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(linux): adapt EP build + link for Linux ELF/dlopen

Three things to make the per-model DLL pipeline produce a working ELF
shared object on Linux:

- DLLLinker.cpp: drive ld.lld with the GNU-ABI flag set on Linux
  (-flavor gnu, --shared, -soname, -rpath); pass a Linux PIC object
  through the same lld-as-library API as the Windows path. Crucially
  link crt{i,beginS,endS,n}.o (resolved via `gcc -print-file-name`) so
  the generated DLL has DT_INIT_ARRAY for ctors and DT_FINI_ARRAY for
  dtors — without crtbeginS.o/crtendS.o, dlclose cannot call
  __cxa_finalize(&__dso_handle) and any static dtor in our generated
  ctor list crashes the process at exit.
- lib/Target/LLVM + lib/Runtime + dll CMakeLists.txt: switch from
  Windows MT/MD CRT selection and .lib import-library logic to the
  Linux convention (SHARED libs export everything by default; symbols
  resolved at dlopen time via -Wl,--no-undefined). Link a Linux-side
  list of LLD components via find_package(LLD CONFIG) instead of
  hard-coded .lib paths.
- CompilerDriver: drop the Windows-only assumption that `clang.exe`
  lives next to the running binary; resolve clang via the apt LLVM
  install path on Linux (the Dockerfile installs llvm-22 to
  /usr/lib/llvm-22).

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(linux): set TargetOptions::UseInitArray=true in LLVMBackend

LLVM's default for TargetOptions::UseInitArray is false, which routes
C++ global ctors into the legacy .ctors section. Modern glibc's dynamic
loader silently drops .ctors when DT_INIT_ARRAY is also present (it
just runs the latter), so every `static std::unordered_map`,
`static std::string`, etc. in lib/Runtime/real/ stays at zero-init BSS.
The first emplace then computes `hash % bucket_count(==0)` and dies
with SIGFPE on the first inference op.

This was the root cause of 28/28 SIGFPE E2E_Execute_* failures on the
first Linux port attempt — readelf -d on the model DLL showed
DT_INIT_ARRAY present but empty (length 0), with the actual ctors
sitting in .ctors where the loader ignored them.

Setting UseInitArray=true in createTargetMachine() routes all
`_GLOBAL__sub_I_*` ctors into .init_array so they actually run before
the model entry point.

Co-authored-by: Cursor <cursoragent@cursor.com>

* feat(runner): hip-onnx-runner Linux EP discovery + MORPHIZEN_CONFIG override

Three runner-side adaptations needed for the same binary to load the
MorphiZen EP on Linux:

- EP library path: Windows resolves it as `onnxruntime_morphizen_ep.dll`
  next to the runner; Linux uses libonnxruntime_morphizen_ep.so under
  the install/lib/ tree. Resolve relative to /proc/self/exe at runtime
  rather than hardcoding either layout.
- MORPHIZEN_CONFIG env override: the EP reads its config from
  $MORPHIZEN_CONFIG if set, otherwise a default path next to the EP
  .so. Add an --ep-config CLI flag that forwards to this env so the
  artifact's install/etc/morphizen_config.json can be used without
  copying it into install/bin/.
- Drop the Windows `.exe` suffix assumption in error messages /
  exec_path resolution.

Co-authored-by: Cursor <cursoragent@cursor.com>

* build(linux): GCC + portability fixes (conversion library + runtime)

MSVC tolerates a handful of things GCC rejects; these are all the
diagnostics blocking the Conversion library and lib/Runtime from
compiling cleanly under apt.llvm.org's clang++/g++ on Ubuntu 24.04:

- backend-mlir-compiler/custom-op-mlir/src/InferenceState.h: make the
  PrivateTag passkey actually portable. MSVC permits a private
  constructor + befriended factory via class template, but GCC enforces
  access control more strictly here — switch to the standard friend
  declaration that both compilers accept.
- lib/Runtime/real/linear_attention.cpp: hoist `const` declarations
  above the HIP_CHECK macro that uses `goto`. GCC errors on goto
  jumping over a non-trivially-initialized variable; MSVC didn't.
- lib/Conversion/{HipToLLVM,OnnxToHip}/*.cpp: drop the redundant
  `mlir::hip::` qualification on enums / types that are already
  pulled into scope via `using namespace mlir::hip;`. MSVC ignores
  the duplicate; clang warns under -Wshadow / errors under our
  -Werror policy. Re-run clang-format after.
- cmake/deps.cmake: skip the Windows-only FindXXX fallbacks on Linux;
  cpptrace / GSL / etc. resolve via standard pkg-config there.

Mechanical fanout (~40 files in Conversion/) — same shape of fix per
file, no behavior change.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(cmake): platform gating for Linux HIP / ROCm discovery

3rd-party/custom_kernels/cmake/hip_utils.cmake was hardcoded to discover
HIP via the Windows ROCm SDK layout (HIP_PATH env -> %HIP_PATH%/bin).
Linux installs HIP under /opt/rocm/ (or a sibling-layout therock-dist/
in our case), and enable_language(HIP) needs CMAKE_HIP_COMPILER pinned
to amdclang++ + the ROCM_ROOT env exported before the call.

Forward HIP_PATH to ENV{HIP_PATH} so cmake's enable_language(HIP) sees
it; pin CMAKE_HIP_COMPILER and ROCM_ROOT to the Linux paths when the
host is not WIN32.

Co-authored-by: Cursor <cursoragent@cursor.com>

* feat(docker): containerized build environment with auto-staged install/lib

End-to-end reproducible Linux build environment that mirrors the CI
runner without the host needing apt LLVM 22, TheRock ROCm SDK, or
sccache installed locally. Same image + scripts power both
`./docker/run.sh build` (local dev) and the CI workflow.

- Dockerfile (ubuntu:24.04 noble): apt LLVM 22 + MLIR + Clang + lld
  via apt.llvm.org; build-essential + ninja + cmake + python3-dev
  (OGA's pybind11 build needs CPython headers); sccache 0.8.2 binary
  for compiler-caching hits against the GitHub Actions cache backend.
- entrypoint.sh: re-creates the host UID/GID inside the container at
  startup so files written to the bind-mounted workspace stay owned
  by the host user (no `chown -R` mess on exit).
- run.sh: host-side wrapper providing `image / build / shell / stop`.
  Auto-mounts the workspace (sibling-layout: onnx-hipdnn-ep/ +
  prebuilt-local/ + therock-dist/ + onnxruntime/ + onnxruntime-genai/
  + build/ + install/). Forwards SCCACHE_GHA_ENABLED /
  ACTIONS_CACHE_URL / ACTIONS_RUNTIME_TOKEN into the container when
  set (CI uses these; local builds leave sccache idle).
- build.sh: implements A.4-A.9 of docs/quick_start_linux.md
  (TheRock download, ORT from-source + install into prebuilt-local/,
  protobuf v34, flatbuffers v25, project build, optional OGA build).
  Each step is idempotent via a have_* check on the install marker.
  Submodule update is skipped when 3rd-party/morphizen is already
  populated — the container does not carry SSH credentials for the
  private MorphiZen repo, so the host (via ssh-agent locally, or
  webfactory/ssh-agent in CI) handles `git submodule update --init`.
  A.7b stages TheRock + ORT transitive .so files (via 2-pass ldd)
  into install/lib/ so the install/ tree is self-contained.
  CMAKE_*_COMPILER_LAUNCHER=sccache is wired only when
  SCCACHE_GHA_ENABLED=true is set, so local non-CI runs do not depend
  on the sccache binary doing anything useful.

Co-authored-by: Cursor <cursoragent@cursor.com>

* feat(ci): Linux build workflow invoking ./docker/run.sh

Replace the previous ~755-line inline build (apt LLVM install, ORT
from-source, protobuf, flatbuffers, project cmake, OGA build, 130-line
ldd-transitive-staging) with a single Docker invocation that reuses the
same image + build.sh local developers run via `./docker/run.sh build`.

Net: 755 -> 193 lines.

Layout:
- Checkout to ${github.workspace}/onnx-hipdnn-ep/ so prebuilt-local/,
  therock-dist/, onnxruntime/, onnxruntime-genai/, build/, install/
  can live as siblings — matches the layout docker/run.sh expects
  (WORKSPACE = SOURCE_DIR/..).
- Submodule update happens on the HOST with webfactory/ssh-agent
  loaded; docker/build.sh A.2 detects the populated checkout and
  skips its own submodule update, so the container never needs SSH
  credentials for the private MorphiZen repo.
- mozilla-actions/sccache-action sets ACTIONS_CACHE_URL +
  ACTIONS_RUNTIME_TOKEN + SCCACHE_GHA_ENABLED on the runner;
  docker/run.sh forwards them into the container so the in-container
  sccache binary talks to the same GHA cache backend.
- Caches: prebuilt-local (ORT + protobuf + flatbuffers install,
  keyed on CACHE_VERSION + ORT version), therock-dist (TheRock SDK,
  keyed on THEROCK_VERSION), onnxruntime source clone,
  onnxruntime-genai source clone. Build dir itself is not cached
  (sccache caches compile units at finer granularity).
- Staging: docker/build.sh A.7b already runs a 2-pass ldd inside the
  container and bundles the TheRock + ORT transitive .so set into
  install/lib/, so the artifact step is just `cp install/{bin,lib}`
  + a couple of side gather operations (config from repo etc/,
  wheels from build/).

Co-authored-by: Cursor <cursoragent@cursor.com>

* test(linux): regression check for DT_INIT_ARRAY / DT_FINI_ARRAY

Add a Linux-only ctest E2E_DLL_HasInitFiniArrays that asserts on the
per-model DLL produced by E2E_Compile_test_add_model:

  1. DT_INIT_ARRAY present  (loader will run our ctors at dlopen)
  2. DT_FINI_ARRAY present  (dlclose can run __cxa_finalize / dtors)
  3. NO `.ctors` section    (legacy section absent — its presence
                             would mean LLVM routed ctors into it
                             instead of .init_array, which glibc's
                             loader silently drops)

Both regressions actually shipped during the Linux port:
  - UseInitArray=false default in LLVMBackend → ctors landed in .ctors,
    every `static std::unordered_map` etc. stayed at BSS, first emplace
    SIGFPE'd inside the model (28/28 E2E_Execute_* failures).
  - crtbeginS.o / crtendS.o not linked into the DLL → no FINI_ARRAY,
    dlclose couldn't drain __cxa_finalize, exit() SIGSEGV'd after the
    runner returned `OK - <N> output tensor(s)`.

This catches them at the build stage. Reverse-validated by reverting
either fix in LLVMBackend.cpp / DLLLinker.cpp and confirming the test
flips to FAIL.

Co-authored-by: Cursor <cursoragent@cursor.com>

* docs(linux): extract Docker-first Linux quick start to its own file

Pull the Linux quick start out of docs/quick_start.md into its own
docs/quick_start_linux.md. Mirrors the Windows guide's structure
(Path A: build from source, Path B: download CI artifact) but defaults
to Docker for Path A (`./docker/run.sh build`) rather than asking the
user to recreate the apt-LLVM + sccache + TheRock setup on the host.

Covers:
- Path A: Docker build via ./docker/run.sh image + ./docker/run.sh build
- Path B: gh run download from the linux-gpu-test-package CI artifact
- Test tools: hip-onnx-runner, onnxruntime_perf_test, model_benchmark
- model_benchmark requires `-ml -1` to keep genai_config.json's
  search.max_length (otherwise OGA's MakeGeneratorParams overrides it
  with prompt_len + gen_len and the static-mask shape no longer
  matches decode_*.onnx's hardcoded 256 → "Got 132 Expected 256")
- Troubleshooting + advanced sections

Windows quick_start.md keeps only a 4-line pointer to the Linux file.

Co-authored-by: Cursor <cursoragent@cursor.com>

* chore: bump 3rd-party/morphizen submodule for Linux port fixes

Pin to feat/linux-support tip (b419b9b):

- 76f4373 fix: zero-init OrtEp base for ORT >=1.24 compatibility
  OS-agnostic — the new OrtEp C-API callback pointers ORT 1.24 added
  (GetKernelRegistry, CanRunConcurrent, ...) were read uninitialized
  off the stack/heap by GetPluginEpKernelRegistry. Zero-init via
  OrtEp{} in the member-initializer list.
- b419b9b fix(linux): Plugin::guess_name abs-path + open_plugin_dyn
  dlerror + unit test. Treat names containing '/' as filesystem paths
  (no `lib` prefix); surface dlerror() when dlopen fails; expose
  guess_name for unit testing.

(submodule history was split out of the previous single
"fix(linux): Linux port adaptations" commit so the OS-agnostic OrtEp
fix and the Linux dlopen adaptations are separately cherry-pickable.)

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(linux): address PR review feedback

Addresses review comments on PR #213 + a couple of CI/build follow-ups.

- DLLLinker.cpp:
  - Cache gcc-resolved CRT objects (crti/crtbeginS/crtendS/crtn) +
    libgcc dir + multiarch triplet in a per-process struct so each
    per-model DLL link does 0 popen()s after the first call (was 5).
  - `gcc -print-multiarch` replaces the hard-coded `x86_64-linux-gnu`
    multiarch path so aarch64 / non-Debian distros work without code
    change. Falls back to "x86_64-linux-gnu" when -print-multiarch is
    silent.
  - Wrap user libs + sys libs in `--start-group / --end-group` so static
    archives can satisfy each other regardless of pass order.
  - Hard-fail when crtbeginS.o or crtendS.o are not resolvable via
    `gcc -print-file-name`. Without these, the model DLL has empty
    .fini_array and dlclose SIGSEGVs at process exit; better to surface
    that at link time than ship a broken .so.
  - Drop pre-existing dead C-style `args` array (was for the prior
    in-process lld::lldMain path; the subprocess path uses execArgs).

- lib/Target/LLVM/CMakeLists.txt:
  - find_program(LD_LLD_EXECUTABLE) uses HINTS instead of PATHS so the
    LLVM tree we configure against wins over a stale `/usr/bin/ld.lld-N`
    on PATH.

- Conversion/HipToLLVM/*.cpp:
  - Restore UTF-8 mojibake in 11 comment lines across 6 files (a
    Windows editor with CP1252 view had double-encoded `→`/`—`/`α`/etc.
    on a previous save). Add .gitattributes `working-tree-encoding=UTF-8`
    on .cpp/.h/.md to prevent the same regression.

- 3rd-party/morphizen submodule bump (b419b9b → 7327b56):
  - Per-target `-Wno-error=conversion` on morphizen-pattern + onnx-ir
    instead of the global cmake/deps.cmake FORCE override on
    MORPHIZEN_COMPILER_OPTIONS. Same effect, but only the two targets
    that actually include protobuf-generated *.pb.h headers see the
    knob; the rest of morphizen keeps `-Werror -Wconversion` strict.
  - cmake/deps.cmake override block removed accordingly.

- docker/entrypoint.sh:
  - usermod -aG passes the resolved group name (not the bare numeric
    GID) when adding the host user to /dev/kfd's GID-matched group.
    Some shadow-utils silently no-op on numeric input when no literal-
    digits group name exists.

- docker/build.sh:
  - safe.directory narrowed to SOURCE_DIR + morphizen + ORT_SRC +
    OGA_SRC instead of `*`, so unrelated owner mismatches still surface
    as errors.
  - sccache compiler launcher gated on ACTIONS_CACHE_URL or
    ACTIONS_RESULTS_URL being non-empty (cache v2 rollout). Fixes the
    `ACTIONS_CACHE_URL not found` cold-fail observed on CI run
    25804869662 after sccache --start-server.

- docker/run.sh:
  - Forward ACTIONS_RESULTS_URL alongside ACTIONS_CACHE_URL into the
    container (cache v2).

- docs/quick_start_linux.md:
  - Add Troubleshooting section covering bare-metal `render` group,
    MORPHIZEN_EP_LIB search order, and the gcc-CRT requirement.

- backend-mlir-compiler/test/test_e2e_mlir.cpp:
  - Wrap MSVC-only `_putenv_s` behind a HIPDNN_SETENV macro that uses
    POSIX `setenv` on Linux so the e2e mlir test target builds.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(ci): bump in-container sccache 0.8.2 -> 0.10.0 for GHA cache v2

CI run 25811044608 failed in the project's cpptrace dependency compile
with:

  sccache: error: Server startup failed: create gha cache failed:
  ConfigInvalid (permanent) at Builder::build => ACTIONS_CACHE_URL
  not found, maybe not in github action environment?

The mozilla-actions/sccache-action runner step now exports the v2 cache
URL (ACTIONS_RESULTS_URL=https://results-receiver.actions.github..., set
by GitHub's rolling 2026 cache backend switch) and no longer sets the
v1 URL (ACTIONS_CACHE_URL). The 0.8.2 sccache binary we ship in the
container only knows about the v1 URL, so it errors at startup even
though docker/run.sh is forwarding the v2 URL into the container.

sccache 0.10.0 ships the opendal 0.52 bump (PR mozilla/sccache#2339)
that adds ghac v2 support. Pin to that version. Earlier sccache 0.9.x
also supports v2 but 0.10.0 is the first stable release that bundles
it, so we don't have to chase pre-releases.

Side-effect: docker layer cache will invalidate the sccache layer in
docker/Dockerfile (Layer 4 — only this layer rebuilds, the apt LLVM 22
install above stays cached). docker/run.sh build picks up the new
image automatically via cmd_image.

Co-authored-by: Cursor <cursoragent@cursor.com>

* chore: bump 3rd-party/morphizen submodule (morphizen-core -Werror knob)

03d5163 fix(linux): also disable -Werror=conversion on morphizen-core-static

Patches the gap in the previous bump (7327b56): morphizen-core-static
also includes the protobuf .pb.h headers and needs the same per-target
-Wno-error=conversion knob that morphizen-pattern + onnx-ir got.

Fixes CI run 25817633882 build failure at
3rd-party/morphizen/morphizen-core/CMakeFiles/morphizen-core-static.dir/src/pass_context_imp.cpp.o

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(ci): restore MORPHIZEN_COMPILER_OPTIONS -Wno-error=conversion FORCE

CI run 25823902861 failed after the submodule's per-target narrowing
under-covered the protobuf .pb.h dependency graph: ort-bridge,
morphizen-graph, and likely more morphizen-* targets transitively
include the same generated *.pb.h headers via morphizen-core's PUBLIC
include directories, so they trip the same -Werror=conversion the
"narrow" pattern was meant to fix in morphizen-pattern + onnx-ir-imp.

Rather than chase per-target opt-outs across every transitive consumer
(another whack-a-mole round expected before all of morphizen-graph,
ort-bridge, morphizen-pass-init, ... are covered), restore the
parent-side FORCE override on MORPHIZEN_COMPILER_OPTIONS the user
removed in c3047a7. -Wconversion still fires as a warning
(useful as a code-review signal); only the -Werror promotion is
disabled. Other diagnostics stay strict.

The submodule's per-target knob (03d5163 on morphizen-core-static)
becomes a no-op duplicate under this gate but is harmless — leave it
in place so the submodule remains independently buildable against
a hypothetical downstream that doesn't FORCE the parent cache value.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(ci): flatten ORT headers under prebuilt-local/include for OGA

CI run 25829673460 failed in OGA build (A.9) with:

  CMake Error at cmake/global_variables.cmake:113 (message):
    Expected the ONNX Runtime C API header to be found at
    "/.../prebuilt-local/include/onnxruntime_c_api.h".
    Actual: Not found.

ORT 1.25.1's `cmake --install` installs the public C/C++ API headers
under $PREFIX/include/onnxruntime/*.h (nested), but OGA's
cmake/global_variables.cmake expects them at $ORT_HOME/include/*.h
(flat). The two install layouts are incompatible.

After A.5's `cmake --install` (or after a prebuilt-local cache restore),
mirror the nested headers up to include/ root with `cp -rn`. The -n
flag makes the copy idempotent and never clobbers files already at
include/ root (protobuf / flatbuffers / utf8_range headers from
A.6a / A.6b coexist safely).

The flatten runs OUTSIDE the `if have_ort` gate on purpose: a cache-hit
run skips the install step but may still need to fix a cache populated
by a previous run that lacked the flatten. Idempotent + 0-cost when
already done.

Co-authored-by: Cursor <cursoragent@cursor.com>

* chore: bump 3rd-party/morphizen submodule (ROCm/MorphiZen#214)

03d5163 -> ddce7de: pulls in ddce7de "fix(linux): mark generated
protobuf BINARY_DIR as SYSTEM (root-cause)", which replaces the
per-target -Wno-error=conversion opt-outs in morphizen-pattern +
morphizen-core-static with a SYSTEM include of the BINARY_DIR that
holds the generated *.pb.h. The SYSTEM marking propagates to PUBLIC
consumers (ort-bridge, morphizen-graph, ...) via
INTERFACE_SYSTEM_INCLUDE_DIRECTORIES, so the parent-side FORCE
override on MORPHIZEN_COMPILER_OPTIONS can be dropped (see the
follow-up commit on cmake/deps.cmake).

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(linux): address PR review feedback (round 2)

Four reviewer-flagged items from the second pass on PR #213.

cmake/deps.cmake:
  Drop the MORPHIZEN_COMPILER_OPTIONS FORCE override that was added in
  56d7a85 as a workaround for protobuf -Wconversion in transitive
  morphizen-* consumers. The previous submodule bump (ac74e90 ->
  ROCm/MorphiZen#214) marks the generated-protobuf BINARY_DIR as
  SYSTEM PUBLIC inside morphizen-core-static and SYSTEM PRIVATE inside
  morphizen-pattern, so GCC now auto-suppresses -Wconversion inside
  the .pb.h headers and the SYSTEM marking propagates to ort-bridge /
  morphizen-graph / etc. via INTERFACE_SYSTEM_INCLUDE_DIRECTORIES.
  No parent-side override needed; -Werror -Wconversion stays strict
  on every non-protobuf morphizen source.

docker/build.sh:
  Make the install/ closure check actually catch broken artifacts.
  Pre-fix it only ldd'd libonnxruntime_morphizen_ep.so, and ran BEFORE
  A.9 staged OGA's model_benchmark + libonnxruntime-genai.so* into
  install/, so any unresolved DT_NEEDED on the OGA surface shipped
  unnoticed. Move the verification AFTER A.9, walk every executable
  in install/bin and every lib*.so* in install/lib, fail on any
  "not found" line. Also wrap A.9 with a fresh stage_from_ldd_pass
  call so OGA's transitive deps (pybind11 runtime, tokenizers, ...)
  land in install/lib. Pass 2 of stage_from_ldd_pass now loops to a
  fixed point (was: single shot, could miss second-level deps).

lib/Target/LLVM/CMakeLists.txt + lib/Target/LLVM/DLLLinker.cpp:
  The Linux path switched to subprocess `clang++ -shared -fuse-ld=lld`
  in c3047a7, so the in-process lld::lldMain ELF driver is now dead
  code on Linux. Gate `#include "lld/Common/Driver.h"` and the
  LLD_HAS_DRIVER(coff) registration behind `#ifdef _WIN32`, drop the
  unused LLD_HAS_DRIVER(elf) entirely (Windows only produces COFF),
  and confine find_package(LLD) + lld_libs (lldCommon + lldCOFF) to
  the WIN32 branch of lib/Target/LLVM/CMakeLists.txt. Linux build no
  longer link-pulls liblldELF / liblldCommon / liblldCOFF -- ~5-10 MB
  smaller libhip-compiler.so plus one fewer class of "lld static
  state corruption on dlclose" surface area.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(ci): probe correct flatbuffers cmake-config filename (rebuild loop)

A.6b's `have_flatbuffers` checked for FlatBuffersConfig.cmake
(CamelCase), but flatbuffers v25.12.19 actually installs the file as
flatbuffers-config.cmake (lowercase + hyphen). The check returned
false on every cache-hit CI run, triggering a fresh flatbuffers
build (~2-3 min per run) on top of the restored prebuilt-local/.

Probe the lowercase filename first, fall back to the CamelCase one in
case a future flatbuffers release switches back. find_package(flatbuffers)
accepts either, so this only matters for our skip-step check.

Side-effect: flatbuffers builds + installs are now actually cached
between runs. No semantic change in the artifact contents.

Co-authored-by: Cursor <cursoragent@cursor.com>

* perf(ci): cache OGA install outputs + drop wheel + remove TheRock cache

Three CI-perf tweaks layered on top of the post-c3047a7 stack:

- Cache OGA install outputs (install/bin/model_benchmark +
  install/lib/libonnxruntime-genai.so*) instead of the full build/
  tree, keyed on OGA_REF + ORT version. docker/build.sh A.9 grows a
  have_oga() guard that bridges the GHA cache hit to a skip-build
  path — when both install artifacts are in place (either from a
  cache restore or built earlier this run) A.9 emits `[skip]` and
  saves the 5-10 min OGA build.

- OGA --skip_wheel: drop the OGA python wheel from the Linux build.
  The wheel needs `import onnxruntime` at runtime, but A.5
  deliberately omits ORT --build_wheel (would drag in Python::NumPy
  + dev headers), so the OGA wheel alone is unusable on Linux. Saves
  ~1-3 min pybind11 + pip wheel pack on cold builds. Stage step
  drops the matching `find ... onnxruntime_genai-*.whl` and the
  staging/wheels dir entirely (no other wheels were being shipped
  either — the ORT wheel `find` was a no-op).

- Remove TheRock SDK cache: per CI run 25834159040 timing the
  gzipped GHA cache restore (~1m22s) is no faster than the AMD CDN
  direct download (~1m02s), and dropping it saves ~13 GB of GHA
  cache storage per CACHE_VERSION-key. docker/build.sh A.4's
  have_therock + curl path handles the fresh-install case the same
  way an actions/cache miss would have.

Bonus cleanup: OGA_REF was hard-coded in docker/build.sh AND duplicated
in linux-build.yml's env block, requiring both to be edited in lockstep
to roll OGA forward. Lift it to a `: "${OGA_REF:=<pin>}"` default at
the top of build.sh, forward via docker/run.sh, and reference from the
workflow env. linux-build.yml's env is now the single source of truth;
a bump there auto-invalidates the OGA outputs cache key (which already
includes OGA_REF) and triggers the rebuild path.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(linux): stop bundling TheRock libs in install/lib (match Windows artifact contract)

The Linux artifact was unilaterally bundling ~62 MB of TheRock-origin
.so files (libamdhip64, libhsa-runtime64, librocprofiler-register,
libamd_comgr_loader, librocm_sysdeps_*) into install/lib via
docker/build.sh A.7b's 2-pass ldd staging. This was inconsistent with
the Windows artifact contract, where .github/workflows/gpu-perf-
accuracy-test.yml downloads TheRock separately on the GPU host and
adds it to PATH/LIB — the artifact never contains TheRock.

Three concrete problems with bundling TheRock:

1. libamd_comgr_loader.so.1 (20 KB, shipped) is a stub that dlopens
   the real libamd_comgr.so.3 (9.9 MB, NOT shipped) at runtime. The
   2-pass ldd doesn't see the dlopen, so the stub ends up in the
   artifact while its dlopen target doesn't. Any consumer who didn't
   independently install TheRock then hit:
       implib-gen: libamd_comgr.so.3: failed to load library
                   'libamd_comgr.so.3' via callback
                   'amd_comgr_stub_dlopen'
   followed by an abort() during the first HIP context init.

2. Artifact size drops 310 MB -> ~150 MB (-52%) once the TheRock
   bundle is gone (libamdhip64 alone is 26 MB; rocm_sysdeps_* adds
   another ~4 MB; libhsa-runtime64 + librocprofiler-register account
   for another ~6 MB; symlink doubles approximately).

3. ROCm userland and the host's amdgpu kernel driver share a private
   ioctl ABI that drifts faster than typical shared-lib SONAMEs.
   Bundling a fixed-version libamdhip64 in the artifact creates a
   real version-skew failure mode against an updated kernel. Leaving
   ROCm host-managed keeps userland and kernel in lock-step.

Changes:

- docker/build.sh A.7b: awk filter
      '$3 ~ t || $3 ~ p'         (THEROCK_DIST_DIR or PREBUILT_DIR)
  becomes
      '$3 ~ p'                   (PREBUILT_DIR only).
  ldd's LD_LIBRARY_PATH still includes TheRock so SONAMEs resolve
  (otherwise ldd would print "not found" and the filter would drop
  them anyway); we just stop COPYING the resolved TheRock paths into
  install/lib.

- docker/build.sh closure check: switch from
      LD_LIBRARY_PATH=$INSTALL_DIR/lib
  to
      LD_LIBRARY_PATH=$INSTALL_DIR/lib:$THEROCK_DIST_DIR/lib
  reflecting the actual supported runtime invocation (already
  documented in docs/quick_start_linux.md). Otherwise the check
  fails the build, since TheRock-needing libs in install/lib now
  legitimately rely on $THEROCK_DIST/lib at runtime.

- docs/quick_start_linux.md: drop the "Path A" / "Path C" labels from
  the section headers (the doc has only two entry paths now); add the
  THEROCK_DIST + LD_LIBRARY_PATH export to the "Open a container shell"
  step (was implicit before, now mandatory after the unbundle).

Co-authored-by: Cursor <cursoragent@cursor.com>

* docs: trim verbose comments in docker/

Compress over-explained context in docker/ build scripts. Karpathy §7:
comments explain WHY / non-obvious trade-offs; don't narrate WHAT.

Line counts:
  docker/build.sh        541 -> 398  (-143)
  docker/run.sh          202 -> 169  ( -33)
  docker/entrypoint.sh    74 ->  55  ( -19)
  docker/Dockerfile       97 ->  73  ( -24)
  total                  914 -> 695  (-219, -24%)

Kept:
- step boundaries (A.4/A.5/A.6/A.7/A.7b/A.8/A.9) and their order constraints
- non-obvious gotchas (have_flatbuffers filename mismatch, OGA cache-bridge,
  sccache v1/v2 URL detection, TheRock-not-bundled rationale,
  usermod numeric-GID silent-noop)
- actionable knob descriptions (ORT --build_wheel skip, OGA --skip_wheel,
  Dockerfile cmake 3.29 reason)

Trimmed: layout diagrams duplicated across files, references to specific
historical CI run IDs, paragraph-form WHY that could be a 1-2 line WHY,
restating what the next code block already shows.

No behavior change.

Co-authored-by: Cursor <cursoragent@cursor.com>

* docs: trim verbose comments in build/runtime files

Same pass as the docker/ trim (commit 6a0901a), applied to the cmake +
C++ files touched by this PR. Keep the WHY of non-obvious decisions and
gotcha warnings; drop paragraph-form restatement and historical context.

  cmake/deps.cmake                                   20 -> 11 lines
  3rd-party/custom_kernels/cmake/hip_utils.cmake     48 -> 21 lines
  lib/Target/LLVM/CMakeLists.txt                     45 -> 21 lines
  lib/Target/LLVM/DLLLinker.cpp (linkDLL_Linux)      30 -> 14 lines
  lib/Target/LLVM/LLVMBackend.cpp (UseInitArray)     11 ->  6 lines
  test/e2e/CMakeLists.txt (E2E_DLL_HasInitFini)      21 ->  8 lines

Kept: the failure-mode gist for each non-obvious knob (UseInitArray
SIGFPE, INIT/FINI_ARRAY regression test, GCC -Wconversion on protobuf,
CMAKE_HIP_COMPILER_ROCM_ROOT route, clang++ vs bare clang for libstdc++,
lld CONFIG-vs-find_library, DT_INIT_ARRAY / DT_FINI_ARRAY ELF tags).
Trimmed: cmake-3.31 internal lookup order detail, multi-paragraph
restatement of what the next code block does, "see the long comment in
X" cross-refs.

No behavior change.

Co-authored-by: Cursor <cursoragent@cursor.com>

* ci(linux): print in-container sccache stats at end of build.sh

The screenshot of `Post Setup sccache` showed 0 hits / 0 misses / 0
compile requests, which looked like sccache was idle. It is in fact
working — but the host's sccache 0.15.0 (downloaded by
mozilla-actions/sccache-action) and the in-container 0.10.0
(Dockerfile Layer 4) are separate processes with separate
`--show-stats` state. The action queries the host binary, which
never sees our container's compiles.

The two binaries DO share the same GHA cache backend (both read
ACTIONS_RESULTS_URL + ACTIONS_RUNTIME_TOKEN), so cache writes from
the container are reachable by future runs through either binary.

Print the in-container `sccache --show-stats` at end of build.sh
(only when the launcher was actually wired up) so the real cache
hit/miss/write numbers show up in CI logs.

No behavior change; pure diagnostic.

Co-authored-by: Cursor <cursoragent@cursor.com>

* ci(linux): grant actions:write so sccache GHA cache v2 writes succeed

CI run 25845888765's in-container sccache stats showed:
  Compile requests       314
  Cache misses           314
  Cache write errors     314
  Cache writes             0

The compiles happen and finish (Compilation failures: 0) but every
write to the GHA cache v2 backend errors out, so the next run still
cold-builds — defeating the purpose of running sccache at all.

Root cause: when a workflow declares `permissions:` explicitly, all
scopes default to NONE. We only had `contents: read`. actions/cache
(prebuilt-local, ORT src, OGA src/out) survives this because the
official action implicitly grants what it needs; sccache 0.10.0 talks
to the cache v2 service directly via opendal-ghac and needs the
caller to grant `actions: write` on the token.

No new attack surface — the workflow already runs trusted CI code on
ROCm's own infra and pushes to its own caches via actions/cache.

Co-authored-by: Cursor <cursoragent@cursor.com>

* revert(ci,linux): drop sccache integration

Per #PR-213 review: actions/cache restore of prebuilt-local + ORT/OGA
sources already pulls cold build from ~110 min to ~17 min, which is
acceptable. sccache on top didn't materialize the expected further
speedup: in run 25845888765 it reported 314 compile requests / 314
cache misses / **314 cache write errors**, so writes never landed on
the GHA cache v2 backend and the next run still cold-built. The
follow-up `actions: write` permission grant might have fixed it but
isn't worth carrying the integration overhead for an ~5 min savings
upside on warm runs.

Reverted:
- .github/workflows/linux-build.yml: drop `Setup sccache` step, the
  `SCCACHE_GHA_ENABLED` env, and the `actions: write` permission grant
  we added in ad7f18a.
- docker/Dockerfile: drop Layer 4 sccache 0.10.0 install (saves ~10 MB
  + one curl in the image build).
- docker/run.sh: drop the GHA cache env-forwarding loop
  (SCCACHE_GHA_ENABLED + ACTIONS_CACHE_URL + ACTIONS_RESULTS_URL +
  ACTIONS_RUNTIME_TOKEN).
- docker/build.sh: drop SCCACHE_LAUNCHER_ARGS gate, the
  `${SCCACHE_LAUNCHER_ARGS[@]}` cmake arg, and the in-container
  `sccache --show-stats` diagnostic step.

actions/cache-based caching (prebuilt-local / ORT src / OGA src /
OGA install outputs) is unaffected and remains the primary
CI-warm-path mechanism.

Co-authored-by: Cursor <cursoragent@cursor.com>

* chore: bump 3rd-party/morphizen submodule

Pin to feat/linux-support tip (cfd31a2). New since 03d5163:

  ddce7de fix(linux): mark generated protobuf BINARY_DIR as SYSTEM
          (root-cause fix; replaces the per-target -Wno-error=conversion
          knobs across morphizen-core / morphizen-pattern / ort-bridge
          / morphizen-graph etc. — SYSTEM propagation handles all
          transitive consumers automatically).
  bf16187 fix(linux): address PR review (drop guess_name test, gate
          dlerror surfacing behind MORPHIZEN_DEBUG, doc updates).
  cfd31a2 revert(plugin): keep morphizen_plugin.hpp identical to main
          (the earlier expose-for-test of guess_name went with bf16187's
          test removal; restoring header parity removes the only
          public-API delta against main).

Parent-side cmake/deps.cmake already reflects the SYSTEM-include
solution (no parent FORCE override of MORPHIZEN_COMPILER_OPTIONS).

Co-authored-by: Cursor <cursoragent@cursor.com>

* docs(linux): export LIBRARY_PATH alongside LD_LIBRARY_PATH

The deploy recipe in "Open a container shell" set LD_LIBRARY_PATH (the
runtime loader env) but not LIBRARY_PATH (the build-time linker search
env GCC / clang read). Per-model DLL link calls clang++ -shared which
prepends every LIBRARY_PATH entry as `-L<dir>`, so without it the
name-only `-lhip_custom_kernels` fallback in CompilerDriver missed the
in-tree archive on hosts where the CMake-baked HIP_CUSTOM_KERNELS_LIB_PATH
points at a build-host absolute path that no longer exists. Result:

  ld.lld: error: unable to find library -lhip_custom_kernels

Adding `export LIBRARY_PATH="$ROOT/lib:$THEROCK_DIST/lib"` lets the
fallback resolve cleanly and matches the search-path convention the
clang driver expects for build-time -l resolution.

Verified strictly against the doc recipe (no sudo, no
HIP_CUSTOM_KERNELS_DIR, no manual symlinks): hip-onnx-runner Add_qkv
exits 0 with 1 output tensor; model_benchmark Llama-3.1-8B-seq128
prints TTFT 141.5 ms / decode 12.14 tok/s — within ±5% of the Windows
CI baseline on the same gfx1151 silicon.

Co-authored-by: Cursor <cursoragent@cursor.com>

* ci(linux): tighten artifact staging (drop dead weight, add perf_test)

linux-gpu-test-package consumers reported the staged tree carried
files they don't use:

  staging/bin/morphizen_config.json     (duplicate of staging/etc/<config>;
                                         the submodule's install rule
                                         puts a copy under bin/)
  staging/lib/libHipCInterface.a        (already WHOLE_ARCHIVE'd into
                                         libhip-compiler.so via
                                         dll/CMakeLists.txt; no external
                                         consumer)
  staging/lib/libcpptrace.a             (statically linked into
                                         hip-compiler.so for crash trace;
                                         the .a is FetchContent install
                                         leakage)
  staging/lib/libdwarf.a                (cpptrace's dependency; same)
  staging/lib/libzstd.a                 (libdwarf's dependency; same)
  staging/lib/pkgconfig/{libdwarf,libzstd}.pc  (dev-time pkg-config)

`onnxruntime_perf_test` was also missing from staging/bin/. The
binary lives at prebuilt-local/bin/<exe> after A.5; copy it into
staging/bin/ explicitly.

Net artifact change: -7 files, +1 file. Size drop is modest (~5 MB);
the win is hygiene — every file under staging/ is now reachable by a
documented Linux user flow.

No change to docker/build.sh: local-dev install/ trees can keep the
shipped-by-default content (some developers do statically link
against libHipCInterface.a or pkg-config their dwarf/zstd).

Co-authored-by: Cursor <cursoragent@cursor.com>

* review(linux): address PR #213 review comments

#2 DLLLinker.cpp: parameterize the LLVM version in the
   `clang++ not found` error message. Previously hardcoded "clang-22",
   which would go stale on the next LLVM bump. Inject LLVM_VERSION_MAJOR
   from find_package(LLVM) as the HIPDNN_LLVM_MAJOR compile definition
   and stringify it at the error-message site. (review comment from
   fhanuman.)

#3 DLLLinker.cpp: drop `-Wl,--export-dynamic` from the clang -shared
   invocation. The flag is mainly useful for executables; -shared
   already exports default-visibility symbols, and the MLIR-emitted
   model object uses default visibility (no -fvisibility=hidden). The
   EP's dlsym lookups continue to resolve. Verified by full ctest run
   (57/57 E2E pass, 65 s wall) plus a clean hip-onnx-runner Add_qkv
   round-trip with the rebuilt binary. (review comment from fhanuman.)

#6 LLVMBackend.cpp: add a brief comment clarifying that
   UseInitArray=false is LLVM's backwards-compat default (not a bug)
   and that clang's own driver sets this to true for modern Linux
   (clang/lib/Driver/ToolChains/Gnu.cpp). We bypass the driver and
   drive the TargetMachine API directly, so we have to flip it
   ourselves. (review comment from fhanuman.)

Co-authored-by: Cursor <cursoragent@cursor.com>

* chore: bump 3rd-party/morphizen submodule to main (#214 merged)

ROCm/MorphiZen#214 (Linux port adaptations) was squash-merged into
MorphiZen's main branch. Switch our submodule pointer from the
feat/linux-support tip to the corresponding main commit so the parent
no longer pins to a now-orphaned working branch. The new commit on
main is content-equivalent to the previous pointer (squash preserved
final state).

Local submodule branch also switched from feat/linux-support to main
so `git submodule update` resolves cleanly going forward.

Co-authored-by: Cursor <cursoragent@cursor.com>

---------

Co-authored-by: Cursor <cursoragent@cursor.com>