gt50 · Oct 4, 2026 · Oct 4, 2026 · Oct 4, 2026
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
@@ -9,7 +9,7 @@ jobs:
     strategy:
       fail-fast: false
       matrix:
-        os: [ubuntu-latest, windows-latest]
+        os: [windows-latest]
         python: ["3.11"]
     runs-on: ${{ matrix.os }}
     steps:

diff --git a/.gitignore b/.gitignore
@@ -28,3 +28,6 @@ venv/
 .idea/
 .pytest_cache/
 *.egg-info/
+
+# other weights directories: weights-strix/ (strix_halo_large.yaml), backups
+weights-*/
diff --git a/README.md b/README.md
@@ -474,25 +474,34 @@ python tools/bench_hw.py --config configs/strix_halo_large.yaml  # measure befor
 
 | key | what it does |
 |---|---|
-| `gpu_mem_gb` | the budget growth's ROOM brake measures against. **Set it to what the GPU may really use** - the Variable Graphics Memory in AMD Software on Windows, the GTT limit on Linux. torch reports the firmware carve-out as the card's size, which on an APU says little |
+| `gpu_mem_gb` | the budget growth's ROOM brake measures against. **Set it to what the GPU may really use** - the Variable Graphics Memory set in AMD Software. On an APU the size torch reports need 
8000
not match it |
 | `ram_tier_on_gpu` | holds the expert pool's RAM tier as device tensors, so paging an expert onto the card is a device-to-device copy rather than a blocking host transfer. System memory *is* the card's memory here |
 | `rocm_aotriton` | `TORCH_ROCM_AOTRITON_ENABLE_EXPERIMENTAL=1` - the fast SDPA kernels on RDNA 3.5. Without them the cached forward falls back to math attention, which builds the full score matrix at every row |
-| `tunableop` | `PYTORCH_TUNABLEOP_ENABLED=1` - tunes the expert dispatch's batched matmuls once and keeps the results in `runs/` |
+| `tunableop` | `PYTORCH_TUNABLEOP_ENABLED=1`. Off in both profiles: the expert dispatch's matmuls change shape every forward, so it re-tunes nearly every call - a held-out pass ran past half an hour on the 8060S |
 | `expandable_segments` | the allocator setting train.py has always used; turn it off if your ROCm build warns it is unsupported |
 
 Every run prints a `hardware:` line naming the device, backend, whether memory is unified, the budget and where it came from.
 
-**Install.** Stock PyTorch wheels do not cover gfx1151. AMD publishes builds for it, for Linux and Windows, from its own index:
+**Install.** Stock PyTorch wheels do not cover gfx1151. AMD publishes Windows builds for it from its own index:
 
 ```bash
 pip install --index-url https://rocm.nightlies.amd.com/v2/gfx1151/ --pre torch
 ```
 
 Then `pip install -e .[serve,corpora]` for the rest. A later `pip install -U` can pull a CUDA torch from PyPI over the ROCm one, so re-run the line above after any upgrade. Check with `python -c "import torch; print(torch.version.hip, torch.cuda.get_device_properties(0))"`, which should name gfx1151.
 
-**Memory.** On Linux, a small BIOS carve-out with a large GTT allowance is the usual setup - raise the TTM limit on the kernel command line (`ttm.pages_limit` counts 4 KiB pages: 27648000 is about 105 GiB) and set `gpu_mem_gb` a little below it. On Windows, set Variable Graphics Memory in AMD Software and set `gpu_mem_gb` to match.
+**Memory.** Set Variable Graphics Memory in AMD Software and set `gpu_mem_gb` to match. This port is tested on Windows only.
 
-**What to expect.** The 8060S is in the same compute class as the reference RTX 3070 Laptop, with less memory bandwidth, so characters per second will be similar or lower - not higher. What it has is memory, roughly twelve times an 8 GB card's, and the profiles spend it on the axis this model is built around: `configs/strix_halo.yaml` keeps the released model's shape and gives it 64 experts per forward instead of 32, the whole pool in device memory, an 8k reach and a 100 GB disk ceiling; `configs/strix_halo_large.yaml` starts a new model with twice the experts at birth, four times the resident set and a trunk 768 wide. Raising `resident` changes routing, not only memory - a forward is admitted more of what its text asks for - so compare held-out loss before and after rather than assuming it is free. Its numbers are starting points: run `tools/bench_hw.py` on the machine first.
+**What to expect.** The 8060S is in the same compute class as the reference RTX 3070 Laptop, with less memory bandwidth, so characters per second will be similar or lower - not higher. What it has is memory, roughly twelve times an 8 GB card's.
+
+`configs/strix_halo.yaml` continues the released model and changes nothing that decides what it computes: it holds the whole pool in device memory and raises the disk ceiling. Two things that look free are not, and were measured on the released weights:
+
+- **`resident` stays 32.** At 64 the same weights scored 0.92 nats on held-out chat instead of 0.60, and 1.19 on code instead of 0.71 - the model learned to choose within 32, and a larger card changes what each character routes through.
+- **`context_end` stays 4096.** Held-out is read at the ceiling, and at 8192 the released weights scored 1.66 nats against 0.62: positions past 4,096 are ones it has never been trained on.
+
+`configs/strix_halo_large.yaml` starts a new model trained with the larger card from the beginning - twice the experts at birth, four times the resident set, a trunk 768 wide and an 8k reach. Its numbers are starting points: run `tools/bench_hw.py` on the machine first.
+
+Measured on a Ryzen AI Max+ 395 (Windows, ROCm 7.13 nightly): `strix_halo.yaml` reads about 1,250 characters a second while training, `strix_halo_large.yaml` about 530 at 17.4 GB peak, and paging an expert onto the card takes 0.05 ms from the device-resident tier against 0.5 ms from host memory.
 
 ### Everything else
 

diff --git a/configs/strix_halo.yaml b/configs/strix_halo.yaml
@@ -5,9 +5,8 @@
 #
 # The model block below is the released model's shape, so this profile
 # continues weights/ as they are (including the Hugging Face snapshots). Only
-# run-time settings change: more experts per forward, the whole pool held in
-# device memory, a longer reach, a larger disk ceiling. For a NEW, larger
-# model see configs/strix_halo_large.yaml.
+# run-time settings change: the whole pool held in device memory and a larger
+# disk ceiling. For a NEW, larger model see configs/strix_halo_large.yaml.
 #
 # Expect characters per second close to an RTX 3070 Laptop's, not above it:
 # the 8060S is in the same compute class with less memory bandwidth. What it
@@ -58,7 +57,11 @@ model:
   # and grows only when the model is still gaining from the far end of what
   # it has; context_end is the ceiling and binds at load and at every growth.
   context_start: 2048     # where a fresh model begins
-  context_end: 8192      # rotary, so the existing model can extend into it
+  context_end: 4096      # the released model's. NOT raised: held-out is read
+                          # at this ceiling, and at 8192 the released weights
+                          # scored 1.66 nats against ~0.6 - positions past
+                          # 4,096 are ones it has never been trained on.
+                          # Raise it only alongside training that ramps into it
 
   context_step: 1                     # characters added per growth
   context_grow_every_chars: 100_000   # characters between growths
@@ -86,10 +89,12 @@ pool:
   #
   # Three tiers: disk holds every expert, RAM caches the recently wanted,
   # VRAM holds what the current forward admitted. Only VRAM is scarce.
-  resident: 64            # experts one forward may use - TWICE the 8 GB card's.
-                          # Changes routing, not only memory: a forward is
-                          # admitted more of what its text asks for
-                          # holding weights, gradient and Adam's moments
+  resident: 32            # experts one forward may use - the released model's
+                          # own. NOT raised here: measured on the released
+                          # weights, 64 took held-out chat from 0.60 to 0.92
+                          # and code from 0.71 to 1.19 nats, because the model
+                          # learned to choose within 32. A larger card is for a
+                          # model trained with it - see strix_halo_large.yaml
   ram_cache: 0            # 0 = every expert stays in the tier, which here
                           # is device memory: disk is read once per expert
 
@@ -218,12 +223,10 @@ hardware:
   # AMD Ryzen AI Max (Strix Halo): Radeon 8060S, gfx1151, RDNA 3.5, sharing
   # up to 128 GB of LPDDR5X with the CPU at ~256 GB/s.
   #
-  # SET THIS to what the GPU may actually use on your machine:
-  #   Windows - the Variable Graphics Memory set in AMD Software (e.g. 96)
-  #   Linux   - the GTT limit the kernel allows (ttm.pages_limit), less
-  #             headroom for the OS; or the BIOS UMA size if you set it large
-  # torch reports the firmware carve-out as the card's size, which on an APU
-  # says little about what can be allocated, so growth measures against this.
+  # SET THIS to what the GPU may actually use on your machine: the Variable
+  # Graphics Memory set in AMD Software (e.g. 96). On an APU the size torch
+  # reports need not match it - on the test machine it reported 99.7 GB while
+  # Windows kept 63.6 GB for itself - so growth measures against this.
   gpu_mem_gb: 96
   unified: true
   ram_tier_on_gpu: true      # the expert RAM tier lives on the device: a
@@ -232,4 +235,9 @@ hardware:
                              # unsupported (some Windows builds)
   rocm_aotriton: true        # fast SDPA kernels on RDNA 3.5; without them a
                              # 4k window builds its whole T x T scores per row
-  tunableop: true            # tune each GEMM shape once, results in runs/
+  tunableop: false           # OFF on purpose. The expert dispatch's batched
+                             # matmuls change shape with every forward - an
+                             # expert's batch is however many characters chose
+                             # it - so TunableOp re-tunes nearly every call:
+                             # measured, a held-out pass that should take
+                             # minutes ran past half an hour, 2,000+ shapes in
diff --git a/configs/strix_halo_large.yaml b/configs/strix_halo_large.yaml
@@ -221,12 +221,10 @@ hardware:
   # AMD Ryzen AI Max (Strix Halo): Radeon 8060S, gfx1151, RDNA 3.5, sharing
   # up to 128 GB of LPDDR5X with the CPU at ~256 GB/s.
   #
-  # SET THIS to what the GPU may actually use on your machine:
-  #   Windows - the Variable Graphics Memory set in AMD Software (e.g. 96)
-  #   Linux   - the GTT limit the kernel allows (ttm.pages_limit), less
-  #             headroom for the OS; or the BIOS UMA size if you set it large
-  # torch reports the firmware carve-out as the card's size, which on an APU
-  # says little about what can be allocated, so growth measures against this.
+  # SET THIS to what the GPU may actually use on your machine: the Variable
+  # Graphics Memory set in AMD Software (e.g. 96). On an APU the size torch
+  # reports need not match it - on the test machine it reported 99.7 GB while
+  # Windows kept 63.6 GB for itself - so growth measures against this.
   gpu_mem_gb: 96
   unified: true
   ram_tier_on_gpu: true      # the expert RAM tier lives on the device: a
@@ -235,4 +233,9 @@ hardware:
                              # unsupported (some Windows builds)
   rocm_aotriton: true        # fast SDPA kernels on RDNA 3.5; without them a
                              # 4k window builds its whole T x T scores per row
-  tunableop: true            # tune each GEMM shape once, results in runs/
+  tunableop: false           # OFF on purpose. The expert dispatch's batched
+                             # matmuls change shape with every forward - an
+                             # expert's batch is however many characters chose
+                             # it - so TunableOp re-tunes nearly every call:
+                             # measured, a held-out pass that should take
+                             # minutes ran past half an hour, 2,000+ shapes in
diff --git a/minagi/device.py b/minagi/device.py
@@ -7,10 +7,10 @@
 
   - the memory budget. Growth refuses to add an expert once the peak this
     process allocated passes a share of the card. A discrete card reports its
-    size honestly; an APU such as Strix Halo (gfx1151) reports whatever the
-    firmware carved out, which can be 512 MB or 96 GB on the same machine and
-    says nothing about the GTT memory the driver will also hand out. So the
-    budget can be stated, and a stated budget wins.
+    size honestly; on an APU such as Strix Halo (gfx1151) the size depends on
+    how much system memory the driver lends the GPU, and what torch reports
+    need not match what can really be allocated. So the budget can be stated,
+    and a stated budget wins.
   - whether CPU and GPU share memory. When they do, the RAM tier of the
     expert pool can sit on the device itself, and paging an expert onto the
     card becomes a device-to-device copy instead of a blocking host transfer.