Repository navigation
Auto-detect Apple Silicon (MPS) as a device; default --device ignores MPS - #3
Open
MetalstormHS wants to merge 1 commit into
Open
MetalstormHS wants to merge 1 commit into
MetalstormHS wants to merge 1 commit into
Conversation
MetalstormHS
force-pushed
the
mps-device-autodetect
branch
2 times, most recently
from
September 21, 2026 05:58
0f7d15b to
b8f1bff
Compare
MetalstormHS
force-pushed
the
mps-device-autodetect
branch
from
September 21, 2026 06:00
b8f1bff to
702e60f
Compare
GroveMinting
added a commit
to GroveMinting/mini-AGI
that referenced
this pull request
Oct 14, 2026
From PR volotat#3 by MetalstormHS: --device defaulted to "cuda" if torch.cuda.is_available() else "cpu", so on Apple Silicon mini-AGI silently fell back to CPU even though MPS trains fine when passed explicitly - no signal to the user. _auto_device() makes CUDA the first choice (the only backend that enables fused AdamW, bf16 autocast and the VRAM telemetry), MPS the deliberate second, and CPU the safety net. CUDA and CPU behaviour is byte-identical to the old inline default. The test asserts the ordering by stubbing both availability probes, so it holds without CUDA or MPS hardware. Carried here; upstream has not merged it. Co-authored-by: MetalstormHS <30507072+MetalstormHS@users.noreply.github.com>
GroveMinting
added a commit
to GroveMinting/mini-AGI
that referenced
this pull request
Oct 14, 2026
From PR volotat#3 by MetalstormHS: --device defaulted to "cuda" if torch.cuda.is_available() else "cpu", so on Apple Silicon mini-AGI silently fell back to CPU even though MPS trains fine when passed explicitly - no signal to the user. _auto_device() makes CUDA the first choice (the only backend that enables fused AdamW, bf16 autocast and the VRAM telemetry), MPS the deliberate second, and CPU the safety net. CUDA and CPU behaviour is byte-identical to the old inline default. The test asserts the ordering by stubbing both availability probes, so it holds without CUDA or MPS hardware. Carried here; upstream has not merged it. Co-authored-by: MetalstormHS <30507072+MetalstormHS@users.noreply.github.com>
GroveMinting
added a commit
to GroveMinting/mini-AGI
that referenced
this pull request
Oct 14, 2026
From PR volotat#3 by MetalstormHS: --device defaulted to "cuda" if torch.cuda.is_available() else "cpu", so on Apple Silicon mini-AGI silently fell back to CPU even though MPS trains fine when passed explicitly - no signal to the user. _auto_device() makes CUDA the first choice (the only backend that enables fused AdamW, bf16 autocast and the VRAM telemetry), MPS the deliberate second, and CPU the safety net. CUDA and CPU behaviour is byte-identical to the old inline default. The test asserts the ordering by stubbing both availability probes, so it holds without CUDA or MPS hardware. Carried here; upstream has not merged it. Co-authored-by: MetalstormHS <30507072+MetalstormHS@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Auto-detect Apple Silicon (MPS) as a first-class device
What
--devicecurrently defaults to"cuda" if torch.cuda.is_available() else "cpu", so on Apple Silicon the tool silently falls back to CPU even though MPSworks fine when passed explicitly. This patch adds a small
_auto_device()helper and makes MPS the default on Apple Silicon.
Why
MPS is never auto-selected.
--device mpsworks today, but a user who doesn'tknow to pass it gets CPU with no signal. Apple Silicon should be a first-class
target, not a flag you have to remember.
Change
_auto_device()helper: CUDA if available, else MPS if available, else CPU.--devicedefault now calls it.same string the old inline default produced). CPU-only boxes keep the
cpufallback.
fused=(device.type == "cuda")(fused AdamW is CUDA-only),
torch.autocast(..., enabled=(device.type == "cuda"))(bf16 autocast is CUDA-only), and the CUDA VRAM telemetry allalready guard correctly for MPS.
Evidence (Apple Silicon, MPS; torch 2.14.0, python 3.14)
_auto_device()returns"mps"(torch.backends.mps.is_available()is True,torch.cuda.is_available()is False).readwith no--devicebuilds a fresh full-scale model (64 experts,201.33M pool, resident 32, 838.5 MB on disk) and reads at 0.9k char/s.
--device mpsreads at 0.8k char/s (same ballpark, as expected).--device cpustill works (0.5k char/s).streamfailed at the (then pre-existing)SharedPool.prune()signature issue(
TypeError: ... got an unexpected keyword argument 'survival') — since fixedupstream under issue stream crashes on first prune: SharedPool.prune() got an unexpected keyword argument 'survival' #1, and not touched by this patch.
Evidence (NVIDIA RTX 3080, 10 GB; torch 2.6.0+cu124, python 3.11)
"CUDA is still checked first" is a claim about behaviour, so it seemed worth
testing on real hardware rather than arguing from the diff. Verified on a
separate CUDA machine, same patched branch:
_auto_device()returns"cuda". CUDA still wins over MPS, as intended.readwith no--deviceflag builds the model and reads at 4.5k char/s(0.27M characters in 1.0m), held-out loss falling 3.1387 → 2.4679.
process; ~4.1 GB idle after stop.
One gotcha 8000 for anyone reproducing this: a bare
pip install torchgives you aCPU-only build (we got
2.14.0+cpu), andtorch.cuda.is_available()thenreturns False on a perfectly good GPU. The CUDA wheel must be selected
explicitly (
cu124in our case).Note
This is our second contribution to the project. The first, issue #1, reported
the
SharedPool.prune()signature mismatch — fixed upstream — and the missinglicense it flagged has since been added (MIT). Thanks for the quick turnaround
on both.