Repository navigation
Enable bf16 autocast on Intel XPU devices (Arc) - #31
Open
TomTheHand wants to merge 1 commit into
Open
TomTheHand wants to merge 1 commit into
TomTheHand wants to merge 1 commit into
Conversation
Treat XPU as a bf16-native accelerator: autocast follows the configured dtype instead of silently staying fp32, and XPU never consults the CUDA/ROCm capability queries (a colocated NVIDIA card must not decide Intel precision policy, including when MINAGI_GPU_BF16=0 disables the native-bf16 predicate). MINAGI_GPU_BF16=1/0 still overrides. Validated on Arc Pro B60 with torch 2.14.1+xpu: bf16 forward/backward and an SGD update run and stay finite. Adds CPU regression tests and an opt-in MINAGI_TEST_XPU=1 hardware smoke test.
Author
|
If it's not obvious enough, the above was written by AI 😅 Sorry. With that said, mini-AGI nearly runs on Intel Arc GPUs out of the box, and the attached changes seal the deal. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Treat
xpuas a bf16-native accelerator so the configured precision actually applies on Intel Arc:gpu_bf16_native()returns True for XPU (Intel XMX computes bf16 in hardware; validated on Arc Pro B60).autocast_on()treatsxpulikecuda: bf16/fp16 autocast is used unless fp32 was explicitly selected._gpu_info()returns None for XPU, so CUDA/ROCm property queries and the VRAM-based fp32 fallback can never consult a colocated different-vendor GPU. This matters even withMINAGI_GPU_BF16=0: the override only disables the native-bf16 predicate, and without the guard the CUDA memory check would otherwise decide XPU policy.MINAGI_GPU_BF16=1|0still overrides, and CUDA/ROCm/CPU behavior is unchanged.Why
On a Battlemage Arc (B60),
torch.xpu.is_bf16_supported()reports False (a known capability-query gap that fails closed) even though the hardware computes bf16 fine - measured 7.4x matmul speedup fp32→bf16 on this card. Trusting that flag leaves Arc in fp32 for no benefit. We measured before switching, per the same discipline the module already applies to RDNA2.Tests
tests/test_precision.py: CPU/mock coverage for device-type forms, dtype selection, autocast dispatch, env overrides, and unchanged CUDA/ROCm/CPU policy (5 tests).tests/test_precision_xpu.py: opt-in (MINAGI_TEST_XPU=1) tiny-tensor bf16 forward/backward + SGD update on real XPU - no checkpoints, no training data.Hardware validation covers this B60 only; no cross-vendor or performance claims beyond the measurement above.