Repository navigation
Conversation
Follow-up to the 1-D indexedarray gradient support: add the 2-D, 3-D, and 4-D adj_address overloads for indexed arrays, mirroring the forward index() functions (negative view coordinates wrap per dimension before each dimension remaps through its optional index array). Both adjoint parameter types are covered, since the CUDA codegen declares indexedarray adjoints as plain array_t while the CPU codegen reuses the forward parameter type. Without these overloads the generated backward kernels matched the intentionally empty generic adj_address and silently produced zero gradients, which the previous commit guarded with NotImplementedError. Lift those guards in indexedarray.grad and pack_arg and replace the guard regression test with 2-D, 3-D, and 4-D gradient tests, including the manual adjoint launch with a plain base gradient and mixed indexed/passthrough dimensions.
The new multi-dimensional adj_address overloads reuse the same indexedarray_t/array_t CPU-CUDA codegen split explained above the 1-D overloads, but nothing pointed a reader there. Add a one-line back-reference instead of repeating the explanation once per dimension, matching how the plain forward index() overloads document the pattern once and let the 2-D/3-D/4-D repeats stand on their own. Signed-off-by: Anthony Angeles <aangeles@nvidia.com>
The 1-D indexedarray gradient fix had a P1 review finding because tape-only tests missed the manual adjoint launch path (adjoint=True, adj_inputs=[...]), which exercises a different adj_address overload than wp.Tape does. The 2-D test already covered both paths; extend the same coverage to 3-D and 4-D so a regression in either overload variant cannot slip through tape-only testing. Signed-off-by: Anthony Angeles <aangeles@nvidia.com>
Unlike the 1-D case, each new 2-D/3-D/4-D adj_address overload hand-codes its own per-dimension negative-coordinate wrap-around (if (i < 0) i += buf.shape[0]; if (j < 0) ...), which is new, copy-paste-prone code that no existing test, forward or backward, exercised. Add a 2-D case verifying a negative view coordinate on the gathered dimension wraps before the index remap, matching the existing 1-D negative-index test. Signed-off-by: Anthony Angeles <aangeles@nvidia.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Adds gradient support for 2-D/3-D/4-D
wp.indexedarrayinputs, completing multi-dimensional support for NVIDIA#1479. Builds directly on NVIDIA#1504, which fixed 1-D gradients and left multi-dim as an explicitNotImplementedError.NVIDIA#1504 must land first. This PR targets that branch, not
main, and will be retargeted once NVIDIA#1504 merges.Changes
adj_addressoverloads inarray.h, mirroring the 1-D pattern: per-dimension negative-index wrap, per-dimension index remap, and the same CPU/CUDA adjoint-type split.NotImplementedErrorguards inindexedarray.gradandpack_argforndim > 1.Checklist
Validation summary
uv run warp/tests/test_indexedarray.py -k gradpasses on CPU and CUDA, including the new 2-D/3-D/4-D manual-adjoint-launch and negative-index tests. Also verified with a standalone script exercising forward, tape backward, and manual adjoint launch on both devices.New feature / enhancement