8000 Support gradients for multi-dimensional indexed arrays by aneangel · Pull Request #1 · aneangel/warp · GitHub
[go: up one dir, main page]

Skip to content

Support gradients for multi-dimensional indexed arrays - #1

Open
aneangel wants to merge 4 commits into
aangeles/fix-indexedarray-adjointfrom
aangeles/indexedarray-adjoint-multidim
Open

aneangel wants to merge 4 commits into
aangeles/fix-indexedarray-adjointfrom
aangeles/indexedarray-adjoint-multidim

Conversation

@aneangel
@aneangel aneangel commented Sep 4, 2026
Copy link
Copy Markdown
Owner

Description

Adds gradient support for 2-D/3-D/4-D wp.indexedarray inputs, completing multi-dimensional support for NVIDIA#1479. Builds directly on NVIDIA#1504, which fixed 1-D gradients and left multi-dim as an explicit NotImplementedError.

NVIDIA#1504 must land first. This PR targets that branch, not main, and will be retargeted once NVIDIA#1504 merges.

Changes

  • Add 2-D/3-D/4-D adj_address overloads in array.h, mirroring the 1-D pattern: per-dimension negative-index wrap, per-dimension index remap, and the same CPU/CUDA adjoint-type split.
  • Lift the NotImplementedError guards in indexedarray.grad and pack_arg for ndim > 1.
  • Add gradient tests for 2-D/3-D/4-D: tape backward, manual adjoint launch, and negative indices.

Checklist

  • I am familiar with the Contributing Guidelines.
  • New or existing tests cover these changes.
  • The documentation is up to date with these changes.
  • I added a changelog fragment if this change affects users.

Validation summary

uv run warp/tests/test_indexedarray.py -k grad passes on CPU and CUDA, including the new 2-D/3-D/4-D manual-adjoint-launch and negative-index tests. Also verified with a standalone script exercising forward, tape backward, and manual adjoint launch on both devices.

New feature / enhancement

import warp as wp
import numpy as np

@wp.kernel
def sum_2d(samples: wp.indexedarray2d(dtype=float), total: wp.array(dtype=float)):
    i, j = wp.tid()
    wp.atomic_add(total, 0, samples[i, j])

base = wp.array(np.arange(16, dtype=np.float32).reshape(4, 4), dtype=float, requires_grad=True)
rows = wp.array([1, 3], dtype=wp.int32)
samples = wp.indexedarray2d(base, [rows, None])
total = wp.zeros(1, dtype=float, requires_grad=True)

tape = wp.Tape()
with tape:
    wp.launch(sum_2d, dim=samples.shape, inputs=[samples], outputs=[total])
tape.backward(loss=total)
print(base.grad.numpy())  # without this PR: NotImplementedError

Follow-up to the 1-D indexedarray gradient support: add the 2-D,
3-D, and 4-D adj_address overloads for indexed arrays, mirroring
the forward index() functions (negative view coordinates wrap per
dimension before each dimension remaps through its optional index
array). Both adjoint parameter types are covered, since the CUDA
codegen declares indexedarray adjoints as plain array_t while the
CPU codegen reuses the forward parameter type.

Without these overloads the generated backward kernels matched the
intentionally empty generic adj_address and silently produced zero
gradients, which the previous commit guarded with
NotImplementedError. Lift those guards in indexedarray.grad and
pack_arg and replace the guard regression test with 2-D, 3-D, and
4-D gradient tests, including the manual adjoint launch with a
plain base gradient and mixed indexed/passthrough dimensions.
The new multi-dimensional adj_address overloads reuse the same
indexedarray_t/array_t CPU-CUDA codegen split explained above the 1-D
overloads, but nothing pointed a reader there. Add a one-line
back-reference instead of repeating the explanation once per
dimension, matching how the plain forward index() overloads document
the pattern once and let the 2-D/3-D/4-D repeats stand on their own.

Signed-off-by: Anthony Angeles <aangeles@nvidia.com>
The 1-D indexedarray gradient fix had a P1 review finding because
tape-only tests missed the manual adjoint launch path
(adjoint=True, adj_inputs=[...]), which exercises a different
adj_address overload than wp.Tape does. The 2-D test already covered
both paths; extend the same coverage to 3-D and 4-D so a regression
in either overload variant cannot slip through tape-only testing.

Signed-off-by: Anthony Angeles <aangeles@nvidia.com>
Unlike the 1-D case, each new 2-D/3-D/4-D adj_address overload
hand-codes its own per-dimension negative-coordinate wrap-around
(if (i < 0) i += buf.shape[0]; if (j < 0) ...), which is new,
copy-paste-prone code that no existing test, forward or backward,
exercised. Add a 2-D case verifying a negative view coordinate on
the gathered dimension wraps before the index remap, matching the
existing 1-D negative-index test.

Signed-off-by: Anthony Angeles <aangeles@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

0