8000 feat(storage): implement Spanner DDL migrations for components and snapshots by VLukiantsev · Pull Request #484 · google/xmc · GitHub
[go: up one dir, main page]

Skip to content

feat(storage): implement Spanner DDL migrations for components and snapshots - #484

Merged
VLukiantsev merged 2 commits into
mainfrom
feat/components-spanner-migrations-b548436412
Aug 20, 2026
Merged

VLukiantsev merged 2 commits into
mainfrom
feat/components-spanner-migrations-b548436412

Conversation

@VLukiantsev
@VLukiantsev VLukiantsev commented Aug 19, 2026 •
Copy link
Copy Markdown
Collaborator

Summary

Implements Cloud Spanner schema migrations and updated baseline DDL for the XMC Components system catalog and experiment snapshots (Option C Hybrid SOT model).

Motivation & Context

As specified in the AI Components Design Doc and POC PR #456, XMC Components require persistent storage for registered custom components (1P/3P), dependency tracking, immutable per-experiment snapshot locking, and real-time experiment consistency evaluation.

Key Changes

  • Migration 010 (010_components_system.up.sql / 010_components_system.down.sql):
    • Components: Primary key ComponentId INT64. Includes ActiveName generated column (IF(State != 5, Name, NULL)) with unique null-filtered index Idx_Components_ActiveName to guarantee active name uniqueness while retaining soft-deleted records. Check constraint CK_Components_State validates state enums (1..5).
    • ComponentDependencies: Interleaved in parent Components on delete cascade, with foreign key reference FK_Dependency and secondary index Idx_ComponentDependencies_Dependency on DependencyComponentId to optimize foreign key scans on parent deletes.
    • ExperimentComponentSnapshots: Interleaved in parent Experiments on delete cascade, with secondary index Idx_Snapshots_Component and foreign key reference FK_Snapshots_Component.
    • ExperimentConsistencyInfo: Security invoker view evaluating drift between live component ObservedGeneration and experiment snapshots for active experiments using INNER JOINs.
  • Baseline Schema (ess-schema.sql): Appended table, index, and view definitions matching migration 010.

Verification & Testing

  • Commands Executed:
    • bazel test //internal/storage/spanner/migrations:migrations_test --nocache_test_results (Passed - verified TestSchemaSynchronization and TestMigrationUpDownConsistency against Spanner Emulator)
    • bazel test -- //... -//cmd/ui/... (All 110 backend test targets passed)
  • Pre-Commit Checks: Pre-commit hooks (Buildifier, gofmt, golangci-lint, addlicense, Ruff) executed and passed cleanly.
  • CI Status: 100% green across all GitHub Actions workflows on PR #484.

Related Issues / Bugs

Fixes bug 548436412

TAG=agy
CONV=946866f9-38d1-497a-ac2d-0eda5d3e6080

@gemini-code-assist gemini-code-assist Bot left a comment
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces Spanner schema changes and migrations to support a new components system, adding the Components, ComponentDependencies, and ExperimentComponentSnapshots tables, as well as the ExperimentConsistencyInfo view. The review feedback highlights critical performance and optimization improvements: first, a secondary index should be added to ComponentDependencies(DependencyComponentId) to avoid full table scans on parent table updates/deletes (and dropped in the down migration); second, the LEFT JOINs in the ExperimentConsistencyInfo view should be simplified to INNER JOINs since the foreign key constraints guarantee matching rows.

@VLukiantsev
VLukiantsev force-pushed the feat/components-spanner-migrations-b548436412 branch from 2e72d93 to 03e2e1c Compare August 19, 2026 21:52
@VLukiantsev
VLukiantsev changed the base branch from main to fix/ui-server-test-regression-and-rdeps August 19, 2026 21:53
@VLukiantsev
VLukiantsev marked this pull request as ready for review August 19, 2026 22:05
@github-actions
github-actions Bot requested a review from AntonTyb August 19, 2026 22:05
Base automatically changed from fix/ui-server-test-regression-and-rdeps to main August 20, 2026 06:56
…apshots

Implement Cloud Spanner schema migrations and baseline DDL for the XMC
Components system catalog and experiment snapshots (Option C Hybrid SOT).

Key Additions:
- Components table: Stores registered 1P/3P components, generation, and active state.
  Uses a stored generated column ActiveName and a unique null-filtered index
  Idx_Components_ActiveName to enforce active name uniqueness while retaining soft-deleted rows.
- ComponentDependencies table: Interleaved under Components with cascade deletion
  and foreign key constraint to track inter-component dependencies.
- ExperimentComponentSnapshots table: Interleaved under Experiments with cascade
  deletion to lock component configuration versions to specific experiment runs.
- ExperimentConsistencyInfo view: Evaluates runtime consistency between live
  component generations and experiment snapshots.
- Up and down migration scripts for clean schema upgrade and rollback.

Fixes bug 548436412

TAG=agy
CONV=946866f9-38d1-497a-ac2d-0eda5d3e6080
…y view join

Add secondary index Idx_ComponentDependencies_Dependency on
ComponentDependencies(DependencyComponentId) to optimize foreign key
scans on Components table updates and deletions. Update down migration
to drop this index.

Simplify ExperimentConsistencyInfo view joins from LEFT JOIN to INNER JOIN
since foreign key constraints enforce referential integrity.

Comment link: #484 (comment)
Comment link: #484 (comment)
Comment link: #484 (comment)
Comment link: #484 (comment)
Comment link: #484 (comment)

TAG=agy
CONV=946866f9-38d1-497a-ac2d-0eda5d3e6080
@VLukiantsev
VLukiantsev force-pushed the feat/components-spanner-migrations-b548436412 branch from 03e2e1c to a160165 Compare August 20, 2026 06:56
@VLukiantsev
VLukiantsev added this pull request to the merge queue Aug 20, 2026
Merged via the queue into main with commit b71187e Aug 20, 2026
15 checks passed
@VLukiantsev
VLukiantsev deleted the feat/components-spanner-migrations-b548436412 branch August 20, 2026 07:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

0