8000 Tags · googio/AutoGPT · GitHub
[go: up one dir, main page]

Skip to content

Tags

Tags

preview-seed-fixture

Toggle preview-seed-fixture's commit message

Partially verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
We cannot verify signatures from co-authors, and some of the co-authors attributed to this commit require their commits to be signed.
feat(platform): move onboarding paywall after personalization (Signif…

…icant-Gravitas#15196)

### Why / What / How

New users currently encounter the card-required trial/plan checkout
before they personalize their account. Move checkout after the role,
pain-point/brain-dump and optional expert-hiring steps so users finish
personalization before choosing a plan. The hosted wizard proceeds past
checkout only after confirming an active trial or paid plan.

Save progress asynchronously to the authenticated account and keep a
user-scoped local outbox for navigation/offline recovery. Resume uses
semantic steps instead of persisted numeric positions. Revision checks
reject stale-tab draft writes, and an expected account ID prevents draft
writes to a different signed-in account. Trial and paid checkout flush
pending progress before leaving; final completion also waits for a
successful profile save.

### Changes 🏗️

- Put the paywall after personalization/hiring while preserving
self-hosted flows, existing expert creation, and completion/credit
behavior. No new experiments or A/B tests.
- Add `wizardProgress` and `wizardRevision` to `UserOnboarding` with an
additive migration; extend the authenticated onboarding GET/PATCH API
and generated OpenAPI contract.
- Autosave step/answer changes in the background with a short debounce,
durable local outbox, retries, and explicit conflict/retry UI.
Navigation does not wait on saves; checkout and final completion do.
- Restore confirmed drafts without writing them again. Pending changes
and corrected resume steps still sync, and uncertain saves already
acknowledged by the server become clean locally.
- Wait for every layout flag before freezing the wizard's step order,
using the existing bounded fallback and a visible loading status.
- Share pending profile saves with completion, show retryable failures,
and ignore requests from previous account sessions. Drafts missing a
required role resume at Role for correction.
- Bound new unpaid cloud personalization to 10 attempts per UTC hour and
30 per UTC day, shared across voice, typed and profile processing.
Atomic Redis counters run before costly processing; ordinary draft
saves/status checks, paid users and self-hosted deployments do not
consume the budget. Existing duplicate checks reuse available results.
- Return retryable errors on limits or unavailable protection without
discarding recording data. This is an account-level cost guard; broader
security hardening remains separate.

Deploy the database migration before the updated API and frontend. No
environment variables or Docker configuration changes are needed.

### Agents and large language models used

OpenAI Codex with GPT-6, including three collaborating agents.

### Checklist 📋

#### For code changes:

- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [ ] I have tested my changes according to the test plan:
- [x] Backend focused tests: 151 passed covering durable drafts,
validation, account ownership/CAS, HTTP contract, quota
boundaries/outages, duplicate admission, recording recovery, profile
handling and existing brain-dump behavior. Backend code is unchanged in
the review fixes.
- [x] Frontend format, lint and types. All 130 onboarding tests pass
with coverage; changed hooks have 92–100% line coverage.
- [x] Full frontend CI suite with coverage on `692b2d3`: **9,120 tests
across 846 files passed**, with zero failed/skipped tests across all
four shards.
[Workflow](https://github.com/Significant-Gravitas/AutoGPT/actions/runs/37531571488).
The local broad run did not complete; these full-suite results are from
CI.
- [x] Regression tests reproduced delayed-flag,
failed/pending-profile-save, empty-role recovery, and
redundant-restore-save failures before the fixes; targeted suites pass
afterward.
- [x] Repository pre-commit checks on the review fixes, including secret
scanning, formatting and TypeScript.
- [ ] Staging smoke test with a real test-mode Stripe checkout,
reload/new-session resume, cancellation and delayed webhook
confirmation. Live Stripe, Redis and Postgres integration was not
exercised locally.

#### For configuration changes:

- [x] `.env.default` is already compatible with these changes
- [x] `docker-compose.yml` is already compatible with these changes
- [x] Configuration/migration changes are listed above

---------

Co-authored-by: GPT-6 (Codex) <agent@example.invalid>
Co-authored-by: Nicholas Tindle <nick@ntindle.com>
Co-authored-by: Claude Opus 5.5 (Claude Code) <noreply@anthropic.com>

autogpt-platform-beta-v0.8.3

Toggle autogpt-platform-beta-v0.8.3's commit message

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
feat(platform): signup consent line, marketing opt-out, and MailerLit…

…e gate (Significant-Gravitas#15152)

### Why / What / How

**Why:** [SECRT-2783](https://linear.app/autogpt/issue/SECRT-2783).
Signing up with Google skipped the mandatory terms checkbox, so two
thirds of new accounts have no recorded agreement. Nothing stored when
anyone agreed, or to which version. The form also gave people no way to
refuse marketing email, which the UK soft opt-in needs before we run the
win-back and MailerLite lifecycle emails.

**What:** The signup page now matches the final mockup on the ticket:
- **Layout:** "Already a member? Log in" sits under the title. The
checkbox is gone. One 14px line sits under the buttons: *"By continuing
you agree to our Terms of Use and Privacy Policy. We may email you
product updates and offers; opt out."*
- **Opt-out:** "opt out" swaps the second sentence in place to *"You
won't get marketing emails. Undo."*
- **Recording:** both signup paths (email and Google) write
`termsAcceptedAt`, `termsVersion` and, if the person opted out,
`marketingOptOutAt` / `marketingOptOutSource = signup` on the new user.
A returning account that opts out on `/signup` before continuing with
Google has the refusal recorded too.
- **MailerLite:**
- An opted-out account is never written there, by any live path or
backfill. The audience consumer re-checks the refusal right before each
write.
  - A new refusal unsubscribes an existing MailerLite subscriber.
  - An unsubscribe in MailerLite is recorded back as an opt-out.

| Mockup (from the ticket) | Built (`/signup`, cloud) | Built, after
"opt out" |
|---|---|---|
| <img
src="https://raw.githubusercontent.com/Significant-Gravitas/AutoGPT/f34cdc90b9b7e3e0080943e66b781e0c81ba5058/mockup.png"
width="420"> | <img
src="https://raw.githubusercontent.com/Significant-Gravitas/AutoGPT/f34cdc90b9b7e3e0080943e66b781e0c81ba5058/signup-new.png"
width="300"> | <img
src="https://raw.githubusercontent.com/Significant-Gravitas/AutoGPT/f34cdc90b9b7e3e0080943e66b781e0c81ba5058/signup-opted-out.png"
width="300"> |

The built screenshots come from the real page in Chromium against `next
dev`. The "Watch the demo" pill was
10BC8
 already removed by Significant-Gravitas#15095.

**How:**
- **Frontend**
- `SignupLegalLine` uses the `Text` atom at `body` (14px; `small` is
12px) and the `Link` atom for the policy links, which open in a new tab.
- "opt out" / "Undo" is one real `<button type="button">` styled like
the links. Focus stays on it after a toggle, and the swapped sentence is
in an `aria-live` region.
- The choice is a `marketingOptOut` form value. It is locked once a
signup is in flight, so the page cannot confirm a refusal that was never
sent.
- Opting out (not undoing) captures `marketing_opted_out` with `surface:
"signup"` through `capturePostHogEvent`, so it is held until the cookie
banner is answered rather than dropped.
- **Recording consent**
- **Email path:** the server action calls the new `POST
/auth/user/consent` right after `markAccountCreated("email")`. The page
clears the opt-out cookie first, so neither this choice nor one left by
an abandoned Google attempt reaches a later Google sign-in.
- **With `AUTH_REQUIRE_EMAIL_VERIFICATION` on (Significant-Gravitas#15135):** sign-up
returns no session, so nothing is recorded then. The refusal travels in
the verification link itself
(`/auth/callback?method=email&marketing_opt_out=1`, also on a link
resent from the check-your-inbox screen), so it holds whichever browser
opens the link and however late. The callback records the terms and the
refusal on the account the link provisions. It ignores the parameter for
an existing account or a non-email sign-in, so a crafted link can't
change anyone's preferences. No cookie is set for email sign-ups. If a
mail scanner opens the link first, the account is set up at the owner's
next sign-in instead (`login/actions.ts`), which records the terms. A
refusal can't be recovered on that path, nor on a fresh link requested
later from `/login`; storing it on our side at sign-up is
[SECRT-2851](https://linear.app/autogpt/issue/SECRT-2851).
  - **Google path:**
- `handleProviderSignup` sets a 10-minute `agpt_marketing_opt_out=1`
cookie before leaving for Google, or clears a stale one.
- `/auth/callback` reads and clears that cookie once provisioning
succeeds.
- It records consent when the account was just created, and also for a
returning account when the cookie carries a refusal, since the page has
already confirmed it.
- A consent-write failure goes to Sentry with the user id and the
opt-out choice, so a lost refusal can be re-applied by hand. It never
rolls back the signup.
- **Backend**
  - `record_signup_consent` is idempotent.
    - Terms are stamped again only when the version changes.
- The opt-out write is conditional: the first refusal wins, including a
concurrent one.
    - `marketing_opt_out=false` never clears a refusal.
    - A row deleted mid-write is a 404, not a database error.
- `terms_version` must be a version the service recognizes (`YYYY-MM` or
`YYYY-MM-DD`, ASCII digits).
- **MailerLite**
- Every MailerLite write upserts the subscriber, a group removal or a
field update included. So `notifications/consent.py` gates every
audience change, not just the adds:
    - the live checkout opener;
    - the checkout-completed webhook;
    - the first-subscription tour and the returning-customer changelog;
    - the subscription field updates;
    - churn;
    - the trial group;
    - the trial conversion.
- The audience consumer re-reads the refusal from the database before
applying each queued change, through a new `DatabaseManager` RPC,
`is_marketing_opted_out`. It drops a change for an account that refused
after the change was queued.
    - A deleted account counts as opted out.
    - A failed read is retried rather than written through.
- A newly recorded refusal queues an `UNSUBSCRIBE` change, exempt from
that re-check because it carries the refusal. It marks an existing,
still-mailable subscriber unsubscribed with `PUT /subscribers/{id}` and
never creates one.
- `POST /api/email/mailerlite/webhook` takes MailerLite's
`subscriber.unsubscribed` event and sets `marketingOptOutAt` with source
`email_unsubscribe`.
- Calls are authenticated by the `Signature` header (HMAC-SHA256 of the
body under `MAILERLITE_WEBHOOK_SECRET`).
    - Without a secret, every call is refused with 503.
- Billing and trial emails, their dedupe claims and the welcome claim
behave exactly as before for opted-out users.
- The checkout, group and field backfills drop opted-out accounts in
their `plan` step and report how many they skipped.
  - Skips are logged at debug level by pseudonym, never by address.

### Changes 🏗️

- **Database:** `User.termsAcceptedAt`, `termsVersion`,
`marketingOptOutAt`, `marketingOptOutSource` (all nullable), with
migration `20261007120000_user_consent_fields`. Adding the columns is a
catalog-only change with no table rewrite. The same fields are on the
Pydantic `User` and on `BillingEmailRecipient`.
- **API:**
- `POST /api/auth/user/consent` (`requires_user`; user id from auth
only), backed by `record_signup_consent` in `data/user.py`.
  - `POST /api/email/mailerlite/webhook` (signature-authenticated).
  - The regenerated `openapi.json`.
- **MailerLite:**
- `notifications/consent.py` (`marketing_allowed`,
`audience_change_allowed`).
- Gated in `data/checkout_audience.py`, `notifications/lifecycle.py` and
`notifications/trial_audience.py`. `trial_audience` now takes the user;
`trial.py` is its only caller.
- The consumer re-check in `notifications/notifications.py` and
`DatabaseManager.is_marketing_opted_out`.
- The `UNSUBSCRIBE` action in `notifications/mailerlite.py`, and the
webhook in `api/features/mailerlite/routes.py`.
- `checkout_backfill.py`, `mailerlite_backfill.py` (new `skip_opted_out`
decision) and `mailerlite_field_backfill.py` skip opted-out accounts,
and the `cli/mailerlite_backfill.py` reports show the count.
- **Signup page:** `page.tsx`, `useSignupPage.ts`, new
`components/SignupLegalLine.tsx`, and `types/auth.ts` (`agreeToTerms`
replaced by `marketingOptOut`).
- **Signup paths:** `signup/actions.ts`, `auth/callback/route.ts`,
`lib/auth/email-verification.ts` and `CheckYourInbox` (the opt-out in
the verification link), `login/actions.ts` (terms recorded when a login
sets the account up), new `lib/auth/server/recordSignupConsent.ts`,
`services/analytics/marketing-opt-out-cookie.ts` and
`marketing-opt-out-server.ts`.
- **Shared constants:** new `lib/legal.ts` (`TERMS_OF_USE_URL`,
`PRIVACY_POLICY_URL`, `TERMS_VERSION = "2026-10"`).
- **Analytics:** `marketing_opted_out` in `posthog-events.ts`, its pin
test, and `docs/engineering/tracking-plan.md`.
- **E2E helper:** `playwright/utils/signup.ts` no longer clicks the
removed checkbox.
- **Tests:**
  - Unit, route and real-database tests for consent recording.
  - An opted-out test for every gated path and backfill.
- Tests for the consumer re-check, the unsubscribe action and the
webhook.
  - Page, action, callback and cookie tests.
  - Details are in the test plan below.

**Notes for reviewers / open questions**

1. **Google signups from `/login`.** The Google button on `/login` can
also create an account. `/auth/callback` serves both pages, so such an
account also records terms acceptance, even though `/login` shows no
notice. This follows the spec ("record for every new Google account";
the login page is untouched). If we want a record only when the notice
was shown, the options are the same line on `/login`'s Google button, or
a signup-intent marker.
2. **"Visible in admin".** The fields are queryable columns on `User`.
The in-app admin users page is still an empty placeholder, so there is
no in-app view yet.
3. **Rolling deploy.** `get_user_by_id` is a shared Redis cache with a
5-minute TTL, and a `User` pickled by the previous release has no
`marketing_opt_out_at` attribute.
- The gate treats such a cached user as "not allowed" without raising.
For up to 5 minutes after deploy a few MailerLite writes are skipped
(the backfills catch them up), and no trial email is held up.
- It reads the field defensively, as `get_user_default_chat_route`
already does for the same reason.
- It does not default to "allowed", because an old pod re-caching a
just-opted-out user could then let them through. The consumer re-check
reads the database, not the cache.
4. **MailerLite webhook setup.** To turn the webhook on:
- Create a `subscriber.unsubscribed` webhook in MailerLite pointing at
`/api/email/mailerlite/webhook`.
   - Set its secret as `MAILERLITE_WEBHOOK_SECRET`.
   - Until then the endpoint refuses every call.
5. **Spacing.** "Sign up" keeps the login page's `mt-6` after the last
input (the `Input` atom reserves room for its error line), so the gap is
a little larger than in the mockup.
6. **User-ID checks (`data/*.py`).**
- `record_signup_consent` only ever writes the caller's own row, because
the route takes the id from `Security(get_user_id)` and never from the
body.
- The webhook matches the account by the email in the signed MailerLite
payload.
- `checkout_audience` and `BillingEmailRecipient` keep their existing
lookups (authenticated user id / Stripe customer id).

### Agents and large language models used

- Claude Code with Claude Opus 5.5

### Checklist 📋

#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
- [x] **E2E on a local stack** (`next dev` in cloud mode, REST API,
Postgres, Redis, RabbitMQ), with two real email signups:
- with "opt out": `termsAcceptedAt` set, `termsVersion = 2026-10`,
`marketingOptOutAt` set, `marketingOptOutSource = signup`;
    - without: the terms are stored and the opt-out stays null.
- `/signup?next=/library` keeps `?next=` on the subtitle's Log in link
and lands on `/library`.
- [x] **Live gate check:** `queue_checkout_opened` against the real
database and RabbitMQ.
    - The opted-out user queued nothing (debug skip by pseudonym).
    - The other user queued one `checkout_opened` change.
- [x] **Browser check:** the legal line is 14px; both policy links open
in a new tab; after a click, keyboard focus stays on the toggle (now
"Undo") with no lingering ring.
- [x] **Migration** applies cleanly on `pgvector/pgvector:pg15` after
the full history, and `prisma migrate diff` shows no drift on `User`.
`prisma validate` passes.
- [x] **Frontend:** `pnpm lint`, `pnpm types`, and the full `vitest`
suite in CI's 4 shards all pass. `generate:api` leaves no diff beyond
the new endpoints.
- [x] **Backend:** `ruff`, `isort`, `black` and full `pyright` pass, and
all four CI test shards pass. No new skip or xfail markers.
- [x] **New tests fail without the feature:** the page lock, callback
cookie ordering, returning-account refusal, Sentry context, stale-cache
gate, consent idempotency and race tests each fail when their fix is
reverted (checked by temporarily reverting each fix).
- [ ] Google signup end to end (needs real Google OAuth credentials):
opt out, Continue with Google, check the four fields; and the same
without opting out.
  - [ ] Against a real MailerLite account:
    - An opted-out account never appears after opening checkout.
    - A refusal unsubscribes an existing subscriber.
    - An unsubscribe in MailerLite sets `marketingOptOutAt`.

#### For configuration changes:

- [x] `.env.default` is updated or already compatible with my changes
- [x] `docker-compose.yml` is updated or already compatible with my
changes
- [x] I have included a list of my configuration changes in the PR
description (under **Changes**)

Configuration changes:
- `MAILERLITE_WEBHOOK_SECRET` is added, blank in `.env.default`. It is
MailerLite's webhook signing secret, and while it is blank the webhook
refuses every call.
- The additive, nullable-column migration listed under **Changes**.

<details>
  <summary>Examples of configuration changes</summary>

  - Changing ports
  - Adding new services that need to communicate with each other
  - Secrets or environment variable changes
  - New or infrastructure changes such as databases
</details>

🤖 Generated with [Claude Code](https://claude.com/claude-code)

https://claude.ai/code/session_012p8k42PpEJhnAWcnnskWr4

---
_Generated by [Claude
Code](https://claude.ai/code/session_012p8k42PpEJhnAWcnnskWr4)_


---
_Generated by [Claude Code](https://claude.ai/code)_

---------

Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Krzysztof Czerwinski <kpczerwinski@gmail.com>

autogpt-platform-beta-v0.8.2

Toggle autogpt-platform-beta-v0.8.2's commit message

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
feat(backend): backfill existing subscribers into MailerLite and add …

…a trial group (Significant-Gravitas#15026)

### Why / What / How

**Why.** The MailerLite code only reacts to new Stripe events, so
everyone who subscribed before it shipped is in no group, and there was
no way to sync them. Trials were never handled either:
`on_checkout_completed` returns early for a trial, and a conversion only
sent the notice, so a trial customer never joined any MailerLite group,
even after paying. Nick asked for both: sync the existing subscribers,
and add a trial group. GTM then asked for server-side status and date
fields on every MailerLite subscriber, including plain signups, which
have never reached MailerLite from this code and have not arrived there
at all since 2026-06-24.

**What.**
1. **A one-off backfill:** `poetry run cli mailerlite-backfill`. It is a
**dry run by default**, and it writes only with `--apply` plus a
confirmation prompt.
2. **A MailerLite trial group** that holds exactly the customers
currently on a trial:
   - they **join** when the trial starts, and again on an un-cancel;
- they **leave** on a cancel, a conversion, a failed conversion or the
trial ending.
- A conversion also joins the paying audience, as a first checkout does:
the onboarding tour for a first subscription, otherwise the changelog.
3. **MailerLite status and date fields**, written at signup and on every
trial and paid transition, and backfilled for every account. Details,
and the field keys for GTM, are below.

**How.**
- **Backfill.** It reads every Stripe subscription, and matches each
customer to their account by email, as the live handlers do. It then
reads the current membership of the tour, changelog and trial groups
(every subscriber status), and applies only the difference. Rules:
- paying → changelog, **unless they are in the tour group**. They are
either mid-tour or finished, and MailerLite's own automation owns the
tour → changelog handoff;
  - churned → out of the changelog;
  - trialing and not set to cancel → trial group; anyone else leaves it;
- `incomplete`, `unpaid`, `paused`, and past due on a subscription that
began as a trial → left alone.
- **How it writes.** It sends the same calls the live handlers make, 50
per `POST /api/batch`. Upsert batches are paced 13s apart for
MailerLite's 5-per-minute import limit, and a 429 is retried with
backoff by `Requests`. Output is one pseudonymised line per customer
(`_pseudonym`, never an email) plus counts. It is idempotent: a second
run reads and writes nothing.
- **Trial group in the live code.** Every trial transition already
produces exactly one trial notice with its own dedupe claim, so the
group change rides on it (`trial_audience.py`). The change is queued
*before* the notice, inside the same release-on-failure block, so a
failed publish lets Stripe's retry make it. Both changes are idempotent.
Conversion takes the welcome claim, because the conversion notice is the
welcome, so a later resubscription is treated as a returning customer. A
MailerLite change never costs the customer their notice. If the audience
event refuses an address (for example a reserved domain), the change is
skipped. Joining the paying audience after a conversion reports failures
instead of raising. New `AudienceAction.ADD_TRIAL` and `REMOVE_TRIAL`
follow the existing pattern and routing (`notification.audience.#`).
- **Ownership.** The backend owns both edges of the trial group, and
this is written into `mailerlite.py`'s one-owner-per-transition
docstring. Any MailerLite automation on the trial group may send mail,
but must never move people.

**Considered and not used:** MailerLite's native Stripe integration only
covers products sold through MailerLite's own checkout blocks
(Comfort/Power plans). The alternatives are Zapier, Make or MailerBuddy.
They are event-driven third parties that would write the same edges
without our rules, which is exactly the double-owner conflict the module
avoids.

#### MailerLite status and date fields, and signup sync

GTM asked for server-side MailerLite fields that say where each person
stands with us, plus the real signup date. Plain signups never reached
MailerLite, and nothing new has arrived there since 2026-06-24. This
adds:

- **Fields on every audience event.** `AudienceEventModel` now carries
`fields`, alongside a group change or on its own (the new
`update_fields` action). A group add writes the fields in the same
upsert. A removal writes the fields, then removes. Fields are written
even while a group ID is blank: a blank trial group means fields only,
and a blank tour or changelog group retries as it always has. Missing
fields are created in MailerLite before the first write (checked once
per process).
- **Signup sync.** When `_get_or_create_user` creates the platform
`User` row, it queues `signed` plus the signup date (the row's
`createdAt`) as a background task. Signup never waits on the broker and
never fails because of it. Reserved-domain addresses are skipped. It is
queued whatever the API server's own settings are (see below).
- **Lifecycle updates.** The trial and paid handlers set the status and
the date each transition is named for, from Stripe's own timestamp. A
status update never costs a billing email: on the paid path it is
reported, not raised.
- **Backfill for every account**, not just Stripe customers. It is
diffed against MailerLite's current fields, so a rerun resumes an
interrupted one.

#### Field keys for GTM

| Field key | Type | Set when | Value |
|---|---|---|---|
| `subscription_status` | text | every transition below | one of the six
statuses below |
| `signup_date` | date | account created | the account's creation date
(our DB). Unlike MailerLite's own "Subscribed" date, this is the real
signup. Never cleared |
| `trial_started_date` | date | trial starts | Stripe `trial_start`.
Never cleared |
| `subscription_started_date` | date | first checkout, trial converts,
resubscribe | Stripe `start_date`; for a converted trial, Stripe
`trial_end` (when the first paid period began) |
| `subscription_canceled_date` | date | a paid subscription is set to
cancel at period end | Stripe `canceled_at`. Cleared on resume or a new
subscription |
| `subscription_ended_date` | date | a paid subscription ends | Stripe
`ended_at`. Cleared on a new subscription |

Dates are UTC days (`YYYY-MM-DD`), the only format MailerLite date
fields accept. The dates describe the current or most recent
subscription.

| Event | `subscription_status` |
|---|---|
| Account created | `signed` |
| Trial started, or trial un-cancelled | `in_trial` |
| Trial cancelled, trial's first payment failed, trial ended unpaid |
`trial_canceled` |
| First checkout, trial converted, resubscribed, paid sub un-cancelled |
`subscribed` |
| Paid sub set to cancel at period end | `subscription_canceled` |
| Paid sub ended | `subscription_ended` |

A trial that ends without its first payment clearing is
`trial_canceled`, not `subscription_ended`. They never paid, so counting
them as churned customers would be wrong. It also keeps `in_trial`
exactly equal to the trial group's membership.

#### Where MailerLite is configured: the notification service only

- The API server runs signup and the Stripe webhook. It queues every
audience change without reading any `MAILERLITE_*` setting, so those
settings are needed only on the notification service, where infra
already puts them. The first version checked the token and the trial
group on the API side, which prod never sets there, so after deploy
nothing would have been queued. Found in review and fixed in 722b6de.
- The notification service decides what happens:
- **No token:** the change is acknowledged and dropped, with one info
log per process. It is not retried or dead-lettered, so self-hosted
stacks stay quiet.
  - **Blank trial group:** only the fields are written.
- **Blank tour or changelog group:** the change is retried, then
dead-lettered, as before.

#### Why nothing reached MailerLite after 2026-06-24 (as far as the code
can tell)

- No MailerLite code exists anywhere in `master` before Significant-Gravitas#14003
(2026-08-26), and that only moved group membership on Stripe events,
never signups. Whatever fed MailerLite until 2026-06-24 lived outside
this repo (for example a Supabase or Zapier integration), so it can't be
seen from here. The Supabase to Better Auth switch (Significant-Gravitas#13330, on `master`
2026-07-30) is a plausible cause, but it landed after 06-24.
- On the code in production today, blank settings do not fail silently
or retry forever. The job raises `MailerLiteNotConfigured`, is retried 3
times with backoff, then goes to the dead-letter queue
(`failed_notifications_v3`, key `failed.audience`) with a logged error.
Nothing replays the dead-letter queue. With this PR, a missing token no
longer does this: it is logged once, as above.
- Inference: production has run Significant-Gravitas#14003 since 2026-08-26. If
`MAILERLITE_*` were set there, every new paying customer since then
would have appeared as a new MailerLite subscriber, and GTM sees none
after 06-24. **The production MailerLite token and/or group IDs are most
likely blank.** Someone with production access should check the env,
Sentry for `MAILERLITE_API_TOKEN is not set`, and the dead-letter queue
for `failed.audience` messages. Until the token is set, this PR changes
nothing in production.

### Changes 🏗️

- **New env var `MAILERLITE_TRIAL_GROUP_ID`** (Config, blank by default,
added to `.env.default`). While it is blank, trial changes write only
the status and dates, and nobody is moved into or out of a trial group.
Conversion still joins the tour or the changelog.
- `backend/notifications/mailerlite_backfill.py` +
`backend/cli/mailerlite_backfill.py`: the backfill (`cli
mailerlite-backfill [--apply] [--yes]`).
- `backend/notifications/trial_audience.py`: trial group change per
notice kind, and joining the paying audience at conversion.
- `trial.py`: `notify_trial` queues the group change with the notice,
and joins the paying audience on conversion.
- `mailerlite.py`: `add_to_trial` and `remove_from_trial`. Removal is
shared with the changelog through `_remove_from_group`.
- `data/notifications.py` and `notifications.py`: the new actions and
their consumer routing.
- **New setting `MAILERLITE_API_URL`** (default
`https://connect.mailerlite.com/api`). Only a test stack changes it, to
point at a stub, so it is not in `.env.default`.
- `data/notifications.py`: `AudienceEventModel.fields`,
`AudienceAction.UPDATE_FIELDS`, and the `SubscriberField` /
`SubscriptionStatus` enums.
- `backend/notifications/subscriber_fields.py`: the status and date for
each transition, and the signup event.
- `data/user.py`: `_get_or_create_user` queues the signup fields in the
background for the row it just created. It only reads that new row's own
`id`, `email` and `createdAt`, so no user ID check is needed.
- `lifecycle.py` and `trial_audience.py`: each paid and trial transition
carries its status and date.
- `mailerlite.py`: field writes with every group change or alone, and
`ensure_fields` creates missing fields once per process.
- `notifications.py` and `mailerlite.configured()`: the consumer
acknowledges audience changes when there is no token.
`trial_audience.py` and `subscriber_fields.py` read no MailerLite
setting (722b6de).
- `backend/notifications/mailerlite_field_backfill.py` +
`backend/cli/mailerlite_backfill.py`: the field backfill for every
account (`cli mailerlite-backfill [--fields-only]`), and `cli
mailerlite-fields [--apply]`.

**Behaviour to review:** `payment_failed` (the trial is over but the
customer has not paid) also removes them from the trial group, because
they are no longer on a trial. They join the paying audience if the
conversion later succeeds.

> [!IMPORTANT]
> **New env var: `MAILERLITE_TRIAL_GROUP_ID`.** Create the trial group
in MailerLite and set this on the notification service in each
environment, next to the other `MAILERLITE_*` values. **While it is
blank, nobody joins or leaves a trial group.** Trial changes still write
the status and dates, and a conversion still joins the tour or the
changelog.

**Running the backfill (a human only, never CI or an agent):**
0. First set `MAILERLITE_API_TOKEN` (and the group IDs) on the
notification service. Changes queued before that are acknowledged and
dropped, and the backfill is what catches everyone up, so it has to run
after the token is in place.
1. Dry run first: `poetry run cli mailerlite-backfill`. It needs
`STRIPE_API_KEY`, `MAILERLITE_API_TOKEN`, the tour and changelog group
IDs, DB access, and optionally `MAILERLITE_TRIAL_GROUP_ID`. It writes
nothing, and prints one pseudonymised line per customer plus group
counts, then the field counts per status, how many accounts would be
written, and how many would be **new MailerLite subscribers**. GTM
should approve that last number first: every account becomes an active
subscriber, which counts against the MailerLite plan.
2. Review the counts. Then run `poetry run cli mailerlite-backfill
--apply`, which asks for confirmation before writing unless `--yes` is
given. A rerun is safe: it only writes what is still different, so it
also resumes an interrupted run.
3. Adding people to a group through the API fires any MailerLite
automation triggered by *joining* that group. Check the changelog and
trial groups' automations before `--apply`.

For the fields:
- `poetry run cli mailerlite-fields` lists the six fields and whether
MailerLite has them. `--apply` creates the missing ones. This is
optional: the notification service creates them before its first field
write.
- `--fields-only` writes the status and date fields and leaves group
membership alone. It is also the option for when the group IDs are not
configured, and then needs only `STRIPE_API_KEY`, `MAILERLITE_API_TOKEN`
and database access.
- Pacing is 50 upserts per `/batch`, spaced for MailerLite's import
limit: about 230 accounts a minute. An interrupted run is resumed by
running it again, because only fields that differ from MailerLite are
written.

**Found while here, not fixed (related to Significant-Gravitas#14992):**
`AudienceEventModel.email` is an `EmailStr`, which refuses reserved
domains such as `.test` and `.local`. In
`lifecycle.on_checkout_completed`, the tour event is built *after* the
welcome has been published and claimed. For such a user, the webhook
raises, and every Stripe retry takes the returning-customer branch and
raises again. This PR guards its own trial paths against that (skip,
never fail the notice), but leaves the existing welcome path alone.

### Evidence (tester VM, stub MailerLite + stub Stripe, synthetic
`@example.com` customers)

BEFORE is `dev` at 2f09a06 and AFTER is ad0a3e7. Each was deployed
with `git checkout <sha>` + `deploy-dev --current`, with HEAD recorded.
Both services were local HTTP stubs that record every request; no real
Stripe or MailerLite account was used.

**Backfill, 132 customers:**
- BEFORE: `No such command 'mailerlite-backfill'`, and **0** MailerLite
calls for 133 existing subscriptions.
- AFTER, dry run: 131 decisions checked against the expected ones, **0
mismatches**, 15 reads and no writes.
- AFTER, `--apply`: 7 batch calls (123 changelog adds, 1 trial add, 1
changelog removal, 1 trial removal). The first batch was answered with a
429 and succeeded on retry. The mid-tour customer and a newsletter-only
subscriber were left alone.
- AFTER, second `--apply`: "Nothing to write."

**Live trial flow.** This runs the real `lifecycle.py`, `trial.py`,
audience consumer and `mailerlite.py`. The DB lookups, Redis claims,
email queue and RabbitMQ hop are stubbed at the boundary.

| step | before (dev) | after |
|---|---|---|
| first paid checkout | tour | tour (unchanged) |
| trial started | no call | + trial group |
| cancelled during trial | no call | − trial group |
| un-cancelled | no call | + trial group |
| converted | no call | − trial group, + tour |
| trial ended unpaid | no call | − trial group |
| paid subscription ended | − changelog | − changelog (unchanged) |

**Tests.**
- New tests on `dev` fail: the modules are missing.
- With only the `trial.py` hook reverted: 9 failed, 9 passed.
- On ad0a3e7: `backend/notifications/` + `backend/cli/` **316
passed**. pyright, ruff, black and isort are clean on the changed files.
- Robustness fix (f36addc): its 3 tests fail on the test-only commit
9fd4037. With the fix, `backend/notifications/` **296 passed**, and
ruff, isort and pyright are clean. `backend/cli/` was not re-run for
this commit because it needs the live DB, and the fix does not touch the
CLI.

#### Status and date fields (`db27885e6f`, same stubs)

- **Before (dev `2f09a06bad`):** a real Better Auth signup publishes 0
audience messages. The whole trial-to-paid lifecycle publishes one
(`remove_changelog`, no fields), and the stub receives 0 MailerLite
calls.
- **After (`db27885e6f`):** each step gave the expected status and date:

  | Step | Status | Date set |
  |---|---|---|
  | Signup | `signed` | `signup_date` 2026-09-29 |
  | Trial start | `in_trial` | `trial_started_date` 2026-09-22 |
  | Trial cancel | `trial_canceled` | |
  | Trial un-cancel | `in_trial` | |
  | Convert | `subscribed` | `subscription_started_date` 2026-09-25 |
| Paid cancel | `subscription_canceled` | `subscription_canceled_date`
2026-09-27 |
| Paid un-cancel | `subscribed` | `subscription_canceled_date` cleared |
  | End | `subscription_ended` | `subscription_ended_date` 2026-10-25 |
  | Trial ending unpaid | `trial_canceled` | |

- **Fields:** missing before the first write, created by it.
- **Backfill:** over 143 accounts, with Stripe subscriptions,
pre-existing MailerLite subscribers and invalid emails, every account
got the right values. Unsubscribed people stayed unsubscribed. A second
run found nothing to write. An `--apply` stopped after its first batch
of 50 out of 138 left 88 to write, and the rerun wrote exactly those 88.
- **Unit tests:** fail against the base code (21 failed, 7 errors) and
pass on the branch (163 passed across `backend/notifications` and the
user tests).

Not verified against real MailerLite:
- that creating a field named `subscription_status` gives it that key
(the code fails loudly if it doesn't);
- that `null` clears a date field;
- the 1000-per-page subscriber listing.

#### MailerLite settings only on the notification service (`722b6de996`)

This is prod's shape. The API processes (`backend.app` without its
NotificationManager) and the webhook handlers ran with `MAILERLITE_*`
blank; the process environments were captured. The notification service
ran alone, with the settings pointing at the stub.

- **Before (`db27885e6f`):**
- A real signup published 0 audience messages, and the stub stayed
empty.
  - Trial start, cancel and un-cancel published nothing.
- Conversion and the end published only `enroll_tour` /
`remove_changelog`, with empty fields.
  - The subscriber ended up with no status or dates.
- **After:** 1 message at signup and 8 across the lifecycle, each with
its fields. The stub ended with every status and date in the table
above, and the trial group moved in and out.
- **No token anywhere (self-hosted):** two signups were queued and
acknowledged, the dead-letter queue stayed at its count, one info line
was logged, and no MailerLite calls were made.
- **Unit tests:** the new ones fail on `db27885e6f` (33 failed, 3
errors) and pass with the fix. `backend/notifications` + `backend/cli`:
**383 passed**. ruff is clean.

### Agents and large language models used

- Claude Code with Claude Opus 5.5 (1M context)

### Checklist 📋

#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
- [x] Unit tests: backfill classification, decisions, batching and
pacing, failure accounting, group reads and idempotency; each trial
transition's group change, conversion to the tour or changelog, trial
group unset, a replayed event, and a failed queue releasing the claim;
the consumer routing; the MailerLite trial calls
- [x] VM: backfill dry run, `--apply` and a second run against stubs,
BEFORE and AFTER
- [x] VM: live trial flow start → cancel → un-cancel → convert, and
start → end unpaid, BEFORE and AFTER
- [x] Unit tests: status and dates per transition, signup sync, field
writes with and without a group change, field creation, the field
backfill diff and resume
- [x] VM: signup, every trial and paid transition, and the field
backfill (dry run, interrupted `--apply`, rerun), BEFORE and AFTER
- [x] VM: MailerLite settings only on the notification service (prod's
shape), BEFORE and AFTER, plus no token anywhere

#### For configuration changes:

- [x] `.env.default` is updated or already compatible with my changes
- [x] `docker-compose.yml` is updated or already compatible with my
changes
- [x] I have included a list of my configuration changes in the PR
description (under **Changes**)

---------

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

autogpt-platform-beta-v0.8.1

Toggle autogpt-platform-beta-v0.8.1's commit message
Merge remote-tracking branch 'origin/master' into dev

# Conflicts:
#	autogpt_platform/frontend/instrumentation-client.ts

autogpt-platform-beta-v0.8.0

Toggle autogpt-platform-beta-v0.8.0's commit message

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
fix(backend/copilot): apply the building-mode guide on restart instea…

…d of re-deriving it from history (Significant-Gravitas#14721)

### Why

AutoPilot refuses to save an agent it has just designed.
`enter_agent_building_mode` must load the agent-building guide before
`create_agent` is allowed; on the SDK engine the guide goes into the
system prompt, which can only be changed by relaunching the turn. That
relaunch applied an **empty** guide and then told the model "Building
mode is now active — the complete agent-building guide is in your system
prompt", so the gate could never clear, and the user was told the
platform is broken.

Dev logged it 16 times in six hours across 6 of 11 chat sessions
(2026-09-18 20:00Z → 09-19 02:10Z), every one at ERROR: 9 of 9 restarts
on the pre-Significant-Gravitas#14714 image (20:09–20:17Z), 7 of 12 after the 00:43Z
rollout. Session `c91efb40-559b-45fa-8390-388fa6e516a4` shows it three
times inside one turn — 01:59:05.917Z, 01:59:19.811Z and 02:00:27.360Z,
each `Building mode requested — interrupting for prompt upgrade`
followed ~100 ms later by `Building-mode restart: guide suffix empty —
continuing without prompt upgrade`.

This predates Significant-Gravitas#14714 (merged 00:38Z 09-19), which touches 16 files and
not `builder_context.py`; its rollout took the failure rate from 100% to
58%.

### What

`build_builder_system_prompt_suffix` takes `force`, and the restart
passes it, so the guide is applied from the fact that the enter tool
just ran rather than from a history scan that cannot see it yet.

When the suffix is still empty — which now means only that the guide
failed to load — the relaunch no longer claims the guide is present. It
says the guide could not be loaded, leaves `building_mode_requested` set
so the next turn retries, and leaves `guide_in_system_prompt` False so
the building-mode gates stay closed, which is correct: the guide really
is absent. The ERROR line carries the full session id; the log prefix
truncates it to 11 characters.

### How

`_apply_building_mode_restart` called
`build_builder_system_prompt_suffix(session)`, whose first branch
returns `""` unless `session_entered_building_mode(session)` — a
predicate derived from persisted message history and documented for "a
*prior* turn". The restart calls it microseconds after the enter tool
ran, before that tool call is in `session.messages`. `force=True` skips
that branch for the one caller that already knows the answer; every
other calle
10BC8
r is a turn-start assembly, where the history read is the
right question.

The failure path leaves `building_mode_requested` set, which would
otherwise make `_ready_for_building_mode_restart` fire again at every
message boundary for the rest of the turn, so the guard also reads a new
turn-scoped `_RetryState.building_mode_restart_failed`. The relaunch
itself still happens: the attempt has already been interrupted, so
skipping it would end the turn mid-work.

### Open question

Why the post-Significant-Gravitas#14714 rate is 58% rather than 0% or 100% is not
established. Five restarts on the same image did build the suffix, and
`BaseTool.execute` announces every dispatched tool into the in-flight
buffer `session_entered_building_mode` reads, so the predicate should
have answered True in all twelve. `force` removes the dependency on it
either way, but what separates the two groups is unexplained and not
guessed at here.

### Verified

Executed: `copilot/sdk/building_mode_restart_test.py` and
`copilot/builder_context_test.py` (33 passed);
`copilot/tools/helpers_test.py`, `copilot/capabilities/dispatch_test.py`
and `util/architecture_test.py` (90 passed, 1 deselected —
`test_prepare_block_missing_credentials` hangs on clean dev on this
machine); `blocks/test/test_block.py`; `ruff check` on the four touched
files.

Both new tests are mutation-proven. Dropping `force=True` turns
`test_guide_applied_although_history_lacks_the_enter_call` red (1 failed
/ 12 passed); restoring the unconditional confirmation turns
`test_empty_suffix_relaunches_without_the_confirmation` red (1 failed /
12 passed). The first runs the real suffix builder rather than a mock on
purpose — patching it would have proved the wiring and never that the
predicate underneath answers.

Reasoned about, not executed: the restart against a live SDK turn on a
deployed environment.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>

autogpt-platform-beta-v0.7.4

Toggle autogpt-platform-beta-v0.7.4's commit message

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
fix(frontend/copilot): make completing a Connect card actually connect (

Significant-Gravitas#14316)

### Why / What / How

**Why:** This PR makes completing a Connect card in the copilot actually
connect the integration and send the follow-up turn.

Three separate defects — dropped OAuth scopes, a credentials provider
that never learns a new credential exists in a connectors-only chain,
and a connectors-only chain card with no completion path — combined so
that clicking Connect, signing in, and seeing the "Connected via OAuth"
toast left the row still reading "Connect" with no follow-up turn sent.
Reported by Reinier in #breakage on 2026-09-03 with a screen recording;
Toran noted it was the third bug of this shape in about two weeks.

**What:** Fixes three defects on the connect path, and adds one test
that drives the whole path end to end so the next drift is caught at the
seam rather than in production.

**How:** Any one of the three alone produces "nothing happens", which is
why this reads as unfixable from the outside:

1. **The required scopes are dropped.** `ConnectorRow` →
`ConnectCredentialDialog` → `useConnectCredentialDialog` →
`useOAuthConnect` → `getV1InitiateOauthFlow(provider)` — no scopes. The
endpoint accepts them (`router.py:135`) and the path this replaced
passed them (`useCredentialsInput.ts:259`). The grant comes back without
`repo`, `hasRequiredScopes` rejects the credential, and the row can
never be satisfied — every retry drops them again. Now threaded from the
card's schema down to the initiate call.
2. **Nothing tells the credentials provider a credential exists.**
`invalidateConnectionQueries` invalidates a React Query key, and
`CredentialsProvider` reloads off that query's cache events — but in a
connectors-only chain nothing has that query mounted, so invalidation
emits no event and the list is never re-read. `useOAuthConnect` now
calls the provider's existing `reload()`, the same way
`AyrshareConnectButton` already does.
3. **A connectors-only chain card has no completion path.**
`ChainActionCard` renders Proceed only for inputs or questions, and
`SetupRequirementsCard`'s auto-send gates on a store whose
`markConnected` runs only from `markSent` — after a card has already
sent. Such a card now sends once its credential is satisfied, reusing
the existing `tryClaimAutoDismiss` claim so parallel cards cannot
double-send.

Introduced by [**feat(frontend): add copilot tool chain UI** -
Significant-Gravitas#13773](Significant-Gravitas#13773)
(`a6df3d3381`) behind the `new-tool-ui` flag, which defaulted off;
shipped to everyone by [**refactor(frontend): drop the new-tool-ui flag
and commit to the tool chain UX** -
Significant-Gravitas#14164](Significant-Gravitas#14164)
(`948b55ea81`), which dropped the flag. The standalone (non-chain) card
still uses the scoped path, which is why this reads as "anymore".

### Changes 🏗️

- `useOAuthConnect`: accepts optional `scopes`, passes them to the
initiate call, and reloads the credentials provider after the token
exchange. Every other caller is unaffected — the argument is optional
and a provider-level connect still asks for defaults.
- `ConnectCredentialDialog` / `useConnectCredentialDialog`: forward
`schema.credentials_scopes`. This also fixes the builder's "Add
credential" flow, which had the same gap.
- `SetupRequirementsCard`: a connectors-only card inside a chain
auto-sends once its credential is complete.
- New `ToolChain/__tests__/connect-card-flow.test.tsx`: three cases
driving ToolChain → SetupRequirementsCard → ChainActionCard →
ConnectorRow → ConnectCredentialDialog → useOAuthConnect against MSW.

### Verified

I executed this locally against the real backend and GitHub client —
confirming the authorize URL now carries the correct scope — and through
the test suite, where each of the three fixes is pinned by a case that
fails on the pre-fix code and passes after, with the surrounding
copilot/credentials/onboarding regression suites staying green. The
OAuth consent screen itself was reasoned about rather than executed,
since completing it needs a human at GitHub's permission page; full
command output, the live URL comparison, and the regression sweep are in
the [evidence comment
below](Significant-Gravitas#14316 (comment)).

### Why this keeps happening

All three incidents break the *composition* of the connect chain while
every individual link keeps passing its own tests, because every test on
this path mocks the layer beneath it: `ToolChain.test.tsx` mocks
`SetupRequirementsCard`, `ChainActionCard.test.tsx` mocks
`ConnectCredentialDialog`, `ConnectCredentialDialog.test.tsx` stubs
`useOAuthConnect`. The scope contract *was* pinned by a test —
`CredentialsInput.test.tsx:217` — but on the path
[Significant-Gravitas#13773](Significant-Gravitas#13773)
replaced; the replacement inherited coverage of its own layer and none
of the contract it took over.

The 08-24 incident ([**fix(frontend): MCP OAuth popup fails with
'Sign-in window was closed' on COOP providers** -
Significant-Gravitas#14141](Significant-Gravitas#14141),
`popup.closed` lying under COOP) does **not** share that root cause: it
is a browser-platform behaviour no Node test can reproduce, and it needs
a Playwright case instead. That distinction, and the remaining gaps this
PR does not close, are tracked in [**The copilot's connect chain has no
end-to-end test, so each layer's contract drifts independently** -
Significant-Gravitas#14315](Significant-Gravitas#14315)
(OPEN-3471).

### Checklist 📋

#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
- [x] The chain's Connect button initiates OAuth carrying the block's
required scopes
- [x] A connected card flips to "Connected. Continuing…" and sends the
follow-up turn
- [x] A downgraded grant still reads as not-connected and sends nothing
  - [x] Every case fails on the pre-fix code for the right reason
  - [x] Surrounding copilot / credentials / onboarding suites stay green
  - [x] A finished card re-mounting from chat history sends nothing
  - [x] An API-key connect continues the chat too
  - [x] The sign-in offers the existing account for scope upgrade
- [x] Live: the authorize URL carries `scope=repo` with the fix and
`scope=` without it
- [x] Manual run against a real GitHub consent screen — not done, see
**Verified**

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>

autogpt-platform-beta-v0.7.3

Toggle autogpt-platform-beta-v0.7.3's commit message
Merge remote-tracking branch 'origin/dev'

autogpt-platform-beta-v0.7.2

Toggle autogpt-platform-beta-v0.7.2's commit message

Verified

This commit was signed with the committer’s verified signature.
ntindle Nicholas Tindle
Merge branch 'master' into dev

# Conflicts:
#	autogpt_platform/backend/backend/copilot/graphiti/ingest.py

autogpt-platform-beta-v0.7.1

Toggle autogpt-platform-beta-v0.7.1's commit message

Verified

This commit was signed with the committer’s verified signature.
ntindle Nicholas Tindle
Merge branch 'master' into dev

# Conflicts:
#	.github/workflows/platform-single-container-docker.yml

autogpt-platform-beta-v0.7.0

Toggle autogpt-platform-beta-v0.7.0's commit message

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
fix(backend/copilot): stop empty untitled dream sessions polluting th…

…e chat list (Significant-Gravitas#13332)

## Why

The dream pass (Significant-Gravitas#13243) creates a `ChatSession` per pass for its
user-facing narrative. Three compounding issues made this user-visible
noise: every nightly pass for a user with old facts but no new activity
ran all three LLM phases, produced empty operations with an empty
narrative, and still minted a session — which then leaked into the
normal chat sidebar as an untitled chat containing only "Dream pass
completed with no narrative output." Observed locally: one empty
untitled chat per enabled user per night.

## What

- Dream-kind sessions are hidden from the user chat list, pagination
count, and `/search/global` (they remain fetchable by id for the admin
Memory Visualizer). Surfacing dreams in the chat UI stays future work
(P6).
- `apply_operations` no longer creates a session or message when the
pass produced zero operations.
- Dream sessions that ARE created get a dated title ("Dream summary —
YYYY-MM-DD").
- The sanitize prompt's empty-result example no longer teaches the model
to return an empty `summary_for_user`; a new hard rule requires a
non-empty summary.
- Passes now skip before any LLM cost when the user has no episodes
newer than their last completed dream (`dream:last_completed:{user_id}`
Redis marker — single-key/cluster-safe, fail-open on Redis errors,
stamped on the sync success path).

## How

- The list filter is raw SQL with `metadata->>'kind' IS DISTINCT FROM
'dream'` — `IS DISTINCT FROM` is load-bearing: legacy sessions have no
`kind` key, `->>'kind'` evaluates to SQL `NULL`, and a naive `<>` would
hide every normal chat. The Python Prisma client's `JsonFilter` has no
path access, hence raw SQL via the existing `query_raw_with_schema`
helper; list/count share one WHERE fragment so pagination totals always
match. Pinned by mock tests on the WHERE construction plus real-Postgres
integration tests for the NULL semantics.
- The empty-pass guard returns stats without a `session_id` key (both
consumers read via `.get`, and the batch callback already narrows
non-str to `None`), so no signature changes ripple into
`batch_callbacks`.
- The no-new-activity skip compares episode `valid_at` (with
`created_at` fallback) against the marker; unparseable timestamps count
as new (fail-open). The batch path doesn't stamp the marker yet — batch
users simply never skip (fail-open, noted in code).

## Checklist

- [x] Changes covered by tests (69 passing: mock + real-Postgres
integration)
- [x] `poetry run format` / lint / pyright clean
- [x] Out-of-scope changes < 20%
- [x] `data/*.py` untouched; `copilot/db.py` queries are user-id scoped

🤖 Generated with [Claude Code](https://claude.com/claude-code)


Fixes SECRT-2480

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Krzysztof Czerwinski <kpczerwinski@gmail.com>
0