Repository navigation
Conversation
bjoaquinc
force-pushed
the
agent-jobs-scaffold
branch
from
September 16, 2026 23:12
fdcb81c to
f6f8627
Compare
bjoaquinc
marked this pull request as ready for review
September 17, 2026 07:35
`make workspace-agent` scaffolds a workspace for the background agents hackathon: pinned to the dlt alpha that carries the agent launcher, pointed at api.dlthub.dev, and set up in three commands. Two tracks share one scaffold, so the fragile pin set lives in one place: verified_agent.py runs the workbench's job-inspector, my_agent.py is a template to edit. Dev-only and outside src/ on purpose. dlt is pinned to a branch archive, which must never ship in the published wheel. scripts/place_toolkit_agents.py works around `dlthub ai toolkit install` skipping a toolkit's agents: dlt looks for them at <toolkit>/agents/ while the workbench ships them at <toolkit>/dlthub/agents/, so the install reports success and the agent silently never arrives.
bjoaquinc
force-pushed
the
agent-jobs-scaffold
branch
from
September 17, 2026 10:44
f6f8627 to
02dd25b
Compare
Replaces the single broken_ingest job with dlt-hub/test-workspace's jaffle_shop pipelines: one correct and four broken, covering four distinct failure classes. Two fail outright and wake the agents; two run green while losing data, which no job.fail trigger can see. That gap is the point, and it gives both tracks something real to work on. alerts.py becomes Track B's starting point: it already fires on failure and records that something broke, but cannot say why. Adds deltalake and s3fs, which the playground destination needs to write Delta tables. Without them the pipelines extract fine and then fail at load. Drops the cron schedules, keeping tag:jaffle, so nothing fires unattended in a participant's workspace. Moves the mask helper and endpoint_report into diagnostics.py.
The template skipped `run.Entity`, which the dlt branch treats as how an agent
job reaches the rest of the platform. Without it the manifest carries no
`expose.object_input`, so the web UI cannot offer the agent from a failed run's
row, and the run reports no entity so it never appears on that run's page. A
template people copy should not teach the omission.
Verified on a local stack: the manifest now carries object_input for both agents,
and my_agent's result reports job-run/<id> where it previously reported nothing.
Also passes loop inputs by keyword. `AgentLoop.run`'s first positional argument
is `instructions`, so a dict lands on a string path; the old `run({})` only
survived because an empty dict is falsy.
my_agent.py carried 15 comment lines explaining dlt's API semantics, which
buried the three things a participant is meant to edit. The descriptions now sit
in Doc() on every input and output field, where the model reads them too.
The README dropped from 193 lines to 72. It documented gotchas we hit while
building the scaffold and then fixed in it: the pins, the fastmcp extra, the
pydantic-ai floor, the toolkit workaround, and the agent declarations. A
participant gets the fixed version, so the working code is the reference. Also
drops a validation log that belonged in the pull request and was already stale.
Passes run_context into the loop inputs. A decorated function drives the loop
itself, so {{ run_context.* }} placeholders rendered blank; the launcher does
this for an agent declared by reference.
A fresh deploy had four working agents, which made the scaffold a demo rather than a starting point. Per the hackathon flow: everyone declares the verified job-inspector first, then writes their own. verified_agent.py and my_agent.py are now stubs whose docstrings say what to write; example_agent.py is the worked Python agent, kept as a reference and not deployed. Both steps are the same motion: write a file, add it to __deployment__.py, deploy. A fresh deploy has eight jobs and no agents. Drops row_count_agent, which caught the two green pipelines. That is the most interesting thing a team could build, so it should not ship. my_agent.py names the gap and mentions that triggers also come in .success, without answering it.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode cha
683F
racters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds
make workspace-agent, a dev-only workspace for the background agentshackathon. Three commands from an empty checkout to a deployed agent workspace.
Two tracks, one scaffold
verified_agent.pydlthub-platform:job-inspector, nothing to authormy_agent.pyThe failure material
jaffle_shop/comes from dlt-hub/test-workspace (thanks @AstrakhantsevaAA) andruns against the live Jaffle Shop API. One correct pipeline and four broken ones,
fired together with
dlthub job trigger tag:jaffle:correctbad_configbase_urlon/api/v2/, 404 on the first requestbad_incrementalcursor_pathon a field the API does not havebad_paginationnextfrom the body; this API uses theLinkheaderbad_selectordata_selectorpoints atdata.results; the API returns a bare arrayOnly the first two wake an agent. The two green ones are the interesting half:
a
job.failtrigger cannot see them, so catching those means comparing rowcounts against
correct. That gap is what both tracks work on.alerts.pyis Track B's starting point. It fires on failure and records thatsomething broke; it cannot say why, because a trigger carries no logs or counts.
Cron schedules are stripped, so nothing fires unattended in a participant's
workspace. No credentials are needed for the pipelines themselves: the API is
public and the only token in the code is a deliberate
REPLACE_MEplaceholder.Why it is dev-only
It pins two prereleases,
dlt==1.30.1a0(the agent launcher is not in anystable release) and
dlthub-client==0.28.5a1. Those must not ship in thepublished wheel, so the template lives in
dev/rather thansrc/dlthub_init/scaffolds/and the make target overlays it after scaffolding.The model endpoint is empty by default, so nothing org-specific lives in this
repo. Pass it at scaffold time:
The toolkit workaround
scripts/place_toolkit_agents.pyexists becausedlthub ai toolkit installnever installs a toolkit's agents: dlt looks for them at
<toolkit>/agents/while the workbench ships them at
<toolkit>/dlthub/agents/. Filed asdlt-hub/dlt#4454.
verified_agent.pyuses the ordinary<toolkit>:<agent>reference and needs no change once dlt is fixed: delete the script and its
Makefile line.
Verified
End to end against a local stack running dlt-hub/runtime#2109, with
dlt 1.30.1a0,dlthub-client 0.28.5a1,pydantic-ai 2.43.0, on Azure OpenAI:bad_paginationloggingthis run is green but incomplete — expected 935 customers, loaded 100.job_inspectordiagnosedbad_configasconfig/highconfidence, citingboth the run log and the line of source that causes it.
my_agentdiagnosedbad_incrementalthe same way.alertsfires only on failure.background_agentmanifest;AGENT__API_KEYas a workspacevariable reaches the run; agent results and traces read back through
/telemetry/v1/*.Three Azure findings worth knowing:
api_versionis required even thoughAzure's v1 GA API no longer needs it, the endpoint is the
cognitiveservicesURL rather than a Foundry project URL, and
AGENT__MODELnames the deployment.Setting only
AGENT__API_KEYleaves the model at the agent's default and sendsthe key to Anthropic.
Not for merging yet
Agent jobs need dlt-hub/runtime#2109 merged and rolled out before platform tools
work on
api.dlthub.dev; on the local stack with that branch they work today.The hackathon clones this branch directly, so merging is not urgent.