8000 WebDriver: frames, shadow roots, alerts, windows, uploads and print by adeelahmad · Pull Request #12 · adeelahmad/Tether · GitHub
[go: up one dir, main page]

Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
8000 Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,13 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
browser's cookies. HTTPS via a local CA (`/tether/v1/ca.pem`).
- W3C WebDriver endpoints for the common Selenium command set
(see `docs/WEBDRIVER.md`), sharing the same sessions as the proxy.
- WebDriver: frames (same- and cross-site, nested) and shadow roots (open
and closed); alerts, confirms and prompts with `unhandledPromptBehavior`;
multiple windows (New Window, popups, switching, closing); window rect,
maximise, minimise and fullscreen for windows the session owns; file
uploads, including Selenium's remote file upload; and Print Page.
- Background tabs get Chrome focus emulation, so real input reaches pages
with cross-site iframes promptly.
- WebDriver Actions API (`POST`/`DELETE /session/{id}/actions`): key,
mouse, pen, touch and wheel input as real CDP events, so `ActionChains`
hover, drag and drop, double/context click, modifier chords and scrolling
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -219,7 +219,7 @@ local CA, and it only drives Chromium-family browsers.
| Uses your real logged-in profile | ✅ | ✅ | ❌ (re-login, stored creds) | ❌ |
| Needs a browser extension | **No** | Yes | No | No |
| Plain HTTP proxy (curl, any language) | ✅ | ❌ | Rarely | ❌ |
| W3C WebDriver / Selenium | ✅ common set | ❌ | Often | ✅ |
| W3C WebDriver / Selenium | ✅ incl. frames, alerts, windows, uploads, PDF | ❌ | Often | ✅ |
| MCP for agents | Planned | Often | Often | Via wrappers |
| Runs entirely on your machine | ✅ | ✅ | ❌ | ✅ |
| Remote access | Tailscale + token | Usually local only | ✅ | DIY |
Expand Down
2 changes: 2 additions & 0 deletions SECURITY.md
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,8 @@ a sandbox against a determined attacker who also shares the browser profile
| One agent reads or steers another agent's tab | Per-token ownership: non-admin tokens only resolve their own tabs, including via cookie, header and WebDriver session ID | Admin tokens see everything by design. |
| Tether used to read a page the user has open | Only tabs Tether opened, or tabs explicitly addressed by ID, are driven; adopting requires an admin token | Any admin token can address any tab by ID. |
| Chrome DevTools port exposed | `--launch` binds it to loopback; Tether never re-exposes it | If you start Chrome yourself with `--remote-debugging-address=0.0.0.0`, that is outside Tether's control. |
| A client attaches local files to a page (WebDriver file inputs) | Clients are fully trusted (see above); remote uploads land in a per-session temp dir deleted when the session ends | A token holder can make a page upload any file the Tether user can read. |
| Automation rearranges the person's screen | Window resize/maximise/minimise/fullscreen only act on windows holding nothing but Tether tabs | — |
| Secrets printed to the console by a page (tokens, PII in debug logs) | Console capture is only readable through the authenticated listener and is never written to Tether's own logs | Any token holder can read the captured console of any tab. `--console-buffer 0` turns capture off. |
| Sites detecting automation via `Runtime.enable` | — | Console capture enables the `Runtime` domain, which some anti-bot scripts can detect. `--console-buffer 0` leaves it off. |
| Denial of service (huge bodies, many tabs) | 32 MiB request body cap; per-tab serialisation; idle reaping; console buffer capped per tab and per entry (8 KiB) | A token holder can open many tabs. |
Expand Down
9 changes: 8 additions & 1 deletion docs/ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,14 @@ A **session is exactly one Chrome tab** (a CDP page target), keyed by its
target ID. That ID is the only handle a client ever needs: it is the
`X-Tether-Tab-ID` header, the `tether_tab` cookie value, the WebDriver
session ID, and the path segment in `/tether/v1/tabs/{id}`. A tab opened by
one face can be driven by any other.
one face can be driven by any other. (A WebDriver session can grow extra
windows — new tabs and popups — and each window handle is again a tab ID
backed by its own session.)

Frames are not sessions: `cdp.Tab.InFrame` gives a view of a tab whose
script and DOM calls run inside a child frame, same-process or
out-of-process (cross-site frames are auto-attached as CDP sessions of the
tab), while navigation, input and screenshots stay with the tab.

Resolution precedence for proxy requests:

Expand Down
4 changes: 4 additions & 0 deletions docs/POLICIES.md
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,10 @@ duplicate secrets are startup errors.
- The proxy's default tab per site is per token: two agents browsing
`github.com` get two tabs.
- Only admin tokens can adopt tabs Tether did not open.
- WebDriver popups and new windows belong to the session that opened them:
same owner, same session rules, guarded the same way. A popup that loaded
a forbidden page before Tether attached is closed when the session
switches to it, and the switch fails.
- With no authentication configured (loopback, no token) everything behaves
as before: one implicit admin.

Expand Down
117 changes: 97 additions & 20 deletions docs/WEBDRIVER.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,8 +32,15 @@ With a token: `ClientConfig(remote_server_addr=..., auth_type=AuthType.BEARER, t
token rules (see [POLICIES.md](POLICIES.md)). Navigating outside them fails
with `unsupported operation`; *Execute Script* needs a token with the
`scripts` permission.
- One session = one tab = one window handle. Window switching, new windows and
frames are out of scope.
- To give the session a browser window of its own (so resizing it never
touches yours), pass `"tether:options": {"newWindow": true}`.
- Every window handle is a tab ID. A session starts with one window and
gains one for each **New Window** it opens and each **popup** its pages
open (`window.open`, `target=_blank` links, popups of popups). Tabs that
existed before the session started are never counted as its popups.
- Closing the last window ends the session. Deleting the session closes
every window it owns: the tabs it opened and its popups, but not an
attached tab (unless `keepTab` is false).

## Supported commands

Expand All @@ -45,41 +52,53 @@ With a token: `ClientConfig(remote_server_addr=..., auth_type=AuthType.BEARER, t
| Navigate To / Get Current URL | `POST`, `GET /session/{id}/url` |
| Back / Forward / Refresh | `POST /session/{id}/back`, `/forward`, `/refresh` |
| Get Title | `GET /session/{id}/title` |
| Get Page Source | `GET /session/{id}/source` (live DOM) |
| Get Page Source | `GET /session/{id}/source` (live DOM of the current frame) |
| Get Window Handle / Handles | `GET /session/{id}/window`, `/window/handles` |
| Switch To Window | `POST /session/{id}/window` |
| New Window | `POST /session/{id}/window/new` (`"type": "tab"` or `"window"`) |
| Close Window | `DELETE /session/{id}/window` |
| Get / Set Window Rect | `GET`, `POST /session/{id}/window/rect` |
| Maximize / Minimize / Fullscreen Window | `POST /session/{id}/window/maximize`, `/minimize`, `/fullscreen` |
| Switch To Frame / Parent Frame | `POST /session/{id}/frame`, `/frame/parent` |
| Find Element(s) | `POST /session/{id}/element`, `/elements` |
| Find Element(s) From Element | `POST /session/{id}/element/{eid}/element`, `/elements` |
| Get Element Shadow Root | `GET /session/{id}/element/{eid}/shadow` |
| Find Element(s) From Shadow Root | `POST /session/{id}/shadow/{sid}/element`, `/elements` |
| Get Active Element | `GET /session/{id}/element/active` |
| Element Click | `POST /session/{id}/element/{eid}/click` |
| Element Clear | `POST /session/{id}/element/{eid}/clear` |
| Element Send Keys | `POST /session/{id}/element/{eid}/value` |
| Element Send Keys (including file inputs) | `POST /session/{id}/element/{eid}/value` |
| Get Element Text / Tag Name / Rect | `GET …/element/{eid}/text`, `/name`, `/rect` |
| Get Element Attribute / Property / CSS Value | `GET …/element/{eid}/attribute/{n}`, `/property/{n}`, `/css/{n}` |
| Is Element Enabled / Selected / Displayed | `GET …/element/{eid}/enabled`, `/selected`, `/displayed` |
| Execute Script / Execute Async Script | `POST /session/{id}/execute/sync`, `/execute/async` |
| Get All / Named Cookie | `GET /session/{id}/cookie`, `/cookie/{name}` |
| Add Cookie | `POST /session/{id}/cookie` |
| Delete Cookie / All Cookies | `DELETE /session/{id}/cookie/{name}`, `/cookie` |
| Dismiss / Accept Alert | `POST /session/{id}/alert/dismiss`, `/alert/accept` |
| Get / Send Alert Text | `GET`, `POST /session/{id}/alert/text` |
| Take Screenshot / Element Screenshot | `GET /session/{id}/screenshot`, `…/element/{eid}/screenshot` |
| Get Window Handle(s) | `GET /session/{id}/window`, `/window/handles` |
| Switch To Window (own handle only) | `POST /session/{id}/window` |
| Close Window (closes the tab, ends the session) | `DELETE /session/{id}/window` |
| Print Page | `POST /session/{id}/print` |
| Upload File (Selenium extension) | `POST /session/{id}/se/file` (also `/file`) |
| Perform / Release Actions | `POST`, `DELETE /session/{id}/actions` — key, pointer (mouse, pen, touch) and wheel input |
| Get Log / Get Log Types (Selenium extension) | `POST /session/{id}/se/log`, `GET …/se/log/types` (also `/log`, `/log/types`) — types `browser`, `driver` |
| Get / Set Window Rect, Maximize | `GET`/`POST …/window/rect`, `POST …/window/maximize` — reports the rect; resizing your real window is intentionally a no-op |

Locator strategies: `css selector`, `xpath`, `link text`,
`partial link text`, `tag name` (Selenium maps `By.ID`, `By.NAME` and
`By.CLASS_NAME` to CSS).
`By.CLASS_NAME` to CSS). From a shadow root, every strategy but `xpath`.

Behaviour notes:
## Behaviour notes

- **Element references are stable**: they are Chrome backend node IDs, so
finding the same node twice returns an equal element. After navigation
they become `stale element reference`.
- **Click** scrolls the element into view, checks it is the topmost element
at its centre (`element click intercepted` otherwise), dispatches a real
mouse click, and waits for a navigation it triggers. Clicking an
`<option 10BC8 >` selects it.
they become `stale element reference`. Shadow roots get references the
same way (`detached shadow root` once gone).
- **Click** scrolls the element (and the frames around it) into view,
checks it is the topmost element at its centre (`element click
intercepted` otherwise), dispatches a real mouse click, and waits for a
navigation it triggers. Clicking an `<option>` selects it. Tether's tabs
stay in the background; Chrome's focus emulation lets real input reach
them, including cross-site iframes, without taking focus from you.
- **Send keys** focuses the element and types real key events; W3C special
keys (Enter, Tab, arrows, Backspace, …) and sticky modifiers are supported.
Enter waits for a form submission's navigation.
Expand All @@ -90,8 +109,67 @@ Behaviour notes:
`WARNING` and `SEVERE`. Capture starts when Tether attaches to the tab and
keeps the last `--console-buffer` entries; no `goog:loggingPrefs` needed.
The `driver` log is always empty.
- **Cookies** are your browser's real cookies. `DELETE /cookie` deletes the
current site's cookies from your profile — that *will* log you out.
- **Cookies** are your browser's real cookies, for the current frame's
document. `DELETE /cookie` deletes the current site's cookies from your
profile — that *will* log you out.

### Frames

`Switch To Frame` takes `null` (the top-level page), a frame index (the
`n`th `<iframe>`/`<frame>` in the current document) or an `<iframe>`/`<frame>`
element. Same-site and **cross-site (out-of-process) frames** both work, at
any depth: finding, clicking, typing, scripts, page source, cookies and
element screenshots all run in the current frame. As in the spec, title,
URL and full screenshots stay top-level, and navigating (including back,
forward and refresh) returns to the top-level page. An element from another
frame answers `no such element`; a frame that disappears answers
`no such window` until you switch.

### Shadow DOM

`Get Element Shadow Root` returns open **and closed** shadow roots (DevTools
can see closed roots even though page scripts cannot); the browser's own
built-in roots (inside `<input>`, `<video>`, …) are not exposed. Shadow
roots can be passed to and returned from scripts.

### Alerts, confirms and prompts

JavaScript dialogs (`alert`, `confirm`, `prompt`, `beforeunload`) are
handled through the alert commands. A click, key press or navigation that
opens one returns normally; a script that opens one returns `null`.
`unhandledPromptBehavior` decides what happens when any other command runs
while a dialog is open: `dismiss and notify` (default), `accept and notify`,
`dismiss`, `accept` or `ignore`. The "notify" variants (and `ignore`)
answer `unexpected alert open` with the dialog text in `data.text`.

### Windows

New windows open in the background, like every Tether tab. **Window
geometry is only changed when the window holds nothing but this session's
tabs** (or other Tether tabs) — a window from `New Window` with
`"type": "window"`, or a session started with `"newWindow": true`. In a
window shared with tabs you are using, set rect, maximise, minimise and
fullscreen leave it alone and report its current rect, so scripts that call
`maximize_window()` keep working without rearranging your screen.

### File uploads

Sending keys to an `<input type="file">` sets its files: one path per line
(several only if the input has `multiple`). Paths are on the machine Tether
and Chrome run on, and must exist (`invalid argument` otherwise). Inputs
may be hidden, as styled file pickers usually are.

Remote Selenium clients use their file detector, which uploads the file to
`/session/{id}/se/file` and types the returned path; with Python,
`driver.file_detector = LocalFileDetector()`. Uploaded files live in a
temporary directory that is deleted when the session ends.

### Print to PDF

`Print Page` follows the spec's options (`orientation`, `scale`,
`background`, `page`, `margin` in centimetres, `pageRanges`) and returns a
base64 PDF of the current window's page. Chrome builds that can only print
when headless answer `unsupported operation` in a normal window.

## Actions

Expand Down Expand Up @@ -136,8 +214,7 @@ ActionChains(driver).scroll_by_amount(0, 800).perform()

## Not supported (yet)

Frames and shadow roots, alerts/prompts, new
window/tab creation, window resizing/fullscreen/minimise, file uploads,
print-to-PDF, and BiDi. These return `unknown command` (or
Not planned for now: BiDi, `Get Computed Role / Label`, and the Selenium
Grid extensions other than file upload. These return `unknown command` (or
`unsupported operation`). Contributions welcome — they belong in
`internal/webdriver` on top of the existing `cdp.Tab` primitives.
84 changes: 84 additions & 0 deletions internal/cdp/cdp.go
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,8 @@ import (
type Browser interface {
// NewTab opens a fresh tab (about:blank) and attaches to it.
NewTab(ctx context.Context) (Tab, error)
// NewWindow is NewTab in a new browser window of its own.
NewWindow(ctx context.Context) (Tab, error)
// AttachTab attaches to an existing page target. It returns
// ErrNoSuchTab if the target does not exist or is not a page.
AttachTab(ctx context.Context, id string) (Tab, error)
Expand All @@ -31,6 +33,11 @@ type Browser interface {
// OnTabClosed registers fn to be called (from the event goroutine) when
// a page target is destroyed, for any reason.
OnTabClosed(fn func(id string))
// Window describes the browser window that holds a tab.
Window(ctx context.Context, tabID string) (Window, error)
// SetWindow changes the state or bounds of the window that holds a tab
// and returns the result.
SetWindow(ctx context.Context, tabID string, b WindowBounds) (Window, error)
// Version is the browser product string, e.g. "Chrome/141.0…".
Version(ctx context.Context) (string, error)
// Done is closed when the DevTools connection drops.
Expand Down Expand Up @@ -77,6 +84,33 @@ type Tab interface {

Click(ctx context.Context, x, y float64) error
Key(ctx context.Context, k KeyEvent) error

// InFrame returns a view of the tab whose script and DOM calls run in
// a child frame. path lists the backend node IDs of the <iframe> or
// <frame> elements from the top document down. Each step is resolved
// on every call, so the view follows navigations inside the frame and
// works for out-of-process (cross-site) frames. Page-level calls
// (navigation, input, screenshots, cookies, dialogs, PDF) still go to
// the tab. A step that is not, or is no longer, a frame yields
// ErrNoSuchFrame.
InFrame(path []int) Tab
// ShadowRoot returns the backend node ID of an element's open or
// closed shadow root, or 0 if it has none.
ShadowRoot(ctx context.Context, backendNodeID int) (int, error)
// SetFiles sets the selected files of an <input type=file>.
SetFiles(ctx context.Context, backendNodeID int, files []string) error
// PrintPDF renders the page as a PDF.
PrintPDF(ctx context.Context, opts PDFOptions) ([]byte, error)

// Dialog is the JavaScript dialog (alert, confirm, prompt,
// beforeunload) open in the tab, or nil. While one is open, script
// calls fail fast with ErrDialogOpen instead of hanging, and calls
// that open one (a click, a navigation) return ErrDialogOpen.
Dialog() *Dialog
// HandleDialog accepts or dismisses the open dialog, answering a
// prompt with promptText when it is non-nil.
HandleDialog(ctx context.Context, accept bool, promptText *string) error

// Mouse dispatches one mouse or pen event (move, press, release,
// wheel) in viewport CSS pixels.
Mouse(ctx context.Context, m MouseEvent) error
Expand All @@ -102,6 +136,49 @@ type TabInfo struct {
ID string `json:"id"`
URL string `json:"url"`
Title string `json:"title"`
// OpenerID is the tab that opened this one (window.open, a
// target=_blank link), if any.
OpenerID string `json:"openerId,omitempty"`
}

// Window is a browser window's geometry, in screen pixels.
type Window struct {
ID int `json:"windowId"`
Left int `json:"left"`
Top int `json:"top"`
Width int `json:"width"`
Height int `json:"height"`
State string `json:"windowState"` // normal, minimized, maximized, fullscreen
}

// WindowBounds is a change to a window. A State other than "" or
// "normal" ignores the bounds; nil bounds are left as they are.
type WindowBounds struct {
Left, Top, Width, Height *int
State string
}

// Dialog is an open JavaScript dialog.
type Dialog struct {
Type string `json:"type"` // alert, confirm, prompt, beforeunload
Message string `json:"message"`
DefaultPrompt string `json:"defaultPrompt"`
URL string `json:"url"`
}

// PDFOptions configure PrintPDF. Lengths are in inches; zero values take
// Chrome's defaults.
type PDFOptions struct {
Landscape bool `json:"landscape,omitempty"`
PrintBackground bool `json:"printBackground,omitempty"`
Scale float64 `json:"scale,omitempty"`
PaperWidth float64 `json:"paperWidth,omitempty"`
PaperHeight float64 `json:"paperHeight,omitempty"`
MarginTop float64 `json:"marginTop"`
MarginBottom float64 `json:"marginBottom"`
MarginLeft float64 `json:"marginLeft"`
MarginRight float64 `json:"marginRight"`
PageRanges string `json:"pageRanges,omitempty"`
}

// Wait selects how long Navigate blocks.
Expand Down Expand Up @@ -256,5 +333,12 @@ func (e *NavigationError) Error() string { return "navigation to " + e.URL + " f
// ErrNoSuchTab means the requested target does not exist.
var ErrNoSuchTab = errors.New("cdp: no such tab")

// ErrNoSuchFrame means a frame path no longer leads to a frame.
var ErrNoSuchFrame = errors.New("cdp: no such frame")

// ErrDialogOpen means a JavaScript dialog is open (or opened during the
// call) and is blocking the page.
var ErrDialogOpen = errors.New("cdp: a JavaScript dialog is open")

// networkIdleQuiet is how long the network must be quiet for WaitNetworkIdle.
const networkIdleQuiet = 500 * time.Millisecond
Loading
Loading
0