8000 ci: count clicks and keystrokes per feature, with a baseline by adeelahmad · Pull Request #298 · adeelahmad/lense · GitHub
[go: up one dir, main page]

Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
63 changes: 63 additions & 0 deletions .github/workflows/clicks.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
name: Clicks

# Drives a fresh Lens through setup and its main features and counts the clicks and keystrokes each takes
# (ci/clicks). A journey that takes more than ci/clicks/baseline.json says, or fails, turns this red; a change that
# takes fewer updates the baseline in the same pull request.
on:
push:
branches: [main]
pull_request:
paths:
- "nextjs-frontend/**"
- "fastapi_backend/app/**"
- "installer/**"
- "ci/clicks/**"
- ".github/workflows/clicks.yml"
workflow_dispatch:

concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: ${{ github.event_name == 'pull_request' }}

permissions:
contents: read

jobs:
clicks:
name: Clicks and keystrokes per feature
runs-on: ${{ fromJSON(vars.CI_RUNNER || '"ubuntu-latest"') }}
timeout-minutes: 40
steps:
- uses: actions/checkout@v4
- uses: astral-sh/setup-uv@v7
with:
cache-local-path: ${{ runner.environment == 'self-hosted' && format('{0}/uv-cache', runner.tool_cache) || '' }}
python-version: "3.12"
- uses: pnpm/action-setup@v6
with:
package_json_file: nextjs-frontend/package.json
run_install: false
- uses: actions/setup-node@v7
with:
node-version: "22"
cache: ${{ runner.environment == 'github-hosted' && 'pnpm' || '' }}
cache-dependency-path: nextjs-frontend/pnpm-lock.yaml
- name: Install
run: |
make env
(cd fastapi_backend && uv sync --frozen)
(cd nextjs-frontend && pnpm install --frozen-lockfile && pnpm build)
(cd ci/clicks && npm ci && npx playwright install --with-deps chromium)
- name: Survey
run: ci/clicks/run.sh --compare ci/clicks/baseline.json --fail-on-regression
- name: Summary
if: always()
run: |
if [ -f ci/clicks/results/clicks.md ]; then
sed -n '1,/^## Per page/p' ci/clicks/results/clicks.md >> "$GITHUB_STEP_SUMMARY"
fi
- uses: actions/upload-artifact@v7
if: always()
with:
name: clicks
path: ci/clicks/results/
7 changes: 7 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,13 @@ docs-build: ## Build the documentation site, failing on broken links

# Run it (Docker)
PROD_COMPOSE=$(DOCKER_COMPOSE) -f docker-compose.prod.yml
.PHONY: clicks
clicks: env ## Count the clicks and keystrokes each journey takes on a fresh Lens (ci/clicks), compared with the baseline
cd $(BACKEND_DIR) && uv sync
cd $(FRONTEND_DIR) && pnpm install && pnpm run build
cd ci/clicks && npm ci
ci/clicks/run.sh --compare ci/clicks/baseline.json

.PHONY: run dev stop setup-code logs

run: .env ## Build once and run the whole stack, no hot reload (the fast way to use it): http://localhost:3000
Expand Down
2 changes: 2 additions & 0 deletions ci/clicks/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
node_modules/
results/
34 changes: 34 additions & 0 deletions ci/clicks/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
# Clicks and keystrokes per feature

A survey of what Lens asks of people: it drives a fresh Lens in Chromium from the install wizard through setup and
each main feature, the way a person would with the defaults, and counts the clicks, keystrokes and passkey touches
each takes, per feature, per module and per page. `baseline.json` is the last accepted run; CI (the Clicks
workflow) fails a pull request whose journeys take more, so friction only goes down.

```sh
make clicks # builds, then runs the survey and compares with the baseline
ci/clicks/run.sh --only chat-ask,note-create # some journeys (ids in journeys.mjs)
```

`run.sh` starts a Lens of its own on a temporary database and data folder (ports 3000, 8000, 3990 and 11434 must be
free), a stand-in OpenAI-compatible model on 11434 (`fake-model.mjs`, so the installer and the setup wizard find a
local model and the assistant answers), and the web installer with a stand-in for install.sh (`fake-install.sh`).
Results go to `results/clicks.json` and `results/clicks.md`.

## What counts

- **Clicks**: each mouse click. Typing into a field that doesn't have the focus adds the click that focuses it.
Choosing from a native select is two; a file chooser is two (the button, then the file).
- **Keys**: each character typed, plus each key or shortcut pressed (Enter, Ctrl+K, Ctrl+S count one). A paste is one.
- **Prompts**: passkey touches.
- A step whose control is covered by something else (the floating Ask button) is still one click, marked
"covered" in the steps.
- **Per page**: reach is clicks from Home through the sidebar (sections and tabs are two); controls are the buttons,
links and fields in the page's main area on a fresh install, not counting navigation.

Journeys run in order on the same Lens, each from Home unless it continues the one before (setup does).

## When a change cuts clicks

Run the survey, check `results/clicks.md` shows the journey going down, and copy `results/clicks.json` over
`baseline.json` in the same pull request. A new journey goes at the end of `journeys.mjs` with an id that stays.
Loading
Loading
0