8000 GitHub - moznion/octocoviz: Plot the history of an k1LoW/octocov metric over time · GitHub
[go: up one dir, main page]

Skip to content
moznionPublic

About

Plot the history of an k1LoW/octocov metric over time

Resources

Stars

5 stars

Watchers

0 watching

Forks

Repository files navigation

octocoviz - octocov + vizualization

Plot the history of an octocov metric over time.

octocov's git-branch datastore keeps only the newest report.json at a given path — the history is the branch's commit log. octocoviz walks that log, pulls one number out of each stored report with a jq expression, and writes the result as data or as a line chart.

$ octocoviz -q '.coverage.covered / .coverage.total * 100' \
    --label coverage --unit '%' -o coverage.png

Run from inside the repository being plotted, nothing but the query is needed: -repo defaults to the checkout the working directory is in. -branch and -path are read from .octocov.yml; pass them explicitly to override.

Because the metric is selected with jq rather than by name, anything octocov stores can be plotted, including custom_metrics written by tools like octocov-go-test-bench:

$ octocoviz -r https://github.com/moznion/cccc -b octocov \
    -p complexity-reports/moznion/cccc/complexity/report.json \
    -q '.custom_metrics[0].metrics[] | select(.key == "cyclomatic_sum") | .value' \
    --label 'cyclomatic complexity' -o complexity.svg

Install

$ go install github.com/moznion/octocoviz@latest

Requires the git executable on PATH.

Where the reports come from

A repository that stores reports on a branch is configured like this:

# .octocov.yml
report:
  if: is_default_branch
  datastores:
    - github://${GITHUB_REPOSITORY}@octocov/coverage-reports

which puts the report at coverage-reports/<owner>/<repo>/report.json on the octocov branch. octocoviz reads that config rather than making you assemble the path: it looks for .octocov.yml next to -repo and then in the working directory, or wherever -config says. ${GITHUB_REPOSITORY} is answered from -repo — or from the checkout's origin remote — since the value in a developer's shell is rarely the repository being plotted.

Standing in the repository is therefore enough on its own: with no -repo, the working tree it belongs to becomes the repository, which supplies both the config and, through origin, the ${GITHUB_REPOSITORY} the config is written in terms of. The working tree root is used rather than the working directory, so a subdirectory works too.

A config that names several branch datastores, or one whose reports go somewhere with no history (s3://, gs://, local:// keep only the newest), is reported rather than guessed at. A second metric set usually lives in its own config, which -config selects:

$ octocoviz -r . --config .octocov.complexity.yml \
    -q '.custom_metrics[0].metrics[] | select(.key == "cyclomatic_sum") | .value'

-r accepts a clonable URL or a local path. A URL is mirrored into a bare clone under the user cache directory and updated on later runs; --no-fetch reuses it as-is. A local path may be any working clone — if the report branch only exists as origin/<branch>, that is used, so there is no need to check it out.

Choosing a range

Without bounds, the walk goes back to the first stored report.

Flag Meaning
--from, --to Bound by report timestamp; 2006-01-02 or RFC3339.
--last N Keep the newest N points.
--from-commit, --to-commit Bound by commit of the measured repository.

Commit bounds name the SHA octocov recorded in the report's .commit field — the commit the metrics were measured at, the one you would read off a PR — not a commit of the branch storing the reports. Short SHAs work, bounds are inclusive, and if a commit produced several reports (a re-run CI job) all of them are inside the bound. Naming a store-side SHA by mistake is reported as such rather than silently returning nothing.

Note that octocov only writes a report when its workflow runs, so a commit that skipped CI has no report and cannot be used as a bound.

--dedupe keeps only the newest report per measured commit, collapsing CI re-runs into one point each. It is off by default: duplicate reports are real measurements, and dropping them should be a decision, not a default.

Thinning a long history

A repository that has been running octocov for a year has thousands of reports, and a 7-inch chart has about 600 pixels of plot to draw them in. --every keeps the newest report in each interval:

$ octocoviz -r . -q '...' --every 1d -o coverage.png

It takes 1d, 1w, 12h, 90m — Go's duration syntax plus the d and w suffixes it lacks. Buckets start at local midnight whatever their width, so "one point per day" means the day you are looking at rather than a window offset by your timezone.

The survivor is a real report, never an average. Every point carries the commit it was measured at, and the mean of two commits is not a commit anyone can open; thinning a history should drop measurements, not invent one. It is also what makes --every fast: walking newest-first, reports that lose their bucket are skipped before their megabyte of coverage detail is ever parsed.

Bursts and quiet weeks

octocov histories are not evenly spaced. A day of CI activity puts a dozen reports within hours of each other, then nothing happens for a week — and on a time axis those busy stretches collapse into a vertical wall while the quiet ones take up most of the chart.

-x index places reports at even spacing, in the order they were taken:

$ octocoviz -r . -q '...' -x index -o coverage.png

Every report then gets the same width. The axis no longer shows elapsed time — a quiet week looks like one short step — so it is a choice rather than the default. Where several reports share a date, the tick labels drop to date-and-time so no two read alike.

Gaps in the history

When CI stops running for a while, the line joins the last report before the silence to the first one after it, which reads as "the metric held steady" rather than "nobody measured". --gap breaks it instead:

$ octocoviz -r . -q '...' --every 1d --gap 7d -o coverage.png

The points on either side are still drawn; only the connection between them is dropped. The gap is carried in the interchange JSON, so it survives -f json | charty.

Pinning the value axis

The axis is chosen from the data and is deliberately not anchored at zero — a coverage series living in 93.2–93.8% would be flattened by a zero baseline. --y-min and --y-max override that when you want the full 0–100, or the same scale across two charts.

The x-axis tick layout is chosen from the span in both modes, so there is nothing to configure.

Drawing the acceptable threshold

--acceptable 80 draws a dashed reference line, and --acceptable auto reads it from the same config that supplied the store — so the line and the series describe the same metric:

coverage:
  acceptable: ">= 80%"

It is opt-in rather than automatic because a rule stretches the value axis to reach it. A coverage series sitting at 93.5–93.8% is drawn on a very different scale once 80% has to fit on the same axis. That is the right trade when you are asking "am I above the line", and the wrong one when you are asking "what changed this week", so the choice stays yours.

octocov evaluates acceptable as a full expression, so it also holds conditions like diff >= 0 that describe a movement rather than a level. Those have no line to draw and are reported as such.

Per-point metadata

--meta 'name=<jq expression>' attaches whatever else a report carries to every point. It is repeatable, and the expression is evaluated against the same document as --query:

$ octocoviz ... \
    --meta 'covered lines=.coverage.covered' \
    --meta 'files=.coverage.files | length'

Metadata reaches the tooltip and the table of the HTML page, and becomes a column in CSV. A query that yields null on some reports simply leaves those points unannotated, rather than recording a blank that reads like a measured zero.

Two keys travel by default because a reader cannot reconstruct them: ref, the ref the metrics were measured on, and store, the abbreviated commit of the report branch itself. A --meta of the same name overrides them.

Output

--format takes png, svg, html, json or csv. If it is omitted the format is taken from the -o extension, and failing that the series is written to stdout as JSON.

Rendering is charty's job, not octocoviz's. octocoviz collects the series and hands it over; everything about how a chart looks lives there, including the interactive HTML page and its light/dark palette. --format json emits charty's interchange JSON, so a series can be collected once and rendered later, or piped:

$ octocoviz -r ... -q '...' -f json > coverage.json
$ charty -o coverage.png coverage.json
$ charty -o coverage.html coverage.json

The measured commit becomes the point's label, and --commit-url builds its href, so every point in the HTML page can be clicked through to the commit that produced it. The default URL is derived from the report's own repository field; a different host — GitHub Enterprise, or a mirror — needs --commit-url 'https://ghe.example.com/o/r/commit/{commit}', and --commit-url none turns the links off.

Two limits inherited from charty are worth knowing.

PNG and SVG cannot draw CJK. The fonts embedded in gonum/plot cover Latin, Greek and Cyrillic only, so Japanese text in --title, --label or --unit will be missing from the image. octocoviz says so rather than letting it pass:

$ octocoviz -r . -q '...' --title 'カバレッジ推移' -o coverage.png
octocoviz: png: the chart font has no glyphs for "カバレッジ推移" in the title; those
characters will be missing — render html for non-Latin text

Use --format html for non-Latin text; that page uses the reader's own system fonts.

GitHub cannot embed HTML or JavaScript in a README or a PR comment, so png and svg remain the formats that plug into octocov's own workflow.

Storage adapters

Only a git branch is supported today, but the storage is behind an interface so that other backends can be added without touching the rest of the tool:

// internal/source
type Source interface {
	Walk(ctx context.Context, fn func(context.Context, Record) error) error
	Describe() string
	Close() error
}

An adapter yields records newest-first and stops early when the caller returns ErrStop, which is what keeps --last and --from-commit cheap on a long history. Adapters register themselves by name in init and are selected with --store.

Worth knowing before adding one: of octocov's datastores, only some retain history at all. A git branch does, and BigQuery and Mackerel accumulate rows. S3, GCS and local directories overwrite a single report.json per repository, so they expose only the latest value and cannot back a time series unless object versioning is enabled.

Plotting several series together

octocoviz collects one series per run. charty draws up to eight, so several runs can be merged with jq — the interchange JSON is the seam:

$ octocoviz -r . -q '...' --label core -f json > core.json
$ octocoviz -r . --config .octocov.other.yml -q '...' --label other -f json > other.json
$ jq -s '{title: "coverage by crate", series: map(.series[0])}' core.json other.json \
    | charty -o both.png

charty refuses series in different units on one value axis, which is the guard rail worth knowing about here: percent and seconds together would be a dual-axis chart with the second axis left off.

Development

$ make            # fmt-check, lint, vendor-check, test, build
$ make build      # bin/octocoviz
$ make test
$ make lint       # golangci-lint, pinned in the Makefile so CI agrees
$ make fmt        # gofmt and goimports

Dependencies are vendored, so a build needs no network. While charty is still a local replace directive that also means a checkout builds without a sibling charty clone — but it means the vendored copy of charty goes stale whenever charty is edited, and the build keeps working against the old one. make vendor refreshes it; make vendor-check fails if it needs refreshing, and runs in CI.

Layout

main.go              flags, and the composition of the two halves below
internal/source/     storage adapters: where reports live and how to walk them
internal/series/     octocov's schema: jq evaluation, commit bounds, and the
                     translation into charty's representation

octocov's vocabulary — commits, refs, report stores — stops at internal/series/chart.go, where it becomes charty's: labels, links and metadata. charty never learns what a commit is.

About

Plot the history of an k1LoW/octocov metric over time

Resources

Stars

5 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages

0