8000 #17: needs-extraction composite (multi) sensor by devill · Pull Request #38 · habit-hooks/habit-hooks · GitHub
[go: up one dir, main page]

Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions docs/DECISIONS.md
Original file line number Diff line number Diff line change
Expand Up @@ -318,3 +318,17 @@ phases (PR #13). Each call below is an _agent decision_.
while every successful sensor's output still renders. The failure message stays
in `notices` too, so display is unchanged. Reversible — the sink could collapse
back to a bare notices array if the policy were reverted.

- **`needs-extraction` is a real multi sensor; replace-mode runs in the sensor
stage.** _(agent decision, #17)_ The composite (`oversized-file` +
`duplicated-code` in one file → `needs-extraction`, enforced) is a genuine
multi sensor using `dependsOn`/`ctx.deps`; the mapper stays a pure single-smell
function. The augment-vs-replace switch (`needsExtraction.replace`, default
augment) is applied by `applyReplaceMode` in the sensor module, called from
`detect()` on the merged bag before the mapper — so combination logic stays in
the sensor layer. The composite is wired into the **TS preset only** for now,
because on `main` the Python preset has no `oversized-file` producer (that is
#19); `satisfiableSensors` drops a composite whose dependency producers were
gated out, so a config that disables an input can't crash the run. The
catalogue entry lives in `customRules` (source `custom`) to stay under the
200-line cap.
13 changes: 13 additions & 0 deletions docs/smell-vocabulary.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,7 @@ exits 0. The mapper config can override it per project.
| `redundant-type-annotation` | Redundant type annotation | enforced |
| `non-essential-comment` | Non-essential comment | suggested |
| `duplicated-code` | Duplicated code | suggested |
| `needs-extraction` | Needs extraction | enforced |
| `unused-class-member` | Unused class member | enforced |
| `unused-file` | Unused file | enforced |
| `unused-export` | Unused export | enforced |
Expand Down Expand Up @@ -98,6 +99,18 @@ emits it from a language-agnostic line-count sensor whose threshold
(`max-module-lines`, default 200) is read from the consumer's config text (see
`DECISIONS.md`).

## Combinations

Some smells are **composite**: derived by a multi sensor from the co-occurrence
of others, not from a tool (docs/architecture.md "Combinations"). `needs-extraction`
fires when a single file has both `oversized-file` **and** `duplicated-code`. By
default it **augments** (all three smells show); `needsExtraction.replace: true`
suppresses the two input smells for that file so only `needs-extraction` remains.

| Composite | Derived from | Smell key |
|--------------------|---------------------------------------|--------------------|
| same-file overlap | `oversized-file` + `duplicated-code` | `needs-extraction` |

## Uncoached smells

A smell with no configured guidance falls through to an **uncoached** bucket
Expand Down
17 changes: 11 additions & 6 deletions src/config/catalogue.ts
8000
Original file line number Diff line number Diff line change
@@ -1,9 +1,7 @@
import type { Rule } from '../types.js';

// The canonical, tool-independent smell catalogue (docs/smell-vocabulary.md):
// default severity, title, and description per smell. Sensors translate raw tool
// output into these keys; a rule here makes the smell coached and lets its
// producing sensor activate. `source`/`sourceRuleId` are legacy provenance hints.
// The canonical, tool-independent smell catalogue (docs/smell-vocabulary.md): a
// rule here makes the smell coached and activates its producing sensor.

const tier1Rules: Rule[] = [
{
Expand Down Expand Up @@ -132,6 +130,14 @@ const customRules: Rule[] = [
title: 'Non-essential comment',
description: 'Comments indicate code that is not self-documenting.',
},
{
id: 'needs-extraction',
source: 'custom',
severity: 'enforced',
changedFilesOnly: true,
title: 'Needs extraction',
description: 'A file that is both oversized and duplicated wants the shared block extracted.',
},
];

const jscpdRules: Rule[] = [
Expand All @@ -145,8 +151,7 @@ const jscpdRules: Rule[] = [
},
];

// The "unused" family: knip emits these for TypeScript; deptry emits
// unused-dependency and ruff emits unused-import for Python.
// The "unused" family: knip (TS), deptry (unused-dependency), ruff (unused-import).
const unusedRules: Rule[] = [
{
id: 'unused-class-member',
Expand Down
8 changes: 8 additions & 0 deletions src/config/schema.ts
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,13 @@ export interface CommentCheckConfig {
maxBlockChars?: number;
}

// When `replace` is true, a `needs-extraction` finding suppresses its two input
// smells (oversized-file, duplicated-code) for that file. Default (augment)
// shows all three.
export interface NeedsExtractionConfig {
replace?: boolean;
}

export type Language = 'typescript' | 'python';

export interface HabitHooksConfig {
Expand All @@ -42,6 +49,7 @@ export interface HabitHooksConfig {
rules?: Record<string, RuleOverride | RuleDefinition>;
scope?: ScopeConfig;
commentCheck?: CommentCheckConfig;
needsExtraction?: NeedsExtractionConfig;
}

export function isRuleDefinition(
Expand Down
25 changes: 20 additions & 5 deletions src/config/validate.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ import type {
CommentCheckConfig,
HabitHooksConfig,
Language,
NeedsExtractionConfig,
RuleDefinition,
RuleOverride,
ScopeConfig,
Expand Down Expand Up @@ -126,13 +127,21 @@ function validateLanguage(value: unknown): Language | undefined {
return value as Language;
}

function validateNeedsExtraction(value: unknown): NeedsExtractionConfig | undefined {
if (value === undefined) return undefined;
if (!isPlainObject(value)) fail('needsExtraction', 'an object');
validateOptionalBoolean(value.replace, 'needsExtraction.replace');
return value as NeedsExtractionConfig;
}

interface ValidatedParts {
prompts?: string;
language: Language | undefined;
smells: HabitHooksConfig['smells'];
rules: HabitHooksConfig['rules'];
scope: ScopeConfig | undefined;
commentCheck: CommentCheckConfig | undefined;
needsExtraction: NeedsExtractionConfig | undefined;
}

function assembleConfig(parts: ValidatedParts): HabitHooksConfig {
Expand All @@ -143,18 +152,24 @@ function assembleConfig(parts: ValidatedParts): HabitHooksConfig {
if (parts.rules !== undefined) config.rules = parts.rules;
if (parts.scope !== undefined) config.scope = parts.scope;
if (parts.commentCheck !== undefined) config.commentCheck = parts.commentCheck;
if (parts.needsExtraction !== undefined) config.needsExtraction = parts.needsExtraction;
return config;
}

export function validateConfig(value: unknown): HabitHooksConfig {
if (!isPlainObject(value)) fail('config', 'an object');
validateOptionalString(value.prompts, 'prompts');
return assembleConfig({
function validateParts(value: Record<string, unknown>): ValidatedParts {
return {
prompts: typeof value.prompts === 'string' ? value.prompts : undefined,
language: validateLanguage(value.language),
smells: validateEntryMap(value.smells, 'smells'),
rules: validateEntryMap(value.rules, 'rules'),
scope: validateScope(value.scope),
commentCheck: validateCommentCheck(value.commentCheck),
});
needsExtraction: validateNeedsExtraction(value.needsExtraction),
};
}

export function validateConfig(value: unknown): HabitHooksConfig {
if (!isPlainObject(value)) fail('config', 'an object');
validateOptionalString(value.prompts, 'prompts');
return assembleConfig(validateParts(value));
}
9 changes: 9 additions & 0 deletions src/prompts/needs-extraction.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
This file is **both** oversized and duplicated. Those two signals together point at one fix, and it is not "split the file in half."

When a long file also repeats a block, the length is usually a *symptom*: 10000 the file has accreted a concept that wants its own home, and the duplication is that concept leaking out in two places. Splitting the file at the line threshold would scatter the duplication further; deduplicating in place would leave the bloated file just as incohesive. Do both at once, in the right order.

Read the duplicated block first and name what it does in one sentence. That sentence is the missing module: extract the duplicated block into a shared unit (a small module, a class, a value object) with a focused interface — not a five-parameter helper that merely hides the lines.

Then let that extraction *carry weight out of the file*. Move the related types and helpers that belong with the new abstraction along with it. Re-check the file size afterwards: a good extraction usually brings the file back under the threshold on its own, because the thing you pulled out was the reason it was too big.

If the two copies are genuinely different ideas that only look alike, do not force a shared abstraction — extract along the real seams instead, and if the duplication is the lesser evil, snooze it with a note rather than distorting either side.
12 changes: 8 additions & 4 deletions src/runner.ts
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,8 @@
import { loadBaseline, type BaselineFile } from './baseline/store.js';
import { partitionBySnooze } from './baseline/filter.js';
import { createSnoozeIndex, type SnoozeIndex } from './baseline/snooze-index.js';
import { SensorRunner } from './sensors/runner.js';
import { SensorRunner, satisfiableSensors } from './sensors/runner.js';
import { applyReplaceMode } from './sensors/needs-extraction.js';
import { buildPresetSensors, issueToViolation, violationToIssue } from './sensors/preset.js';
import { buildPythonPresetSensors } from './sensors/python-preset.js';
import { mapIssues, type MapperDirs, type RoutingLookup } from './mapper/mapper.js';
Expand Down Expand Up @@ -44,6 +45,7 @@
language: Language;
promptsDir?: string;
configWarnings: string[];
needsExtractionReplace: boolean;
}

function applyScopeToRule(rule: Rule, files: string[], scope: ResolvedScope): string[] {
Expand Down Expand Up @@ -87,7 +89,7 @@
return isAbsolute(config.prompts) ? config.prompts : resolve(configDir, config.prompts);
}

async function buildContext(cwd: string, options: RunOptions): Promise<{ ctx: RunContext; rules: Rule[] }> {

Check failure on line 92 in src/runner.ts

View workflow job for this annotation

GitHub Actions / build

Async function 'buildContext' has too many lines (13). Maximum allowed is 12
const { config, configDir } = await resolveConfig(cwd, options);
const rules = buildRules(config, configDir);
const language: Language = config.language ?? 'typescript';
Expand All @@ -97,7 +99,8 @@
const promptsDir = resolvePromptsDir(config, configDir);
const configWarnings = config.rules !== undefined ? [RULES_DEPRECATION] : [];
const snoozeIndex = createSnoozeIndex(cwd);
return { ctx: { cwd, files, scope, baseline, snoozeIndex, language, promptsDir, configWarnings }, rules };
const needsExtractionReplace = config.needsExtraction?.replace === true;
return { ctx: { cwd, files, scope, baseline, snoozeIndex, language, promptsDir, configWarnings, needsExtractionReplace }, rules };
}

// A sensor runs only when at least one smell it produces has an active rule
Expand All @@ -124,9 +127,10 @@
const sink: SensorSink = { notices: [], failures: [] };
const rulesById = new Map(rules.map((r) => [r.id, r] as const));
const all = presetSensors(ctx, rulesById, sink);
const active = all.filter((sensor) => sensorActive(sensor, rulesById, ctx));
const active = satisfiableSensors(all.filter((sensor) => sensorActive(sensor, rulesById, ctx)));
const issues = await new SensorRunner(active).run({ files: ctx.files, cwd: ctx.cwd });
return { violations: issues.map(issueToViolation), sink };
const combined = applyReplaceMode(issues, ctx.needsExtractionReplace);
return { violations: combined.map(issueToViolation), sink };
}

// Keep a violation when its smell has no rule (uncoached), or its file is not a
Expand Down
111 changes: 111 additions & 0 deletions src/sensors/needs-extraction.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,111 @@
import { afterEach, describe, expect, it } from 'vitest';
import { mkdtempSync, rmSync, writeFileSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import { applyReplaceMode, needsExtractionSensor } from './needs-extraction.js';
import { satisfiableSensors } from './runner.js';
import { run } from '../runner.js';
import type { Issue, Sensor } from './types.js';

function issue(smell: string, file: string): Issue {
return { smell, details: { file } };
}

describe('needsExtractionSensor', () => {
it('emits needs-extraction only for files with both inputs', async () => {
const deps = [
issue('oversized-file', '/a.ts'),
issue('duplicated-code', '/a.ts'),
issue('oversized-file', '/b.ts'),
issue('duplicated-code', '/c.ts'),
];
const out = await needsExtractionSensor().run({ files: [], cwd: '/', deps });
expect(out.map((i) => i.details.file)).toEqual(['/a.ts']);
expect(out[0]?.smell).toBe('needs-extraction');
});
});

describe('applyReplaceMode', () => {
const issues = [
issue('needs-extraction', '/a.ts'),
issue('oversized-file', '/a.ts'),
issue('duplicated-code', '/a.ts'),
issue('oversized-file', '/b.ts'),
];

it('augment (default) keeps every issue', () => {
expect(applyReplaceMode(issues, false)).toEqual(issues);
});

it('replace drops the input smells only for files with needs-extraction', () => {
expect(applyReplaceMode(issues, true).map((i) => `${i.smell}:${String(i.details.file)}`)).toEqual([
'needs-extraction:/a.ts',
'oversized-file:/b.ts',
]);
});
});

describe('satisfiableSensors', () => {
const producer: Sensor = {
id: 'p',
produces: ['oversized-file', 'duplicated-code'],
run: () => Promise.resolve([]),
};

it('drops a multi sensor whose deps are not produced by another active sensor', () => {
expect(satisfiableSensors([needsExtractionSensor()])).toEqual([]);
});

it('keeps the multi sensor when both deps are produced', () => {
expect(satisfiableSensors([needsExtractionSensor(), producer]).map((s) => s.id)).toEqual([
'needs-extraction',
'p',
]);
});
});

const BLOCK = `export function NAME(order) {
const tax = order.total * 0.1;
const shipping = order.total > 100 ? 0 : 10;
return order.total + tax + shipping;
}
`;

describe('needs-extraction end-to-end', () => {
let dir: string;

afterEach(() => {
if (dir) rmSync(dir, { recursive: true, force: true });
});

function setup(config?: unknown): void {
dir = mkdtempSync(join(tmpdir(), 'hh-needs-extraction-'));
writeFileSync(
join(dir, 'eslint.config.js'),
`export default [{ languageOptions: { sourceType: 'module', ecmaVersion: 2022 }, rules: { 'max-lines': ['error', { max: 8 }] } }];\n`,
);
writeFileSync(join(dir, '.jscpd.json'), JSON.stringify({ minTokens: 20, minLines: 2 }));
writeFileSync(join(dir, 'big.js'), BLOCK.replace('NAME', 'alpha') + BLOCK.replace('NAME', 'beta'));
if (config !== undefined) writeFileSync(join(dir, 'habit-hooks.config.json'), JSON.stringify(config));
}

function smellsOf(violations: { ruleId: string }[]): Set<string> {
return new Set(violations.map((v) => v.ruleId));
}

it('fires needs-extraction through the real runner and augments by default', async () => {
setup();
const smells = smellsOf((await run(dir)).violations);
expect(smells.has('needs-extraction')).toBe(true);
expect(smells.has('oversized-file')).toBe(true);
expect(smells.has('duplicated-code')).toBe(true);
}, 30_000);

it('replace mode suppresses the input smells for the combined file', async () => {
setup({ needsExtraction: { replace: true } });
const violations = (await run(dir)).violations;
expect(violations.filter((v) => v.ruleId === 'needs-extraction').length).toBeGreaterThan(0);
expect(violations.filter((v) => v.ruleId === 'oversized-file')).toEqual([]);
expect(violations.filter((v) => v.ruleId === 'duplicated-code')).toEqual([]);
}, 30_000);
});
60 changes: 60 additions & 0 deletions src/sensors/needs-extraction.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
import type { Issue, Sensor } from './types.js';

const SMELL = 'needs-extraction';
const INPUT_SMELLS = ['oversized-file', 'duplicated-code'] as const;

function fileOf(issue: Issue): string | null {
const file = issue.details.file;
return typeof file === 'string' ? file : null;
}

function filesWith(issues: Issue[], smell: string): Set<string> {
const files = new Set<string>();
for (const issue of issues) {
if (issue.smell !== smell) continue;
const file = fileOf(issue);
if (file !== null) files.add(file);
}
return files;
}

function needsExtractionIssue(file: string): Issue {
return {
smell: SMELL,
details: {
file,
line: 1,
column: 1,
message: 'File is both oversized and duplicated; extract the duplicated block into a shared module.',
source: 'composite:oversized-file+duplicated-code',
},
};
}

function combine(deps: Issue[]): Issue[] {
const duplicated = filesWith(deps, 'duplicated-code');
return [...filesWith(deps, 'oversized-file')].filter((file) => duplicated.has(file)).map(needsExtractionIssue);
}

// The composite sensor: a real multi sensor that reads oversized-file +
// duplicated-code from ctx.deps and emits needs-extraction for any file that has
// both (docs/architecture.md "Combinations"). All combination logic stays here
// in the sensor layer; the mapper never sees it.
export function needsExtractionSensor(): Sensor {
return {
id: SMELL,
produces: [SMELL],
dependsOn: [...INPUT_SMELLS],
run: (ctx) => Promise.resolve(combine(ctx.deps)),
};
}

// `replace` mode: once needs-extraction fires for a file, drop that file's two
// input smells so only the consolidated finding shows. Default (augment) keeps
// all three. Operates on the merged bag before the mapper runs.
export function applyReplaceMode(issues: Issue[], replace: boolean): Issue[] {
if (!replace) return issues;
const extracted = filesWith(issues, SMELL);
const suppressed = new Set<string>(INPUT_SMELLS);
return issues.filter((issue) => !(suppressed.has(issue.smell) && extracted.has(fileOf(issue) ?? '')));
}
Loading
Loading
0