8000 GitHub - ietf-tools/idnits: Library / CLI to inspect Internet-Draft documents for a variety of conditions to conform with IETF policies. · GitHub
[go: up one dir, main page]

Skip to content
ietf-toolsPublic

About

Library / CLI to inspect Internet-Draft documents for a variety of conditions to conform with IETF policies.

Topics

Resources

Code of conduct

Contributing

Security policy

Stars

22 stars

Watchers

7 watching

Forks

IDNITS

npm node-current License

CLI / Library to inspect Internet-Draft documents for a variety of conditions to conform with IETF policies.

⚠️ This branch is for the new JS-based idnits3. For the older shell-based idnits2, view the v2 branch instead.


Installation

  1. Install Node.js 20.x or later
  2. Install idnits using one of the methods:

Globally (recommended)

npm install -g @ietf-tools/idnits

In an existing npm project

npm install @ietf-tools/idnits

Without Installation

npx @ietf-tools/idnits <args>

Tip

This is only useful for quickly running the command once without installing. If you plan on using this tool regularly, you should install it globally instead.

CLI Usage

idnits [args] [--] <file path|url>

Everything after -- is the document, never an option, so idnits -- -draft.xml checks a file whose name starts with -.

Arguments Alias Description Default
--allowed-domains Comma-separated domains that, with their subdomains, get no domain finding (INVALID_DOMAIN_TLD, POSSIBLE_INVALID_TLD), e.g. --allowed-domains example.net,example.org. Replaces the default list. Can be repeated: the lists add up. --allowed-domains '' means no domain, so every domain is checked. Each domain is labels of letters, digits and hyphens (not at a label's start or end) joined by dots, a trailing dot allowed; anything else, an IPv4 address or an empty element (a.org,,b.org) included, is an error. ietf.org,rfc-editor.org
--datatracker-url Base URL of the Datatracker used for remote lookups. Overrides the IDNITS_DATATRACKER_URL environment variable. https://datatracker.ietf.org
--fail-on Exit with code 2 if there is at least one nit of the given severity or higher (errors > warnings > comments).
Accepted values: errors, warnings, comments
--filter -f Filter output to only certain severity types. Can be declared multiple times to filter multiple severity types.
Accepted values: errors, warnings, comments
--mode -m Validation mode: the stage of the document's life it is checked for, either normal (before RFC publication), forgive-checklist (RFC Editor conversion of an approved draft) or submission (Internet-Draft submission). See Validation modes.
Accepted shorthands: norm, n, f-c, fc, f, sub, s
normal
--no-color Disable colors in pretty output.
No effect in other output formats.
--no-progress Disable progress messages / animations in pretty output.
No effect in other output formats.
--offline No network access: skip every remote lookup (the document's own, and the status of the documents it references), and the checks that need them; one LOOKUPS_SKIPPED_OFFLINE comment per document names the checks that didn't run. See Remote lookups and --offline.
--output -o Output format, must be either pretty, simple, json or count pretty
--remote-timeout Seconds allowed for each remote lookup before it fails. Overrides the IDNITS_REMOTE_TIMEOUT environment variable. 10
--rfceditor-url Base URL of the RFC Editor site used for remote lookups. Overrides the IDNITS_RFCEDITOR_URL environment variable. https://www.rfc-editor.org
--solarized Use alternate colors for a solarized light theme terminal.
Only used with the pretty output format.
--year -y Expect the given year (4 digits) in the copyright line of a text document's boilerplate current year
--help -h Print the help text and exit
--version Print the version and exit

An option that takes one value (all but --allowed-domains and --filter) can be given more than once: the last one counts, so a script's defaults can be overridden (idnits -o json -o simple gives simple output). Each value given is still checked.

Validation modes

The mode says at which stage of its life the document is checked, not how strictly: normal before RFC publication, forgive-checklist when the RFC Editor converts an approved draft into an RFC, and submission when an Internet-Draft is uploaded to Datatracker. Validation modes explains each mode and lists every finding with its severity in each one.

Remote lookups

Some validations look up the document and the documents it references on the Datatracker (e.g. its latest revision and stream) and the RFC Editor site (e.g. the status of RFCs). --offline skips all of them: see Remote lookups and --offline. The lookups of a document are made once per run, and nothing looked up is kept for the next run, so a lookup that failed is tried again. They are made with a User-Agent header naming idnits3 and its version (e.g. idnits3/3.0.0 (+https://github.com/ietf-tools/idnits)).

The services they are sent to, and the time each lookup is allowed, can be changed with these environment variables, or the matching flags, which take precedence. URLs must be http or https URLs, a trailing slash is ignored. The timeout is a positive number of seconds; it covers the whole request, response included.

Environment variable Flag Default
IDNITS_DATATRACKER_URL --datatracker-url https://datatracker.ietf.org
IDNITS_RFCEDITOR_URL --rfceditor-url https://www.rfc-editor.org
IDNITS_REMOTE_TIMEOUT --remote-timeout 10

A lookup that times out fails like one to a service that can't be reached: a warning naming the host is printed once, and the validations carry on without its result. If the Downref Registry can't be fetched (an error or a timeout), downrefs are reported as DOWNREF_TO_LOWER_STATUS_UNCHECKED without telling whether they are listed in it, and a DOWNREF_REGISTRY_UNAVAILABLE comment names the reason. A failed fetch of the registry is not retried during the run, so its checks don't each wait for it; the next run tries again.

The environment variables also apply when idnits is used as a library in Node.js, where each run can also be given its own services and timeout (see Remote lookup options). In the browser, lookups go through the /idnits3/api/ endpoints of the page's server instead.

Note

Datatracker deployments: when idnits runs next to the Datatracker (e.g. in its celery containers), set IDNITS_DATATRACKER_URL to the in-cluster Datatracker web service (e.g. http://datatracker:8000), so that the lookups made for each submission stay in the cluster instead of going out through the public internet and its CDN. IDNITS_REMOTE_TIMEOUT bounds how long a slow or unresponsive Datatracker can hold up the check of a submission.

Exit codes

Code Meaning
0 Validation completed (whatever nits were found, unless --fail-on is used)
1 Validation could not be completed (e.g. the document could not be read or parsed). Datatracker treats any nonzero code this way.
2 Validation completed and found at least one nit at or above the --fail-on severity

Library Usage

Note

The library documentation is a work in progress.

Ensure you installed the library locally to your project (npm install @ietf-tools/idnits).

Simple Validation Run

Use the checkNits() method to quickly run all the validation checks and return a results array.

import { checkNits } from '@ietf-tools/idnits'

const documentRawBuffer = ...
const documentFileName = 'draft-ietf-abcd-efgh-01.xml'

const results = await checkNits(documentRawBuffer, documentFileName)

For XML documents, results that have a path (e.g. rfc.middle.section[2].section[1].t[0]) also get a human-readable label describing that location (e.g. Section 3.2 (Security Considerations), paragraph 1). The CLI shows it next to the path, and includes it as label in the json output. They also get the source line and column of that element in lines (e.g. [{ line: 42, pos: 7 }]), unless the check already gave one.

Progress

Pass an onProgress function to follow the run, as the CLI does for its progress display. It is called with one event at a time, where group and task are the keys of the groups and tasks that getAllValidations() returns:

Event When
{ type: 'groupStarted', group } The tasks of a group are about to run
{ type: 'groupSkipped', group } The condition of a group isn't met, so its tasks don't run
{ type: 'taskDone', group, task } A task has returned
{ type: 'taskFailed', group, task, error } A task has thrown, which stops the run (only decoding and parsing can: the other checks give a CHECK_FAILED finding instead)
const results = await checkNits(documentRawBuffer, documentFileName, {
  onProgress: (event) => {
    console.log(event.type, event.group, event.task)
  }
})

Remote lookup options

Pass datatrackerUrl, rfcEditorUrl or remoteTimeout (in seconds) to send the lookups of one run to other services, or give them another timeout. They take precedence, for that run only, over setRemoteUrls(), setRemoteTimeout() and the environment variables of Remote lookups, so runs made at once in the same process can each use their own. checkNits() throws before checking anything if one of them isn't valid.

const results = await checkNits(documentRawBuffer, documentFileName, {
  datatrackerUrl: 'http://datatracker:8000',
  remoteTimeout: 5
})

Task Runner

You can implement your own task runner to have full control over how the validations are executed. The getAllValidations() method returns a list of all validations that should be run.

import { finalizeFindings, getAllValidations } from '@ietf-tools/idnits'

const ext = filename.endsWith('.xml') ? 'xml' : 'txt'
const result = []
const ctx = {
  raw,
  filename,
  options: {
    allowedDomains,
    mode,
    offline,
    year,
    // -> Optional: services and timeout of the remote lookups of this run
    datatrackerUrl,
    rfcEditorUrl,
    remoteTimeout
  }
}

const validations = getAllValidations(ext)

for (const valGroup of validations) {
  // Skip validation group if condition is not met
  if (valGroup.condition && !valGroup.condition(ctx)) {
    continue
  }

  // Run validations in parallel when possible
  if (valGroup.concurrent) {
    const valGroupResult = await Promise.all(valGroup.tasks.map(valTask => valTask.task(ctx)))
    for (const taskResult of valGroupResult) {
      if (Array.isArray(taskResult)) {
        result.push(...taskResult)
      }
    }
  } else {
    // Run validations sequentially otherwise
    for (const valTask of valGroup.tasks) {
      const taskResult = await valTask.task(ctx)
      if (!valTask.isVoid && Array.isArray(taskResult)) {
        result.push(...taskResult)
      }
    }
  }
}

// Add human-readable locations and source lines to XML results, as checkNits() does
finalizeFindings(result, ctx.doc)

return result

Tests

Tests are made using the Vitest library and are located under the tests directory.

You can run the suite of tests using:

# Make sure you installed dependencies first:
npm install

# Run the tests
npm test

Code coverage is expected to reach 100%. Ensure this is still the case when making edits / adding new functionality.

Development

  1. Clone the project
  2. Run npm install
  3. Run the CLI: (replacing <args> and <file path|url> with the desired flags + file path)
    node cli.js <args> <file path|url>
    

About

Library / CLI to inspect Internet-Draft documents for a variety of conditions to conform with IETF policies.

Topics

Resources

Code of conduct

Contributing

Security policy

Stars

22 stars

Watchers

7 watching

Forks

Releases

Used by

Contributors

Languages

0