8000 GitHub - MrTaherAmine/evidenceveil: Local-first privacy engineering for cybersecurity evidence sharing — sanitize, pseudonymize, audit, and package incident telemetry without exposing sensitive context. · GitHub
[go: up one dir, main page]

Skip to content

About

Local-first privacy engineering for cybersecurity evidence sharing — sanitize, pseudonymize, audit, and package incident telemetry without exposing sensitive context.

Topics

Resources

Code of conduct

Contributing

Security policy

Stars

1 star

Watchers

0 watching

Forks

EvidenceVeil — Share incident data without exposing the incident

Share incident data without exposing the incident.
Local-first privacy engineering for cybersecurity evidence.

Latest release Python 3.11–3.14 Apache-2.0 Local-first, no telemetry

CI Security CodeQL Build


What is EvidenceVeil?

EvidenceVeil helps security teams share incident evidence with less unnecessary exposure.

Logs, exports and investigation artifacts often contain much more than the information a recipient actually needs: usernames, email addresses, internal IPs, hostnames, cloud identifiers, tokens, repository names, timestamps and other sensitive context.

EvidenceVeil gives you a controlled workflow to discover, transform, audit and package that evidence locally before it leaves your environment.

Important

EvidenceVeil reduces disclosure risk. It does not guarantee anonymity, determine legal anonymisation, or automatically decide that a dataset is safe to publish.

Built for

  • SOC and CSIRT teams
  • DFIR and incident-response practitioners
  • Security researchers and detection engineers
  • Privacy and security engineering teams
  • Organizations sharing evidence with vendors, partners or other responders

How it works

flowchart LR
    A[Evidence copy] --> B[Discover]
    B --> C[Apply policy]
    C --> D[Sanitize / pseudonymize]
    D --> E[Audit residual risk]
    E --> F[Verify + package]
    D -. optional reversible mapping .-> V[Encrypted .evlt vault]
    V -. stored separately .-> X[Authorized restoration]
Loading

EvidenceVeil never needs to modify the original evidence in place. Reversible mappings, when used, live in a separate encrypted vault rather than inside the shareable bundle.


Quick start

1. Install

Download the wheel from the latest GitHub release, then:

python -m pip install evidenceveil-1.0.2-py3-none-any.whl

Or install from a clone:

git clone https://github.com/MrTaherAmine/evidenceveil.git
cd evidenceveil
python -m pip install .

Check the environment:

evidenceveil doctor

2. Discover what is sensitive

evidenceveil discover ./evidence-export

3. Sanitize with a built-in policy

export EVIDENCEVEIL_VAULT_PASSPHRASE='use-a-strong-secret'

evidenceveil sanitize ./evidence-export \
  --policy vendor-support \
  --new-key \
  --vault ../secure-vaults/case.evlt \
  --output ./evidenceveil-output \
  --report

4. Audit and verify

evidenceveil audit ./evidenceveil-output
evidenceveil verify ./evidenceveil-output

Need authorized restoration later?

evidenceveil restore ./evidenceveil-output/sanitized \
  --vault ../secure-vaults/case.evlt \
  --output ./restored-evidence

Redaction, dropping and other irreversible transformations cannot be restored.


What you get

evidenceveil-output/
├── sanitized/          transformed evidence for controlled sharing
├── provenance/         risk, utility and transformation records
├── reports/            self-contained offline reports
├── manifest.json       hashes, policy, counts and identifiers
└── checksums.sha256    integrity verification

case.evlt                encrypted mapping vault — stored separately

Core capabilities

Capability What it does
Sensitive-data discovery Finds security-relevant identifiers and secret-like fields across supported evidence formats.
Policy-as-code Makes sharing rules explicit, reviewable and repeatable.
Pseudonymization Preserves useful relationships without exposing original values.
Redaction & transformation Supports masking, dropping, generalization, bucketing, shifting, synthesis and more.
Encrypted reversible vaults Keeps optional restoration mappings outside the shareable package.
Residual-risk audit Flags identifier-like remnants and untransformed free text for human review.
Integrity verification Uses hashes and manifests to detect package modification.
Offline reporting Produces self-contained HTML reports with no external assets or analytics.
Utility checks Helps confirm that selected structural properties survive transformation.
Secure-by-default handling Refuses in-place output, applies archive safety checks and neutralizes CSV formula injection.

Built-in sharing policies

Choose a sensible starting point and adapt it to your context:

Policy Designed for
vendor-support Sharing evidence with a known support/vendor recipient
cross-organization-ir Coordinated incident response between organizations
public-research-strict More restrictive public research release
internal-handoff Least-privilege internal sharing
reversible-investigation Authorized workflows where selected mappings must be recoverable
training-dataset Reusable training fixtures
detection-fixture Detection-engineering test data

See Policy Authoring for custom policies.


Transformations

EvidenceVeil supports:

keep · drop · redact · mask · tokenize · pseudonymize · generalize · bucket · shift · truncate · replace · synthesize · hmac · preserve · quarantine

Deterministic keyed transformations can preserve selected relationships such as repeated identities, hosts, sessions and temporal ordering without relying on predictable unkeyed hashes.


Format support

Ready in v1.x

  • Text and application logs
  • Syslog-like text
  • CEF / LEEF
  • JSON
  • JSONL / NDJSON
  • CSV / TSV
  • ECS / OCSF / OpenTelemetry-style JSON
  • STIX 2.1 JSON bundles
  • Gzip text input

Recognized, but not parsed/sanitized in v1.x

  • EVTX
  • Parquet

Archive safety primitives are included, but direct CLI sanitization of ZIP/TAR archives is not claimed in v1.x. See Format Support for the exact compatibility matrix.


Security model

EvidenceVeil is intentionally local-first. Core processing does not require uploading evidence to an external service.

Security-focused defaults include:

  • no in-place sanitization
  • symlink refusal by default
  • no execution of evidence content
  • safe YAML loading and typed policy validation
  • archive traversal and link protections
  • terminal-control sanitization
  • CSV formula-injection defense
  • Argon2id for passphrase-based vault key derivation
  • ChaCha20-Poly1305 authenticated encryption
  • domain-separated HMAC-SHA-256 for deterministic token derivation
  • separate storage of reversible mapping vaults

For the full design, read the Threat Model, Cryptography, Privacy Model and Limitations.


Residual-risk decisions

evidenceveil audit deliberately uses conservative release states:

  • blocked
  • review-required
  • eligible-for-controlled-review

EvidenceVeil never turns a numerical score into an unconditional “safe to share” decision.


CLI at a glance

evidenceveil version
evidenceveil about
evidenceveil doctor

evidenceveil discover INPUT
evidenceveil plan INPUT --policy POLICY
evidenceveil sanitize INPUT --policy POLICY --output OUTPUT
evidenceveil audit INPUT
evidenceveil verify BUNDLE
evidenceveil restore INPUT --vault VAULT --output OUTPUT
evidenceveil diff ORIGINAL SANITIZED

evidenceveil policies list
evidenceveil policies show POLICY
evidenceveil policies validate PATH
evidenceveil formats list
evidenceveil plugins list

Current validation

EvidenceVeil v1.0.2 was validated locally on macOS 26.6.2 with Python 3.14.7, including:

  • 51 automated tests passing
  • 92.56% branch-aware coverage
  • Ruff lint and formatting checks passing
  • mypy strict type checking passing
  • pip check passing
  • pip-audit reporting no known vulnerable third-party dependencies in the validated environment
  • Bandit security analysis passing
  • sanitization-bypass regression coverage
  • exact Issue #2 collision regression coverage
  • 1,000-public-IPv4 deterministic round-trip stress validation
  • reversed-order collision determinism and cross-semantic mapping isolation validation
  • non-cascading restoration regression coverage
  • archive detection regression coverage
  • wheel and source-distribution builds passing twine check

Current main is additionally validated automatically by GitHub Actions across Linux, macOS and Windows with Python 3.11, 3.12, 3.13 and 3.14, together with dedicated Security, CodeQL and Build workflows.

Documentation

Looking for the details? Start here:


Contributing

Ideas, new evidence formats, policy improvements, edge cases, bug reports and security review are welcome.

Read CONTRIBUTING.md before opening a pull request. For security issues, please follow SECURITY.md rather than disclosing vulnerabilities publicly.

See the Roadmap for planned work.


License

EvidenceVeil is released under the Apache License 2.0. See LICENSE.


EvidenceVeil
Created and maintained by Taher Amine ELHOUARI
taheramine.org · GitHub @MrTaherAmine

About

Local-first privacy engineering for cybersecurity evidence sharing — sanitize, pseudonymize, audit, and package incident telemetry without exposing sensitive context.

Topics

Resources

Code of conduct

Contributing

Security policy

Stars

1 star

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages

0