All notable changes to this project will be documented in this file.
The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.
- New public python APIs:
identify_paths,identify_path,identify_bytes. - The APIs now return a
MagikaResultobject. - When the model's prediction has low confidence and we return a generic content type, print anyways (with a disclaimer) the model's best guess.
- Updated description for "unknown" to "Unknown binary data".
- Magika CLI now defaults to "high-confidence" mode. "default" mode is now called "medium-confidence".
- Magika CLI
-p/--output-probabilityhas been renamed to-s/--output-scorefor consistency. - Default model is now called
standard_v1. - Major refactoring and clean up.
- Various improvements and clean ups.
- Update model to dense_v4_top_20230910.
- Package now contains the model itself.
- Support reading from stdin:
$ cat <path> | magika -$ curl <url> | magika -
- Change how we deal with padding, using 256 instead of 0. This boosts precision.
- "symlink" output label has been renamed to "symlinktext" to better reflect its nature.
- New
--prediction-modeCLI option to indicate which confidence is required for the predictions. We support three modes:best-guess,default,high-confidence. - Support for directories and symlinks similarly to
file. - Adapt
-r/--recursiveCLI option to be compatible with the new way magika handles directories. - Add special handling for small files.
- Magika does not crash anymore when scanning files with permission issues. It now returns "permission_error".
- Do not resolve file paths (i.e., relative paths remain relative).
- Add --no-dereference CLI option: by default symlinks are dereferenced.
this option makes magika not dereferencing symlinks. This is what
filedoes. - Clean up and many bug fixes.
- Removed warnings when using MIME type and compatibility mode.
- By default, magika now outputs a human-readable output.
- Add
-l/--labelCLI option to output a stable, content type label. - JSON/JSONL output now shows all metadata about a given content type.
- Add metadata about magic and description for each relevant content type.
- Logs are now printed to stderr, not stdout.
- Add
--generate-reportCLI option to output a JSON report that can be useful for debugging and reporting feedback. - Be more flexible with the required python version (now we require "^3.8" instead of "^3.8,<3.11")
- Show a descriptive error in case magika can't find any file to scan (instead of silently exiting).
- If the prediction score is higher than a given threshold (0.95), consider it regardless of the per-content-type threshold.
- Output format is back being just
<content type>; group is displayed only when showing metadata. - Update metadata of some content types.
- Several small bug fixes.
- Input files are now processed in multiple small batches, instead of one big batch.
- Per-content-type threshold based on the 0.005 quantile for recall.
- MIME type and "group" metadata for all content types.
- Introduce basic support for compatibility mode.
-c/--compatibility-modeCLI option to enable compatibility mode.--no-colorsCLI option to disable colors.-b/--batch-sizeCLI option to specify the batch size.--guess/--output-highest-probabilityCLI option to output the content type with the highest probability regardless of its probability score.--versionCLI option to print Magika's version.
- Output follows the
<group>::<content type>format. - Probability score is not shown by default; enable with
-p. - Output is colored according to the file content type's group.
- Remove dependency from richlogger, add a much simpler logger.
- First release.