Repository navigation
Expand file tree
/
Copy pathCHANGES.txt
More file actions
4448 lines (3192 loc) · 180 KB
/
Copy pathCHANGES.txt
File metadata and controls
4448 lines (3192 loc) · 180 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
Release 4.1.0 - unreleased
* A missing OOXML relationship target no longer aborts the whole file:
the threaded-comment and person lookups in xlsx and the page lookups in
vsdx went straight to POI's getRelatedPart, whose unchecked
IllegalArgumentException surfaced as "Error creating OOXML extractor" and
dropped the text already extracted. They route through
safeGetRelatedPart, as branch_3x already did (TIKA-4879).
* RawTiffDetector rejects a BigTIFF directory offset near Long.MAX_VALUE
instead of letting the bounds check overflow. Adding the entry-count
size to such an offset wrapped negative and read as "already in the
prefix", so a 16-byte file threw ArrayIndexOutOfBoundsException out of
Detector.detect, which CompositeDetector does not catch: detection
failed for the document and the remaining detectors never ran. Raw
detection runs on every stream, so this was reachable from every entry
point (TIKA-4861).
* Entries of ODF, EPUB, GeoGebra, WACZ, XLZ and iWork containers, mbox
messages and the AppleSingle data fork are re-opened from their
container on rewind instead of cached: digesting rewinds every
embedded document, and the cached copy cost heap for the whole entry
and, past the cache budget or the 1 MB floor, a temp file. AppleSingle
no longer spools its data fork to a temp file on every parse, and
GeoGebra no longer spools every embedded picture to detect it
(TIKA-4878).
* A declared Content-Length is no longer treated as a measurement: the
zip-bomb ratio counts only measured input bytes (a container-declared
size on an embedded document could inflate its denominator), and a
re-openable source no longer reserves cache budget or sizes its buffer
from the declared length (a lying one could push a small payload to
disk or churn the shared budget). Neither is in a release: the
exposure arrived with TIKA-4868 and TIKA-4873 (TIKA-4878).
* Embedded objects in Office documents are re-opened from their container
instead of cached: every OOXML part (pictures, media, attachments), the
OLE 2.0 package inside an OOXML part, the CONTENTS entry of an OLE 2.0
object in a binary Office file, embedded objects in .ppt, XPS page
images and Word EMF icons. Digesting an embedded document rewinds it;
the cached copy that made possible cost heap for the whole object and,
past the cache budget or the 1 MB floor, a temp file. The container
hands the bytes back on demand, so neither is needed (TIKA-4878).
* PDF attachments, PDF XMP packets, 3D on-instantiate scripts and PST
attachments are re-opened from their document instead of cached when
the embedded-document extractor rewinds them (digesting does, for every
embedded document). The cached copy cost heap for the whole attachment
and, past the cache budget or the 1 MB floor, a temp file; PDFBox and
java-libpst hand the bytes back on demand (TIKA-4878).
* tika-server: named configuration presets (TIKA-4856).
* Temp files follow -Djava.io.tmpdir on the parent JVM (Tika, its
libraries, and forks all honor it); TikaLoader fails at config load
if it is unusable. pipes.tempDirectory is deprecated for removal in
5.0: it only covered the forks. Do not use tmpfs: spool size is
bounded by input, and an orphaned fork dir pins RAM (TIKA-4877).
* A fork whose parent dies deletes its own temp dir; the parent
surfaces a fork's hs_err log before every delete. Failure-path temp
file leaks fixed in PDFBoxRenderer, PopplerRenderer, truncated RTF,
and MarianTranslator (TIKA-4877).
* tika-eval Profile/Compare speedups: single-pass URL/mail stripping
replaces the bounded regexes in langdetect preprocessing (same output,
17-290x faster on web text), the default H2 db URL drops MVStore chunk
retention and sizes the page cache at a quarter of the heap clamped to
[64MB, 1GB] (override with -Dtika.eval.h2.cacheSizeKb=<kb>), and the
status log adds a last-interval docs-per-sec rate next to the cumulative
average (TIKA-4875).
* New "content-enrichers" config list (TIKA-4872): select the OCR engine
("tesseract-ocr-parser", "tess4j-parser", "openai-vlm-parser", ...) by
name instead of by classpath registration of the image/ocr-* pseudo
media types. Enrichers advertise real media types (legacy engines that
still advertise image/ocr-* are mapped to the real type, so all are
nameable) and are invoked by the image and PDF parsers rather than
dispatched to by the composite, so an enricher no longer displaces the
parser registered for the same type. Enricher selection uses the
detected media type, captured before a parser can refine Content-Type.
Every enricher matching a media type runs, in config order (e.g. an
OCR engine then a VLM tagger for the same image), best-effort: one
enricher's failure does not stop the others and is still reported;
timeouts abort the chain. The list is authoritative: a media type no
configured enricher matches gets no enrichment -- never a classpath
engine that was not named -- and a named engine that reports no media
types at load (missing binary, unreachable inference server) fails
config load instead of going silently inert. With no
"content-enrichers" configured, the legacy ocr-* dispatch applies
unchanged; a WARN at config load now names colliding OCR engines and
the winner. TesseractOCRParser's
component name is pinned as "tesseract-ocr-parser".
* Inference/OCR hardening (TIKA-4871): OpenAIVLMParser no longer
auto-registers via SPI, matching its Claude/Gemini siblings; select
it by name ("openai-vlm-parser") in config. Per-request parse-context
config for the embedding filters now works and is validated:
{"openai-embedding-filter": {"skipEmbedding": true}} (likewise
"jina-embedding-filter") merges over the server config, and
baseUrl/apiKey/model may not be changed at runtime. The embedding
filters release their HTTP client resources on close(). Inline PDF
page OCR now accumulates tk:chunks from every OCR'd page onto the
parent document instead of keeping only the first page's.
* Placeholder streams -- the empty stand-ins parsers hand parseEmbedded
for content that is never extracted -- report an unknown length rather
than their own zero, and the macro-failure entry is registered without
parsing its sentinel (TIKA-4874).
* TikaInputStream.hasReliableLength() distinguishes measured lengths
from declared Content-Length hints, and one-shot streams now carry a
declared length without spooling; detection sizes its magic read only
from measured lengths, so a lying declared length can no longer
truncate it (TIKA-4868).
* ParseContext entries holding per-parse runtime state (ParseRecord,
ParseTimeout, the parser-map cache) are skipped during serialization
instead of failing as unregistered components (TIKA-4868).
* Mojibuster's adaptive probe strips incrementally instead of
re-stripping the whole buffer on every read (quadratic on tag-heavy
pages); JunkDetector's Unicode block lookup uses a precomputed BMP
table. Output unchanged (TIKA-4868).
* Markdown rendering is another ~8x faster on large documents: a custom
Text-node renderer emits unescaped spans in bulk instead of
commonmark's per-character escape-check-and-append. Byte-identical
output, guarded by a fast-vs-stock differential test (TIKA-4868).
* ZipParser no longer re-decompresses an entry on every rewi
10BC8
nd when the
entry uses a legacy compression method (implode, shrink, bzip2, ...):
such entries replay from the budgeted cache instead of re-opening.
An imploded 606KB zip drops from 610ms to 102ms (TIKA-4868).
* More detection/dispatch savings: the message/rfc822 priority-45 magic
is gated behind a lossless ':' scan of the first 30 bytes;
CompositeParser caches the built type->parser map in the ParseContext
so embedded documents reuse the container's map; MagicDetector
precomputes a per-pattern first-byte table. Adds a RESOURCE_TIMING
log, silenced by default in the shipped log4j2 configs (TIKA-4868).
* WordExtractor (.doc) cleans each character run and tests paragraph
blankness in single passes; ToMarkdownContentHandler collapses line
breaks copy-free for clean runs. ~39% off a text-heavy 2MB .doc
(TIKA-4868).
* tika-server's raw-output endpoints (/tika, /tika/text, ...) carry the
extracted content as raw UTF-8 bytes from the pipes worker to the HTTP
response instead of a Smile-encoded string (9MB text: 115ms -> 75ms).
Opt-in via the new content-bytes-config parse-context component, which
moves CONTENT_ONLY passback content out of tk:content into
EmitData.getContentBytes(); results routed to a regular Emitter get
the content restored to the metadata (TIKA-4868).
* Detection hot-path cleanups: MagicMatch resolves its detector via
double-checked locking; glob patterns are compiled once at
registration; MimeTypes.forName reads a ConcurrentHashMap (fixing an
unsynchronized-read race) and indexes normalized keys; resource names
containing spaces skip the URI-parse-by-exception; the magic-header
buffer is sized by the stream's measured length instead of a fixed
64KB; the Adobe Illustrator ranged regex is gated behind a literal
scan. Detection results unchanged (TIKA-4868).
* Magic detection is ~35% faster on unmatched (e.g. plain-text) input:
range scans find first-byte candidates before running the full
masked/case-folded compare (TIKA-4868).
* CSVSniffer reads its detection window once into a shared buffer and
runs every delimiter hypothesis against it; windows with no delimiter
and no quote skip the scan outright. Results unchanged (TIKA-4868).
* Pipes workers no longer stall between pre-parse and parse waiting
for the client to acknowledge the intermediate-result frame; the
ACK round trip now overlaps the parse. Adds per-request timing logs
on org.apache.tika.pipes.timing.*, silenced by default in the shipped
log4j2 configs; raise that logger to info to enable (TIKA-4868).
* DefaultDetector honors CONTENT_TYPE_USER_OVERRIDE and
CONTENT_TYPE_PARSER_OVERRIDE before running magic detection, matching
CompositeDetector's contract. Removes the second full magic scan every
pipes parse paid per document. Compat note: with either override set,
DefaultDetector no longer lets a more specific magic result overrule
it; parts whose parser declares a type from container headers (e.g.
inline text/* mail parts) now report the declared type, and
CONTENT_TYPE_MAGIC_DETECTED is not recorded when an override short
circuits detection (TIKA-4868).
* Markdown output is ~4x faster on large documents:
ToMarkdownContentHandler now buffers the commonmark renderer's
per-character writes instead of paying the synchronized
Writer.write(int) cost for every character (TIKA-4868).
* Embedded documents carry their size: ParsingEmbeddedDocumentExtractor
sets Content-Length from the stream where the stream knows it and the
parser did not say, which never spools to measure one, and the raw
camera previews and the audio cover art set the length they read from
the file (TIKA-4873).
* AVIF images are parsed rather than only detected: HeifParser accepts
image/avif, which is the same ISO-BMFF container, so dimensions, EXIF
and XMP come out of it the way they do for HEIC (TIKA-4870).
* The video of a Google/Android motion photo, appended after the image and
described by the Motion Photo or MicroVideo XMP, is emitted as an
ATTACHMENT embedded document named after what the file declares it to
be. Nothing is emitted, and nothing is recorded, when the declared
length does not fit the file or the bytes there are not recognized,
which is what sharing a motion photo out of a gallery leaves behind
(TIKA-4869).
* Raster previews for the vector thumbnails of Office documents: the new
poi-metafile-renderer draws EMF and WMF images through POI (a PNG of
a configurable width; Word's bitmap-in-WMF thumbnails from the bitmap
directly), EMFParser and WMFParser are RenderingParsers that emit the
rendering as a RENDERING embedded document with "emf-parser" /
"wmf-parser": {"renderImage": true, "renderWidth": 800}, off by
default and restrictable to e.g. THUMBNAIL embedded documents with
"renderOnlyEmbeddedResourceTypes", and OfficeParser emits the
SummaryInformation thumbnail of the OLE2 formats (a WMF) as a THUMBNAIL
embedded document, as the OOXML parsers do with the docProps thumbnail,
switchable with "office-parser": {"extractThumbnail": false}
(TIKA-4855).
* Add "exception-reporting" parse-context config to redact and bound
exception text in metadata, tika-server error bodies and pipes/grpc
messages; FileSystemEmitter writes atomically (TIKA-4848).
Compat notes: a truncated TSD envelope now records its read failure
under tk:exception:embedded-stream-exception rather than
tk:exception:embedded-exception; recordException and
recordEmbeddedStreamException no longer strip a bare TikaException
wrapper, so the first line of tk:exception:* values may name the
wrapper (affects consumers keyed on that line, e.g. eval cause
counts across the 4.1 boundary).
* Audio cover art is emitted as a THUMBNAIL embedded document, like the
preview image of the document container formats: the front cover (ID3
APIC and FLAC/Vorbis picture type 3), else the first picture of type
"Other" or unknown, else the first picture, and the first covr image
of an MP4. Further pictures
stay INLINE. Clients that looked for cover art as INLINE need to
accept THUMBNAIL as well (TIKA-4850).
* tika-grpc resolves its plugin-roots fallback against the install
layout via DefaultPluginsDir instead of a working-directory-relative
pf4j default, and a WARN names the resolved directory when no plugins
directory exists (TIKA-4865).
* The tika-server full and tika-grpc Docker images install fonts-noto-cjk:
without any CJK face, PDFs using non-embedded CJK fonts render (and OCR)
as .notdef boxes in every renderer, even though the images ship Japanese
tesseract data (TIKA-4866).
* The default plugins directory is resolved against the install layout
(next to the jar, or next to its lib/ directory) and always as an
absolute path, shared by tika-server, PipesForkParser and the async
CLI; it no longer depends on the working directory (TIKA-4864).
* tika-server error bodies (the 422/500 exception mapper, /meta/{field})
now honor the exception-reporting policy; /meta/{field} returns the
already-formatted container exception instead of re-wrapping it with
server frames. Completes TIKA-4848 (TIKA-4848).
* The exception-reporting policy now also governs the messages a pipes
worker returns (fetch/emit/crash) and the container exception it
records; part of TIKA-4848 step 3 (TIKA-4848).
* Allow image compression settings in PDFBox-based renderer (TIKA-4862).
* The tika-server full and tika-grpc Docker images set OMP_THREAD_LIMIT=1:
to avoid oversubscribing the CPU under forked parse workers (TIKA-4863).
* embedded-limits maxDepth counts embedding levels again instead of the
parsers a parse passes through; with AutoDetectParser over DefaultParser
every value above 1 used to stop one level early (TIKA-4857).
* GeoGebraParser emits the icon of a tool (*.ggt, the macro's iconFile)
as its THUMBNAIL embedded document; tool files have no thumbnail of
their own (TIKA-4831).
* Enum values in JSON configuration are matched case-insensitively, so
"no_ocr" works as well as "NO_OCR"; the server docs used the lower-case
form in their examples (TIKA-4859).
* The preview image of iWork '09 packages (QuickLook/Thumbnail.jpg) and of
iWork '18 packages (preview.jpg) is emitted as a THUMBNAIL embedded
document, as it already was for iWork '13 (TIKA-4854).
* Raw camera formats are detected by content: RawTiffDetector tells
Nikon NEF/NRW, Pentax PEF/PTX, Sony ARW/SRF/SR2, Samsung SRW and Adobe
DNG from a plain TIFF by their image directory (DNGVersion, the vendor
Compression codes, or a CFA/LinearRaw image plus Make), and Fuji RAF,
Panasonic RW2, Minolta MRW and the remaining Olympus ORF byte orders
get magic entries. Streams without a file name used to be image/tiff.
image/x-raw-samsung (*.srw) is new and parsed by RawTiffParser
(TIKA-4861).
* DWGReadParser emits the drawing's THUMBNAILIMAGE as a THUMBNAIL embedded
document instead of INLINE (TIKA-4853).
* EpubParser emits the cover image named by the OPF (the EPUB 3
cover-image manifest property, or the EPUB 2 cover meta) as a THUMBNAIL
embedded document (TIKA-4852).
* RawTiffParser marks only the largest embedded JPEG preview as the
THUMBNAIL embedded document; the smaller previews of the same image are
INLINE images named image-N.jpg. Previously every preview was a
THUMBNAIL, so a client had to compare them to find the representative
one (TIKA-4851).
* tika-eval: Profile/Compare accept the batch run's jsonl crash ledger
(--pipesReport, -pa/-pb) and a run-info json (--runInfo, -ra/-rb), and
read both from <extracts>/.run-info/ by default (refusing an ambiguous
dir). containers gains pipes_status/pipes_message; a new run_info table
records eval and batch provenance; reports and
summary.md classify NO_EXTRACT_FILE by ledger status (CRASH, the raw
status, NO_PIPES_RECORD, BATCH_WITHOUT_LEDGER, NO_PIPES_REPORT_SUPPLIED).
Report on a db from an earlier tika-eval skips the reports it cannot run
instead of aborting (TIKA-4847).
* New file-system-jsonl-reporter pipes reporter (TIKA-4846).
* Stop spooling OLE2 objects whose header over-reserves BAT capacity
(TIKA-4845).
* Add Micrometer reporting and opt-in endpoint for tika-server (TIKA-4839).
* Improve spooling/decrease number of spills to disk (TIKA-4835).
* Fixed a bug that made per-request (parse-context) configuration unusable
for parsers that lock some config fields against caller modification --
Tess4J, the VLM parsers and the OpenAI image-embedding parser. Any such
config threw, including an empty one: the defaults were deep-copied
through their own setters, which the runtime config overrides to reject
caller input, so the copy tripped the parser's own guards before the
caller's JSON was read. Locked fields are still rejected when a caller
actually sets them. Configuration supplied at initialization time (the
"parsers" section) was never affected (TIKA-4843).
* OOXML parsers flag package parts that are unreachable through the OPC
relationship graph: msoffice:has-unreferenced-parts (boolean) and
msoffice:unreferenced-part-names. Purely structural (no bytes are
inspected; content types come from [Content_Types].xml by extension), so
expect false positives from tools that leave orphan parts behind. A hiding
place a raw-ZIP scanner can still see, not a statement about what Tika
parsed. Applies to Word, Excel, PowerPoint and Visio OOXML (including
macro-enabled variants); XPS links content by markup rather than
relationships and is not checked (TIKA-4837).
* Shared pipes server (useSharedServer: true, not the default): a client whose
in-flight parse was killed by another client's restart could restart the
healthy replacement. ensureRunning holds its lock across the whole fork, so
siblings cannot report a dead worker until after the replacement is up, and
the pending-restart flag carried no process identity -- so a report about
the process that just died was applied to its successor, which was then
destroyed and re-forked. One worker death produced two restarts and a second
round of destroyed in-flight work; under sustained concurrent load it
sustained itself at one spurious restart per round, appearing as periodic
unexplained worker churn and intermittent parse failures that succeed on
retry. Each fork now carries a generation that clients capture when they
connect and hand back with every report, and reports about a superseded
process are dropped. Also fixed in shared mode: ensureRunning could fork a
replacement after shutdown() that nothing owned and nothing would ever
destroy, and an interrupt during process teardown left the process handle
pointing at a killed process and leaked the temp directory. Affects 4.0.0
and earlier (TIKA-4844).
* tika-pipes: the cache memory budget (how much rewindable content a forked
worker keeps in memory before spilling to disk; new since 4.0.0, which had
no budget at all) defaults to a quarter of the fork's heap, so raising
-Xmx raises it. It is one pool per forked JVM shared by all of its threads.
-Dtika.pipes.cacheMemoryBudgetBytes in forkedJvmArgs overrides it (below
the quarter-heap ceiling; <=0 disables); the fork logs the value and its
source at startup. TikaInputStream.hasFile() now also reports content the
stream cache spilled on its own, not only content a getPath() call put on
disk; note getPath() may still have to drain the rest of the source into
that file. TikaInputStream.toString() no longer forces a spill, so logging
or debugger-inspecting a stream is side-effect-free.
TikaInputStream.inMemoryContent(channel) gives a zero-copy read-only view
of cached content for consumers that need random access. Digester
gains digestSink(), a DigestSink that digests as it is written; nothing is
written to the metadata unless the producer calls commit(), so any failure
-- exception, Error, or a producer that closes the sink itself -- publishes
no digest rather than a digest of the bytes that happened to arrive. A
translator that claims a stream and writes nothing likewise publishes
nothing: embedded PST mail items, whose translator is still a stub, no
longer carry the digest of zero bytes (the same value for every one of
them) and instead carry no digest at all. DigestHelper uses it for
translated embedded streams, which no longer touch a temp file when the
digester implements digestSink (all of Tika's do; one that only implements
digest() still buffers).
TemporaryResources.closeAll(Closeable...) closes every argument even when
one throws unchecked; TemporaryResources, CachingSource, CachingInputStream
and CompositeDigester use it (TIKA-4835).
* Documentation: corrected a batch of pages and javadocs that contradicted
the code. Notably: the ES/OpenSearch attachmentStrategy has no default
(unset means embedded documents get neither the parent field nor the
parent/child relation); Kafka's connectionsMaxIdleMs is passed to the
producer, not ignored; jdbc queryTimeoutSeconds is applied only when > 0,
so 0 does not mean "no limit"; the Solr emitter/iterator support only
basic auth, not ntlm, and only when a userName is set; pipes-reporters
silently loads zero reporters when given a JSON array, and
pipes-iterator/pipes-reporters instances are built at config load rather
than lazily; under CONTENT_ONLY only a parse-context filter replaces the
built-in one, not the top-level metadata-filters chain;
_mime-include/_mime-exclude also accept a bare string; Tess4J locks
poolSize and maxImagePixels as well as the two paths; and pdf:trapped and
xmp:pdf:Trapped are new 4.x keys rather than renames (3.x captured the
flag only as pdf:docinfo:trapped and dropped the XMP value). Also
corrected the config nesting shown in every pipes-plugin fetcher/emitter
javadoc -- 23 of them, which had it inverted (the instance id is the
outer key, the component name the inner) -- and removed references to a
TesseractOCRConfig.properties file that 4.x does not load (TIKA-4842).
* Pipes plugins no longer bundle their own Jackson: jackson-core, -databind
and -annotations are provided by the host (tika-serialization) and the
plugins parent pom now bans bundling them, so a mapper can cross the
plugin boundary without a second copy of the Jackson classes (seven plugin
zips shipped one). Plugin configuration JSON is parsed by one shared
mapper, PluginJson (tika-plugins-core), which rejects unknown keys,
numbers for enums and duplicate keys, and accepts
// and /* */ comments; the 33 per-plugin *Config classes use it instead of
their own bare ObjectMapper (TIKA-4840).
* tika-server and tika-async-cli now start from a config that contains
// or /* */ comments, as the configuration docs have always said they
may. The main loader accepted them; the steps that re-read the user's
file to merge in server/CLI overrides (ConfigMerger, ensurePluginRoots)
used their own bare parser and refused the whole file; they now use the
shared TikaObjectMapperFactory mapper (TIKA-4834).
* The Kafka pipes iterator no longer stops at the first empty poll. A newly
subscribed consumer spends its first poll(s) joining the group and returns
empty even when the topic has a backlog, so the iterator could enqueue zero
files and report success. It now waits for a partition assignment (bounded
by the new assignmentTimeoutMs, default 30s) and requires a continuous quiet
window (drainIdleMs, default 1s) before concluding the topic is drained.
groupInitialRebalanceDelayMs is deprecated and no longer sent to the
consumer: it is a broker setting that Kafka has always ignored (TIKA-4833).
* Pipes IPC: carry inline document bytes as a raw binary field beside the
tuple in the request envelope -- never inside the tuple or its
ParseContext -- and disable Smile's 7-bit binary encoding. Tuple JSON
serialized by 4.0.0 with an "inline-bytes" parse-context entry no longer
loads; it is rejected with a tailored message (TIKA-4829).
* Digesting embedded documents no longer buffers each embedded object to a
temp file. Zip entries are re-read from the parent archive on rewind, and
a new process-wide CacheMemoryBudget (seeded by the pipes forked server;
default 256MB, clamped to a quarter of the fork's heap; tunable via
-Dtika.pipes.cacheMemoryBudgetBytes in the config's forkedJvmArgs, <=0
disables) lets embedded objects stay in memory past the per-object 1MB
threshold. New public API on TikaInputStream: get(IOSupplier,...),
enableRewind(CacheMemoryBudget), getSeekableByteChannel(). Zip/7z/epub/odf
parsing and zip container detection now read through seekable channels, so
after detection/parsing a TikaInputStream may no longer be file-backed
(hasFile() false); getPath()/getFile() still work and spool on demand
(TIKA-4828).
* Pipes now carries the caller-supplied Content-Type across the worker's
fresh-metadata boundary as a soft detection hint, so every forked-parse
endpoint (/tika, /meta, /rmeta, /unpack, /async, /pipes, plus tika-grpc
and embedded PipesForkParser) can route on a client Content-Type, not
only on the filename. Detection keeps the hint only when it equals or
specializes the content-detected type (e.g. refining image/tiff to
image/x-canon-cr2); for bytes with no magic it can select any type,
matching the routing power the filename already had. The
CONTENT_TYPE_USER_OVERRIDE key is deliberately not carried, so the hint
cannot force an unrelated type (TIKA-4825).
* OneNote extraction now follows document order, omits superseded page
revisions, sorts author metadata, extracts embedded object BLOBs, and
bounds malformed-input recursion and file-derived allocations. Parse
warnings and embedded relationship IDs are exposed in metadata. Malformed
or truncated files that cannot be fully parsed, and files whose walk
yields no content, now fall back to the legacy string dump instead of
failing or returning empty output. The legacy MS-ONESTORE walker bounds
its recursion (depth caps plus file-node-list and fragment-chain cycle
guards) and now honors shouldParseEmbedded for embedded file data
* PDF: extractFontNames threw NullPointerException on a page with no
/Resources dictionary (TIKA-4842).
* tika-server: opt-in Micrometer metrics reporting and endpoint
(TIKA-4839).
* Per-request (parse-context) config for parsers that lock fields
(Tess4J, VLM, OpenAI image-embedding) threw even when empty; locked
fields are still rejected when actually set (TIKA-4843).
* OOXML: new msoffice:has-unreferenced-parts and
msoffice:unreferenced-part-names flag package parts unreachable via the
OPC relationship graph. Structural only, expect false positives; not
applied to XPS (TIKA-4837).
* Shared pipes server (useSharedServer: true): a worker death could trigger
a second, spurious restart that killed the healthy replacement. Forks now
carry a generation; stale reports are dropped. Also fixed: a fork after
shutdown() that was never destroyed, and a temp-dir leak on interrupt
during teardown (TIKA-4844).
* tika-pipes cache memory budget defaults to a quarter of the fork heap;
override with -Dtika.pipes.cacheMemoryBudgetBytes in forkedJvmArgs
(<=0 disables). TikaInputStream: hasFile() also reports cache spills,
toString() no longer spills, new inMemoryContent(channel). Digester gains
digestSink(); a digest is published only on commit(), so failed or empty
translations (e.g. stub PST items) publish no digest. New
TemporaryResources.closeAll(Closeable...) (TIKA-4835).
* Docs/javadocs reconciled with the code: ES/OpenSearch attachmentStrategy
has no default; Kafka connectionsMaxIdleMs is honored; jdbc
queryTimeoutSeconds 0 is not "no limit"; Solr basic auth only; pipes
reporters/iterators are built at config load; Tess4J also locks poolSize
and maxImagePixels; pdf:trapped is new, not renamed; plugin config
nesting fixed in 23 javadocs (TIKA-4842).
* Pipes plugins no longer bundle Jackson; the host provides it. Plugin
config is parsed by a shared strict PluginJson mapper (rejects unknown
and duplicate keys; accepts comments) (TIKA-4840).
* tika-server and tika-async-cli accept // and /* */ comments in config
during override merging, as documented (TIKA-4834).
* Kafka pipes iterator no longer stops on the first empty poll; waits for
partition assignment (assignmentTimeoutMs, 30s) and a quiet window
(drainIdleMs, 1s). groupInitialRebalanceDelayMs is deprecated
(TIKA-4833).
* Pipes IPC carries inline bytes as a raw binary field, not in the tuple;
Smile 7-bit binary encoding disabled. 4.0.0 tuples with an "inline-bytes"
parse-context entry are rejected (TIKA-4829).
* Digesting embedded documents no longer spools each to a temp file; a
process-wide CacheMemoryBudget (default 256MB) keeps them in memory. New
TikaInputStream API: get(IOSupplier,...), enableRewind(CacheMemoryBudget),
getSeekableByteChannel(). Zip-family parsing and detection use seekable
channels, so hasFile() may be false afterward; getPath() still spools on
demand (TIKA-4828).
* Pipes carries the client Content-Type into the forked worker as a
detection hint for all forked endpoints; honored only when it equals or
specializes the detected type, or when there is no magic. The
user-override key is not carried (TIKA-4825).
* OneNote: document-order extraction, superseded revisions omitted, embedded
BLOBs extracted, warnings and relationship IDs in metadata, bounded
recursion/allocation; malformed files fall back to the legacy string dump
(TIKA-4814).
* New GeoGebraParser for *.ggb/*.ggs/*.ggt: geogebra:* metadata, text and
the thumbnail as a THUMBNAIL embedded document, with content-based
detection. Previously typed application/zip with every entry as an
attachment. *.ggs and *.ggp are new mime types; *.ggp is glob-only
(TIKA-4831).
* RawTiffParser extracts the camera-generated JPEG previews embedded in
TIFF-based raw images (Nikon NEF/NRW, Sony ARW/SRF/SR2, Pentax PEF/PTX,
Adobe DNG and Canon CR2, including BigTIFF DNG containers) as thumbnail
embedded documents. image/x-raw-{nikon,sony,pentax,adobe} are now
sub-classes of image/tiff, so a named NEF/ARW/PEF/DNG that used to detect
as image/tiff (TiffParser, metadata only) now detects as image/x-raw-* and
emits thumbnail-N.jpg attachments in /rmeta and /unpack; CR2 keeps its
detection but also gains the attachments. Disable via
"raw-tiff-parser": {"extractPreviews": false} (TIKA-4824).
* RawTiffParser extracts embedded JPEG previews from NEF/NRW, ARW/SRF/SR2,
PEF/PTX, DNG and CR2 as thumbnail embedded documents. image/x-raw-* are
now subtypes of image/tiff, so named raw files detect as image/x-raw-*.
Disable with "raw-tiff-parser": {"extractPreviews": false} (TIKA-4824).
Release 4.0.0 - 8/18/2026
This section is the complete delta from 3.x. It includes everything first
released in 4.0.0-alpha-1 and 4.0.0-beta-1; those sections below are stubs.
Upgrading from 3.x? Start with the migration guides at
https://tika.apache.org/docs -- "Migrating to Tika 4.x", "Migrating Tika
Server to 4.x" and "Metadata Changes in Tika 4.x". They carry the detail
and the code examples behind the summaries here.
Important architectural change: parsing now runs in a forked process
where possible. tika-server's endpoints, tika-app's -a/--async and -f/--fork,
and tika-grpc all parse in forked, crash-isolated tika-pipes workers.
Applications embedding Tika should consider getting the same isolation from
PipesForkParser (tika-pipes-fork-parser) rather than parsing in-process with
AutoDetectParser. Note that the project does not treat denial of service --
memory exhaustion, CPU exhaustion, a crashed process -- as a security issue
when files are parsed outside these isolated paths; see
https://tika.apache.org/security-model.html.
BREAKING CHANGES
--- Platform, packaging, configuration and output format (everyone) ---
* Tika 4.x requires Java 17 or later; 3.x built and ran on Java 11. All
published artifacts are compiled with --release 17 (TIKA-4685).
* tika-app and tika-server-standard now ship as zip distributions with an
adjacent lib/ directory; the published jars are thin launchers and fail
with NoClassDefFoundError if run on their own. This catches
tika-server-standard hardest, because its jar is still on Maven Central:
unzip the distribution and run from inside it (TIKA-4733).
* tika-parsers-standard-package is now a pom, not a jar. Users must add
<type>pom</type> in Maven or @pom in Gradle (TIKA-4712).
* The default content handler is now Markdown. tika-app, tika-server (the
/tika and /rmeta endpoints) and the async/pipes CLI emit Markdown content
by default instead of XHTML/XML (plain text for the async CLI). Request
the previous format explicitly: tika-app -x/--xml, the server's /tika/xml
and /rmeta/xml paths, the async CLI --handler x (TIKA-4663).
* Configuration moved from XML to JSON. TikaConfig and the
org.apache.tika.config XML-configuration API are removed: TikaConfig,
ConfigBase, Field, Param, ParamField, LoadErrorHandler,
InitializableProblemHandler, TikaConfigSerializer and TikaTaskTimeout are
gone. Use TikaLoader from tika-serialization. tika-app
--convert-config-xml-to-json converts a 3.x parsers section as a starting
point; every other section needs manual migration
(TIKA-4544, TIKA-4545, TIKA-4553, TIKA-4565).
* An unregistered component name in a default-parser, default-detector or
default-encoding-detector "exclude" list now throws a TikaConfigException
at config load instead of logging a WARN, so a 3.x config that named the
component by class name or misspelled it now refuses to start. Use the
registered name (e.g. "pdf-parser"); tika-app --list-parser-names prints
them (TIKA-3268, TIKA-4808).
* Metadata keys were renamed for consistency and provenance. Every
Tika-asserted key now lives under a single tk: prefix, replacing 3.x's
scattered X-TIKA:, tika:, tika_pg:, rendering:, signature: and
imagereader: prefixes and bare names such as resourceName; names Tika
coined inside format namespaces are kebab-cased (pdf:hasMarkedContent ->
pdf:has-marked-content) while names from a file or an external standard
keep their spelling; and open key families gained prefixes (audio:, ner:,
envi:, ogg:streams-, grobid:, iso19115:, gdal:, geotopic:, mif:, idml:).
Code using the TikaCoreProperties / TikaPagedText / Rendering constants is
unaffected. Code that references keys by String has two paths: update the
strings with the key-for-key tables in metadata-changes-4x.adoc, or turn
on the compatibility filter below and migrate on your own schedule
(TIKA-4816).
* The opt-in legacy-key-migration-filter restores 3.x key spellings at the
emit edge (default direction V4_TO_V3), so an unmigrated consumer keeps
working against 4.x output; V3_TO_V4 maps 3.x names forward instead.
tika-core bundles metadata-migration-3x-4x.json, the machine-readable
rename/drop table (TIKA-4797).
* The reserved tk: (and legacy X-TIKA:) namespace is now a trust boundary
for String-keyed writes. Metadata#set/add(String, String) throw
IllegalArgumentException on a reserved key instead of 3.x's silent
success, where a document-controlled property named X-TIKA:Parsed-By could
overwrite Tika's own value, and Property's public factories reject reserved
names outright. Document- and tool-derived names now go through
Metadata#add(KeyPrefix, String, String) -- append-only, skip-and-WARN on
hostile names -- or its Instant overload for source-typed dates
(TIKA-4816).
* Metadata no longer implements CreativeCommons, Geographic, HttpHeaders,
Message, ClimateForcast, TIFF or TikaMimeKeys: inherited constants move to
their home interface, e.g. Metadata.CONTENT_TYPE becomes
HttpHeaders.CONTENT_TYPE (now a Property, though the key string is
unchanged). TikaMimeKeys and
ClimateForcast are deleted outright; ClimateForecast (corrected spelling)
replaces the latter, with its keys under cf: (TIKA-4816).
* Other Metadata API changes: setAll(Properties) removed with no replacement
-- it bypassed both the limiter and the reserved-key guard, so use
putAll(Metadata) or individual set/add calls; PassthroughPrefix renamed
KeyPrefix; the Property factories internalClosedChoise / internalOpenChoise
/ externalClosedChoise / externalOpenChoise renamed to ...Choice with no
forwarders; the dead enum constants PropertyType.STRUCTURE and
ValueType.{LOCALE, MIME_TYPE, PROPER_NAME, URL, XPATH} removed; package
org.apache.tika.metadata.writefilter renamed to ...metadata.writelimiter.
Metadata's serialVersionUID also changed, so a 3.x-serialized instance now
fails with InvalidClassException instead of deserializing into an object
that throws on first write (TIKA-4816).
--- Java API (library integrators) ---
* The core SPI signatures changed. Parser.parse takes a TikaInputStream
instead of an InputStream (there is no InputStream overload),
Detector.detect takes (TikaInputStream, Metadata, ParseContext), and
EmbeddedDocumentExtractor's shouldParseEmbedded/parseEmbedded gained a
ParseContext and take a TikaInputStream. Every third-party implementation
must be updated; callers can wrap with TikaInputStream.get(...). The Tika
facade still accepts an InputStream, but Tika.detect(InputStream, ...) no
longer returns the caller's stream at its original position. The detector
still resets the TikaInputStream it reads -- but that read-ahead is
buffered inside an internal wrapper that detect() discards, so the
caller's own stream comes back advanced. Pass a TikaInputStream you own
(and rewind it), or re-open the source
(TIKA-4399, TIKA-4541, TIKA-4569).
* TikaInputStream no longer caches by default. A stream is consumed in
passthrough mode unless enableRewind() is called at position 0;
rewind()/getFile()/getPath() after reading without enableRewind() throw
instead of silently spooling. A parser that read part of a stream and then
asked for a file worked in 3.x and now fails. Digesters call
enableRewind() themselves (TIKA-4618, TIKA-4623).
* Parsing with a concrete parser (not AutoDetectParser) and an empty
ParseContext no longer auto-generates an AutoDetectParser to handle
embedded files: they are silently skipped, with no content and no
exception. Nor does it auto-generate a Detector to identify them; they are
reported as application/octet-stream instead. Set Parser.class and
Detector.class in the ParseContext, or go through AutoDetectParser, which
does this for you (TIKA-4819).
* EmbeddedDocumentExtractorFactory and friends are removed;
ParsingEmbeddedDocumentExtractor and UnpackExtractor are now stateless
singletons (use INSTANCE) that take the enclosing ParseContext as a method
parameter rather than capturing one at construction. Code that supplied a
custom factory should bind an EmbeddedDocumentExtractor instance directly.
EmbeddedDocumentUtil's instance API is likewise removed in favor of statics
that take a ParseContext explicitly (TIKA-4819).
* ParseContext configuration is now resolved per component instance rather
than per config class, because a class-keyed write leaked one component's
config to every other component binding the same config class. Two
consequences: parseContext.get(SomeConfig.class) no longer returns a
JSON-resolved config, so a third-party component following the
PDFBoxRenderer pattern must be handed its config explicitly; and precedence
is inverted -- a JSON config now beats a programmatic
context.set(XConfig.class, ...), which used to win (TIKA-4808).
* ForkParser and the entire org.apache.tika.fork package are removed from
tika-core. Out-of-process parsing is now provided by PipesForkParser in
the new tika-pipes-fork-parser module -- the recommended parser for
untrusted documents. tika-app's -f/--fork routes through it, and
--fork-timeout is rejected rather than silently ignored
(TIKA-4554, TIKA-4571, TIKA-4651).
* Unified timeout model across the library, pipes and server: a total-task
budget plus a progress/stall timeout, composed recursively over embedded
documents. TikaTimeoutException is now a checked exception, and several
parser/pipes config fields were renamed (*TimeoutSeconds / *TimeoutMs ->
*TimeoutMillis, including a unit change for Tess4J) (TIKA-4813).
* Parsers and detectors no longer expose bean setters/getters for their
settings. Configuration moves to per-component *Config objects supplied
through the ParseContext (e.g. GeoParserConfig, DWGParserConfig,
AmazonTranscribeConfig, MagikaDetector/SiegfriedDetector configs)
(TIKA-4758).
* The encoding detectors moved out of parser packages into
org.apache.tika.detect.* and into new tika-encoding-detector-* modules:
org.apache.tika.parser.txt.{CharsetDetector,CharsetMatch,
Icu4jEncodingDetector,UniversalEncodingDetector,BOMDetector,...} are now
org.apache.tika.detect.icu4j.*, org.apache.tika.detect.universal.* and
org.apache.tika.detect.BOMDetector, and
org.apache.tika.parser.html.HtmlEncodingDetector is now
org.apache.tika.detect.html.HtmlEncodingDetector.
NonDetectingEncodingDetector is removed (TIKA-4685, TIKA-4720).
* MetadataListFilter has been renamed MetadataFilter, and the 3.x
MetadataFilter has been removed (TIKA-4546).
* API changes in the EmbeddedStreamTranslator (TIKA-4518), and
DigestingParser is removed (TIKA-4607).
* BasicContentHandlerFactory.parseHandlerType now throws
IllegalArgumentException for an unrecognized handler name instead of
silently returning the supplied default (TIKA-4809).
--- tika-server ---
* All parsing now runs out-of-process through tika-pipes. /tika, /rmeta,
/meta, /unpack, /detect, /pipes and /async share a fixed pool of
numClients forked worker JVMs (default derived from host cores), so a
parser crash, OOM or timeout no longer takes down the server. The cost is
a sizing decision 3.x never asked of you: numClients is both the server's
concurrency ceiling and its CPU/memory footprint, and each fork's heap is
set with pipes.forkedJvmArgs (e.g. -Xmx1g), not the server JVM's. Size
both deliberately; see the cpu-sizing docs (TIKA-4809).
* Capability flags are default-deny and split in two. enableUnsecureFeatures
no longer exists -- a config still carrying it fails to start with an
"Unrecognized field" error -- and is replaced by allowPipes (gates /pipes
and /async) and allowPerRequestConfig (gates the /config endpoints and the
multipart config part). /status is no longer gated and is enabled simply
by listing it under endpoints. tika-grpc gains
the same allowPerRequestConfig flag plus allowComponentModifications, which
gates runtime Save/Delete of fetchers and pipes iterators (TIKA-4764).
* Endpoints removed: /translate/* (unusable as shipped), /tika/main and
/tika/form/main (Boilerpipe; use /tika/text), and the /tika/form family.
The 3.x /tika/config and /tika/form/config forms are replaced by the
/tika/config* multipart POSTs, which require allowPerRequestConfig
(TIKA-4809).
* Endpoints collapsed: /detect/stream is now /detect, and /language/stream
and /language/string are both /language. Behavior changed with the rename:
/detect now runs in the fork pool, so it can return 429, 503 or 413, and a
failure reading the body is a 500 where 3.x returned 200 with
application/octet-stream as if detection had succeeded; /language caps
input at the first 100,000 characters and uses the default LanguageDetector
on the classpath, where 3.x pinned Optimaize (TIKA-4809).
* Output-format routing on /tika changed. The bare /tika endpoint returns
Markdown (was XHTML); use /tika/xml for XHTML. /tika/text is body-only
again, as in 3.x. /tika/json and /tika/config/json default to the server
default (markdown) rather than hardcoded plain text. The Accept header no
longer selects the output format -- 3.x routed bare /tika among plain
text, HTML and XHTML by Accept (nondeterministically for */*); now the
path names the format. An unrecognized handler name in the path is a 400
listing the valid types, instead of silently falling back to the default
(TIKA-4663, TIKA-4809).
* Per-request configuration headers are removed, and are now silently
ignored if sent: writeLimit, throwOnWriteLimitReached,
maxEmbeddedResources/maxEmbeddedCount, X-Tika-Handler and the meta_*
metadata-injection family. The limits move to parse-context
(output-limits.writeLimit, output-limits.throwOnWriteLimit,
embedded-limits.maxCount); X-Tika-Handler becomes an explicit handler path;
meta_* has no replacement, and with per-request config off by default a
caller can no longer bound the output of a single request. The
X-Tika-OCR* and X-Tika-PDF* families were removed earlier in the 4.x line
(TIKA-4809).
* Caller errors now map to accurate HTTP status codes instead of always
returning 200 or 500. A saturated worker pool returns 429, a
crashed/timed-out/OOM worker returns 503, an unknown or reserved
fetcher/emitter or bad handler returns 400, and an over-limit body returns
413; the 429 and 503 responses carry a Retry-After header. Error bodies are
now JSON ({"status":"TIMEOUT"}, with a message field when one is
available) where 3.x returned plain text such as "Parse failed: TIMEOUT"
(TIKA-4809).
* The raw /tika family's 422 responses carry the extracted content only; the
exception is no longer appended to the body -- use /rmeta for the
structured exception (TIKA-4809).
* /meta now runs through the same pipes-backed parser as the other
extraction endpoints, so it gains their crash isolation and their error
handling: a container exception comes back as 200 with
tk:exception:container-exception instead of 500, and /meta/{field} returns
422 instead of 500 or 400. A request with no Accept header now returns
JSON; 3.x returned CSV, still available via Accept: text/csv. /meta also no
longer returns a language field -- it parses with the ignore handler, so
there is no text to detect from; configure a language-detection metadata
filter and use /rmeta or /tika/json instead (TIKA-4809).
* /async requires an object body {"tuples":[...]} instead of a bare JSON
array, validates fetcher/emitter ids at POST time (400), rejects a batch
larger than the queue's total capacity with 400 instead of throttling it,
and one bad tuple no longer stops the async workers. /pipes returns the
same JSON body as /tika/rmeta/unpack -- {"status":<RESULT_STATUS>,
"message":...} -- instead of a /pipes-only {"status":"ok"|"process_crash"}
shape, returns 400 with the reason for a malformed request body, and
rejects emit strategies other than EMIT_ALL, whose passed-back data the
/pipes response cannot carry (TIKA-4809).
* Many server config keys were removed or renamed (logLevel, idBase,
digest, returnStackTrace, port ranges, the spawn-child options, ...), and
an unrecognized key now fails startup with an error naming it; see
migrating-tika-server-4x.adoc for the key-by-key migration. One change no
startup error will flag: taskTimeoutMillis is now
parse-context.timeout-limits.totalTaskTimeoutMillis, and its default grew
from 5 minutes to 1 hour (TIKA-4809, TIKA-4813).
* Request bodies are now capped by maxRequestSizeBytes, defaulting to 1 GiB;
larger requests are rejected with 413, including over-limit chunked
uploads, which previously surfaced as an empty 500 (TIKA-4809).
* The 'endpoints' allowlist now also gates SPI-provided resources; a
discovered resource binds only when its root endpoint is enabled
(TIKA-4809).
* Fetcher-based streaming is removed: the InputStreamFactory pattern for
fetching documents via the fetcherName/fetchKey headers is gone, and all
documents now go through the pipes infrastructure. The no-op
-a/--pluginsConfig flag is removed and now fails option parsing, --help
exits 0, and the tika-server-client module is removed (TIKA-4809).
--- tika-pipes and tika-grpc ---
* tika-pipes implementation modules are now pf4j plugins, reorganized by
resource (tika-pipes-solr) vs task (tika-pipes-fetcher-solr). Core classes
moved to tika-pipes-core, and the file-system components moved out of it
into their own tika-pipes-file-system plugin
(TIKA-4334, TIKA-4519, TIKA-4543).
* FetchEmitTuple JSON now names the per-tuple parse context "parse-context"
(was "parseContext") and rejects unknown tuple fields with an error naming
the field (TIKA-4809).
* The pipes config keys staleFetcherTimeoutSeconds and
staleFetcherDelaySeconds have been removed; a config still carrying them
fails startup (TIKA-4809).
* TimeoutLimits: progressTimeoutMillis of 0 combined with a positive
totalTaskTimeoutMillis is now rejected at config load; it would kill
every task immediately (TIKA-4809).
* The http-fetcher now verifies TLS certificates and hostnames by default;
set verifySsl:false to opt out (TIKA-4809).
* SolrJ moves from 8.11.4 to 10.0.0; the Solr fetcher, emitter and pipes
iterator no longer support Solr 8 (TIKA-4789).
* tika-grpc: the generated Java classes moved from package org.apache.tika
to org.apache.tika.pipes.grpc.proto, so every generated type moves and Java
gRPC clients must update their imports. This is a source break only: the
proto package ("tika") and the service name ("Tika") are unchanged, so the
wire protocol is identical and clients in other languages are unaffected
(TIKA-4808).
--- tika-app and tika-eval-app ---
* tika-app's batch mode is gone. The -bc/batch directory-to-directory command
line (backed by the removed tika-batch module) has no successor flag; use
-a/--async, which runs the same work through tika-pipes
(TIKA-4333, TIKA-4340).
* tika-core/tika-app: NetworkParser and tika-app's -c/--client=<uri>
network-client mode were removed -- they dispatched raw sockets to an
arbitrary user-supplied host with no auth or TLS. Use tika-server
instead (TIKA-4808).
* tika-eval-app's command line changed: the FileProfile sub-command is
removed, the -bc batch-config option is gone, and extract directories are
now named with -e/--extracts (Profile) and -a/--extractsA + -b/--extractsB
(Compare); -i/--inputDir, -d/--db, -c/--config, -n/--numWorkers and
-m/--maxExtractLength replace the 3.x spellings
(TIKA-4342, TIKA-4450, TIKA-4452, TIKA-4507).
* tika-app: the inline short forms -eX (output encoding) and -pX (document
password) were removed from standard mode; use --encoding=X and
--password=X. Prefix-matching them silently swallowed single-dash long
names, so every single-dash long name is now rejected with a message
naming the two-dash form (TIKA-4808).
--- Parser, detector and output behavior ---
* The tika-langdetect-tika module is removed (TikaLanguageDetector,
LanguageIdentifier, LanguageProfile, LanguageProfilerBuilder,
ProfilingWriter). tika-app, tika-server and tika-eval now bundle the new
CharSoup detector (tika-langdetect-charsoup) instead of
tika-langdetect-optimaize, so the language reported by default changes
(TIKA-4662).
* PDF: extractIncrementalUpdateInfo now defaults to true (was false), so
every PDF parse emits pdf:incremental-update-count and related keys
without configuration. parseIncrementalUpdates remains false
(TIKA-4354, TIKA-4358).
* Audio cover art is now extracted as embedded documents from MP3 (ID3v2
APIC/PIC), MP4 (covr), Vorbis and FLAC. Embedded-document counts and
/rmeta list lengths for audio files change (TIKA-4801).
* The DOM-based OOXML extractors are removed (XWPFWordExtractorDecorator,
XSLFPowerPointExtractorDecorator, POIXMLTextExtractorDecorator,
XPSTextExtractor) and with them the OfficeParserConfig keys
useSAXDocxExtractor and useSAXPptxExtractor. The SAX extractors are the
only implementation (TIKA-4692, TIKA-4708).
* The MuPDF renderer is removed (org.apache.tika.renderer.pdf.mutool); PDF
page rendering for OCR now uses the PDFBox renderer or the new
PopplerRenderer (TIKA-4664).
* The legacy ExternalParser is removed; external parsers now require explicit
JSON configuration. CompositeExternalParser and ExternalParsersFactory,
which loaded tika-external-parsers.xml definitions from the classpath
automatically, are gone (TIKA-4707).
* Headers are no longer injected into the body/content of MSG files
(TIKA-4345). Please open a ticket if you need this behavior across email
formats.
--- Removed modules and classes ---
* Removed modules with no direct replacement: tika-batch (TIKA-4333),
tika-dl (TIKA-4499), the advanced media module (TIKA-4500), tika-fuzzing
(TIKA-4506), tika-age-recogniser (TIKA-4343), tika-server-eval
(TIKA-4555), the dotnet bindings (TIKA-4332) and snaps deployment
(TIKA-4502).
* Removed parsers and classes with no direct replacement:
SentimentAnalysisParser (TIKA-4574); ObjectRecognitionParser and the
Tensorflow recognisers/captioners; PooledTimeSeriesParser;
org.apache.tika.parser.pdf.AccessChecker (replaced by
PDFParserConfig.AccessCheckMode); and the jempbox-based JempboxExtractor /
XMPMetadataExtractor / pdf.xmpschemas.* classes superseded by the unified
XMP extractor (TIKA-4775).
* Smaller removals: ExceptionUtils.trimMessage moved from tika-core to
tika-eval-core; HttpClientFactory's inert redirect-host allowlist
accessors (get/setAllowedHostsForRedirect) are gone (TIKA-4809); and
org.apache.tika.utils.{RereadableInputStream,AnnotationUtils},
org.apache.tika.io.{IOUtils,InputStreamFactory},
org.apache.tika.sax.DIFContentHandler,
org.apache.tika.parser.{AutoDetectParserFactory,ParserFactory} and
org.apache.tika.parser.internal.Activator are removed.
NEW FEATURES
* tika-pipes gains three parse modes: NO_PARSE (detect only, no parse),
CONTENT_ONLY (emitters write raw content, no metadata envelope) and
UNPACK (write embedded bytes out). All three are wired through tika-app
and tika-server (TIKA-4631, TIKA-4637, TIKA-4656).