tballison commented on PR #3009:
URL: https://github.com/apache/tika/pull/3009#issuecomment-5269442298
From my claude. :rofl:
```
1. Success clobbering — EMIT_SUCCESS_PASSBACK results (already emitted to
S3/ES) get replaced with a failure status → retries re-emit already-indexed
docs. Fix: degrade emitData only, keep the status.
2. Pre-check false rejection — the "lower-bound" estimate is empirically
~3.6×
over for short-string-heavy metadata (measured), so big archive results
that
fit under the limit get discarded — the PR's own target case regresses.
3. Estimate unit change breaks other consumers — AsyncEmitter's heap budget
and the DYNAMIC threshold now admit ~2× the real heap/content; needs a
separate heap vs. wire estimate.
4. OOM catch hazards — bypasses the module's exit-on-OOM/restart policy;
and
since the giant result is still live on the stack, the fallback
serialization
can OOM again, escaping into catch(Throwable) and hanging the client until
socket timeout.
5. Coverage gaps — writeIntermediate() and writeCrash() have none of the
protections; same desync bug remains on those paths. awaitAck() still reads
with the hardcoded 100 MB default.
6. Config/docs — setMaxIpcPayloadBytes javadoc wrongly claims requests use
the
built-in default (the limit is bidirectional; a lowered limit makes big
requests die as undiagnosable UNSPECIFIED_CRASH); knob missing from
configuration.adoc.
7. Branch hygiene — needs rebase: merge base predates #2962, so the GitHub
diff shows ~463 additions when the real change is 5 files / +272 (I
test-merged: clean, all 26 tests pass), and branch CI never exercises the
TIKA-4813 timeout model.
8. Architecture — one fix subsumes Nick's finding plus #2, #4, #5, and the
post-check's ~2× memory spike: serialize through a size-capped counting
stream
instead of the three-layer estimate/OOM-catch/post-check.
Note the counting-stream fix also resolves Nick's bug naturally (the tiny
error frame is the only thing ever buffered). Minor test/comment nits from
earlier still apply.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
