The GitHub Actions job "Java CI with Maven" on stormcrawler.git/main has 
succeeded.
Run started by GitHub user rzo1 (triggered by rzo1).

Head commit for run:
c2493e74e81ffbe40a235057f886c43c5e1ecbca / Julien Nioche 
<[email protected]>
Parse StatusUpdaterBenchmark input in parallel to raise its baseline throughput 
(#2280)

* Speed up the input of StatusUpdaterBenchmark

- parse the JSON lines with a streaming JsonParser instead of building
  a tree, directly from the UTF-8 bytes
- buffer the input and use 64 KB buffers for the gzip decompression
- parse the lines in a pool of threads, the reading thread only
  decompresses and hands over blocks of lines; the size of the pool is
  set with status.updater.benchmark.parsers, half the cores by default

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* Address review of StatusUpdaterBenchmark parse pool

- fixed default of 2 parsers, so that baselines are comparable across
  machines; document that a single parser keeps the input order
- reject a number of parsers lower than 1
- move the splitting of the input into blocks and lines to methods of
  their own and test them, along with the precedence of discovered
  items over known ones

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* Fail StatusUpdaterBenchmark if a parser or bolt thread dies

A refetch date too large for a Date made a parser thread die, the run then
hung or reported a wrong number of URLs sent. Such lines are now invalid and
an uncaught exception in a parser or bolt thread ends the run with 1.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

Report URL: https://github.com/apache/stormcrawler/actions/runs/40845631492

With regards,
GitHub Actions via GitBox

Reply via email to