abhinav-phi commented on PR #2111: URL: https://github.com/apache/stormcrawler/pull/2111#issuecomment-5496002912
Thanks @dpol1 and @jnioche for the review — all points addressed in 93cef05a: - **Cap ordering (dpol1)**: the cap is now applied **after** blank tokens are dropped, so it counts real values — with `maxTokens: 2` and the value `",,,a,b,c"` the result is `[a, b]`, i.e. the first real tokens are kept. Covered by a regression test (`capAppliesAfterBlankTokensAreDropped`). - **Default (jnioche)**: lowered from 65536 to **128**, tested by `defaultCapTrimsLongValues`. The two scale tests from the issue now configure `maxTokens` explicitly, since the bulk write path they exercise is independent of the cap. - **Docs (jnioche)**: the optional `maxTokens` parameter and its default are documented in the filter's javadoc and in the CommaSeparatedToMultivaluedMetadata entry of `internals.adoc`. - The confusing reference to `http.content.limit` is removed. Full core suite: 419 tests, 0 failures, 0 errors. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
