dev  

Messages by Thread

• • Re: [PR] Spouts check the scheme of stored URLs before emitting [stormcrawler] via GitHub
  • Re: [PR] Spouts check the scheme of stored URLs before emitting [stormcrawler] via GitHub
  • Re: [PR] Spouts check the scheme of stored URLs before emitting [stormcrawler] via GitHub
  • Re: [PR] Spouts check the scheme of stored URLs before emitting [stormcrawler] via GitHub
  • Re: [PR] Spouts check the scheme of stored URLs before emitting [stormcrawler] via GitHub
  • Re: [PR] Spouts check the scheme of stored URLs before emitting [stormcrawler] via GitHub
  • Re: [PR] Spouts check the scheme of stored URLs before emitting [stormcrawler] via GitHub
  • Re: [PR] Spouts check the scheme of stored URLs before emitting [stormcrawler] via GitHub
  • Re: [PR] Spouts check the scheme of stored URLs before emitting [stormcrawler] via GitHub
  • Re: [PR] Spouts check the scheme of stored URLs before emitting [stormcrawler] via GitHub
  • Re: [PR] Spouts check the scheme of stored URLs before emitting [stormcrawler] via GitHub
• [PR] Canonicalise hosts for politeness queues and the robots.txt cache [stormcrawler] via GitHub
  • Re: [PR] Canonicalise hosts for politeness queues and the robots.txt cache [stormcrawler] via GitHub
  • Re: [PR] Canonicalise hosts for politeness queues and the robots.txt cache [stormcrawler] via GitHub
  • Re: [PR] Canonicalise hosts for politeness queues and the robots.txt cache [stormcrawler] via GitHub
  • Re: [PR] Canonicalise hosts for politeness queues and the robots.txt cache [stormcrawler] via GitHub
  • Re: [PR] Canonicalise hosts for politeness queues and the robots.txt cache [stormcrawler] via GitHub
  • Re: [PR] Canonicalise hosts for politeness queues and the robots.txt cache [stormcrawler] via GitHub
  • Re: [PR] Canonicalise hosts for politeness queues and the robots.txt cache [stormcrawler] via GitHub
  • Re: [PR] Canonicalise hosts for politeness queues and the robots.txt cache [stormcrawler] via GitHub
  • Re: [PR] Canonicalise hosts for politeness queues and the robots.txt cache [stormcrawler] via GitHub
  • Re: [PR] Canonicalise hosts for politeness queues and the robots.txt cache [stormcrawler] via GitHub
  • Re: [PR] Canonicalise hosts for politeness queues and the robots.txt cache [stormcrawler] via GitHub
  • Re: [PR] Canonicalise hosts for politeness queues and the robots.txt cache [stormcrawler] via GitHub
  • Re: [PR] Canonicalise hosts for politeness queues and the robots.txt cache [stormcrawler] via GitHub
  • Re: [PR] Canonicalise hosts for politeness queues and the robots.txt cache [stormcrawler] via GitHub
  • Re: [PR] Canonicalise hosts for politeness queues and the robots.txt cache [stormcrawler] via GitHub
  • Re: [PR] Canonicalise hosts for politeness queues and the robots.txt cache [stormcrawler] via GitHub
  • Re: [PR] Canonicalise hosts for politeness queues and the robots.txt cache [stormcrawler] via GitHub
  • Re: [PR] Canonicalise hosts for politeness queues and the robots.txt cache [stormcrawler] via GitHub
• [PR] Robots.txt fetch no longer replaces the configured content limit [stormcrawler] via GitHub
  • Re: [PR] Robots.txt fetch no longer replaces the configured content limit [stormcrawler] via GitHub
  • Re: [PR] Robots.txt fetch no longer replaces the configured content limit [stormcrawler] via GitHub
  • Re: [PR] Robots.txt fetch no longer replaces the configured content limit [stormcrawler] via GitHub
  • Re: [PR] Robots.txt fetch no longer replaces the configured content limit [stormcrawler] via GitHub
  • Re: [PR] Robots.txt fetch no longer replaces the configured content limit [stormcrawler] via GitHub
• [PR] URLFilters: treat an exception from a filter as a rejection [stormcrawler] via GitHub
  • Re: [PR] URLFilters: treat an exception from a filter as a rejection [stormcrawler] via GitHub
  • Re: [PR] URLFilters: treat an exception from a filter as a rejection [stormcrawler] via GitHub
  • Re: [PR] URLFilters: treat an exception from a filter as a rejection [stormcrawler] via GitHub
  • Re: [PR] URLFilters: treat an exception from a filter as a rejection [stormcrawler] via GitHub
  • Re: [PR] URLFilters: treat an exception from a filter as a rejection [stormcrawler] via GitHub
  • Re: [PR] URLFilters: treat an exception from a filter as a rejection [stormcrawler] via GitHub
  • Re: [PR] URLFilters: treat an exception from a filter as a rejection [stormcrawler] via GitHub
• [PR] IPFilterRules: add anylocal and multicast keywords, fail on unparseable rules [stormcrawler] via GitHub
  • Re: [PR] IPFilterRules: add anylocal and multicast keywords, fail on unparseable rules [stormcrawler] via GitHub
  • Re: [PR] IPFilterRules: add anylocal and multicast keywords, fail on unparseable rules [stormcrawler] via GitHub
  • Re: [PR] IPFilterRules: add anylocal and multicast keywords, fail on unparseable rules [stormcrawler] via GitHub
  • Re: [PR] IPFilterRules: add anylocal and multicast keywords, fail on unparseable rules [stormcrawler] via GitHub
  • Re: [PR] IPFilterRules: add anylocal and multicast keywords, fail on unparseable rules [stormcrawler] via GitHub
  • Re: [PR] IPFilterRules: add anylocal and multicast keywords, fail on unparseable rules [stormcrawler] via GitHub
  • Re: [PR] IPFilterRules: add anylocal and multicast keywords, fail on unparseable rules [stormcrawler] via GitHub
• [GH] (stormcrawler/issue-2063-tika-4): Workflow run "Java CI with Maven" is working again! GitBox
  • [GH] (stormcrawler/issue-2063-tika-4): Workflow run "Java CI with Maven" is working again! GitBox
• [GH] (stormcrawler/issue-2063-tika-4): Workflow run "Java CI with Maven" failed! GitBox
• [PR] Upgrade to Tika 4.0.0 [stormcrawler] via GitHub
  • Re: [PR] Upgrade to Tika 4.0.0 [stormcrawler] via GitHub
  • Re: [PR] Upgrade to Tika 4.0.0 [stormcrawler] via GitHub
  • Re: [PR] Upgrade to Tika 4.0.0 [stormcrawler] via GitHub
  • Re: [PR] Upgrade to Tika 4.0.0 [stormcrawler] via GitHub
  • Re: [PR] Upgrade to Tika 4.0.0 [stormcrawler] via GitHub
  • Re: [PR] Upgrade to Tika 4.0.0 [stormcrawler] via GitHub
  • Re: [PR] Upgrade to Tika 4.0.0 [stormcrawler] via GitHub
  • Re: [PR] Upgrade to Tika 4.0.0 [stormcrawler] via GitHub
  • Re: [PR] Upgrade to Tika 4.0.0 [stormcrawler] via GitHub
  • Re: [PR] Upgrade to Tika 4.0.0 [stormcrawler] via GitHub
  • Re: [PR] Upgrade to Tika 4.0.0 [stormcrawler] via GitHub
• [PR] Use batch endpoint in URLFrontier module for DISCOVERED urls [stormcrawler] via GitHub
  • Re: [PR] Use batch endpoint in URLFrontier module for DISCOVERED urls [stormcrawler] via GitHub
  • Re: [PR] Use batch endpoint in URLFrontier module for DISCOVERED urls [stormcrawler] via GitHub
  • Re: [PR] Use batch endpoint in URLFrontier module for DISCOVERED urls [stormcrawler] via GitHub
  • Re: [PR] Use batch endpoint in URLFrontier module for DISCOVERED urls [stormcrawler] via GitHub
  • Re: [PR] Use batch endpoint in URLFrontier module for DISCOVERED urls [stormcrawler] via GitHub
  • Re: [PR] Use batch endpoint in URLFrontier module for DISCOVERED urls [stormcrawler] via GitHub
  • Re: [PR] Use batch endpoint in URLFrontier module for DISCOVERED urls [stormcrawler] via GitHub
  • Re: [PR] Use batch endpoint in URLFrontier module for DISCOVERED urls [stormcrawler] via GitHub
  • Re: [PR] Use batch endpoint in URLFrontier module for DISCOVERED urls [stormcrawler] via GitHub
• [GH] (stormcrawler/issue-2108-parse-result-lookup): Workflow run "Java CI with Maven" is working again! GitBox
• [GH] (stormcrawler/issue-2108-parse-result-lookup): Workflow run "Java CI with Maven" failed! GitBox
• [GH] (stormcrawler/fix/warc-crlf-escaping-2105): Workflow run "Java CI with Maven" is working again! GitBox
• [GH] (stormcrawler/warc-configurable-digest-algorithm): Workflow run "Java CI with Maven" failed! GitBox
  • [GH] (stormcrawler/warc-configurable-digest-algorithm): Workflow run "Java CI with Maven" failed! GitBox
• [GH] (stormcrawler/issue-2107-quadratic-comma-split): Workflow run "Java CI with Maven" is working again! GitBox
  • [GH] (stormcrawler/issue-2107-quadratic-comma-split): Workflow run "Java CI with Maven" is working again! GitBox
• [GH] (stormcrawler/regenerate_license_file_after_dep_upgrade): Workflow run "Java CI with Maven" failed! GitBox
  • [GH] (stormcrawler/regenerate_license_file_after_dep_upgrade): Workflow run "Java CI with Maven" failed! GitBox
  • [GH] (stormcrawler/regenerate_license_file_after_dep_upgrade): Workflow run "Java CI with Maven" failed! GitBox
  • [GH] (stormcrawler/regenerate_license_file_after_dep_upgrade): Workflow run "Java CI with Maven" failed! GitBox
  • [GH] (stormcrawler/regenerate_license_file_after_dep_upgrade): Workflow run "Java CI with Maven" failed! GitBox
  • [GH] (stormcrawler/regenerate_license_file_after_dep_upgrade): Workflow run "Java CI with Maven" failed! GitBox
  • [GH] (stormcrawler/regenerate_license_file_after_dep_upgrade): Workflow run "Java CI with Maven" failed! GitBox
  • [GH] (stormcrawler/regenerate_license_file_after_dep_upgrade): Workflow run "Java CI with Maven" failed! GitBox
  • [GH] (stormcrawler/regenerate_license_file_after_dep_upgrade): Workflow run "Java CI with Maven" failed! GitBox
  • [GH] (stormcrawler/regenerate_license_file_after_dep_upgrade): Workflow run "Java CI with Maven" failed! GitBox
  • [GH] (stormcrawler/regenerate_license_file_after_dep_upgrade): Workflow run "Java CI with Maven" failed! GitBox
  • [GH] (stormcrawler/regenerate_license_file_after_dep_upgrade): Workflow run "Java CI with Maven" failed! GitBox
  • [GH] (stormcrawler/regenerate_license_file_after_dep_upgrade): Workflow run "Java CI with Maven" failed! GitBox
  • [GH] (stormcrawler/regenerate_license_file_after_dep_upgrade): Workflow run "Java CI with Maven" failed! GitBox
  • [GH] (stormcrawler/regenerate_license_file_after_dep_upgrade): Workflow run "Java CI with Maven" failed! GitBox
• [GH] (stormcrawler/fix/tika-whitelist-detected-mimetype): Workflow run "Java CI with Maven" failed! GitBox
  • [GH] (stormcrawler/fix/tika-whitelist-detected-mimetype): Workflow run "Java CI with Maven" failed! GitBox
  • [GH] (stormcrawler/fix/tika-whitelist-detected-mimetype): Workflow run "Java CI with Maven" failed! GitBox
  • [GH] (stormcrawler/fix/tika-whitelist-detected-mimetype): Workflow run "Java CI with Maven" failed! GitBox
• [PR] feat: implement MIME type whitelist filtering in ParserBolt for Tika-… [stormcrawler] via GitHub
  • Re: [PR] feat: implement MIME type whitelist filtering in ParserBolt for Tika-… [stormcrawler] via GitHub
  • Re: [PR] feat: implement MIME type whitelist filtering in ParserBolt for Tika-… [stormcrawler] via GitHub
  • Re: [PR] feat: implement MIME type whitelist filtering in ParserBolt for Tika-… [stormcrawler] via GitHub
  • Re: [PR] feat: implement MIME type whitelist filtering in ParserBolt for Tika-… [stormcrawler] via GitHub
  • Re: [PR] feat: implement MIME type whitelist filtering in ParserBolt for Tika-… [stormcrawler] via GitHub
  • Re: [PR] feat: implement MIME type whitelist filtering in ParserBolt for Tika-… [stormcrawler] via GitHub
  • Re: [PR] #2104 feat: implement MIME type whitelist filtering in ParserBolt for Tika [stormcrawler] via GitHub
  • Re: [PR] #2104 feat: implement MIME type whitelist filtering in ParserBolt for Tika [stormcrawler] via GitHub
  • Re: [PR] #2104 feat: implement MIME type whitelist filtering in ParserBolt for Tika [stormcrawler] via GitHub
  • Re: [PR] #2104 feat: implement MIME type whitelist filtering in ParserBolt for Tika [stormcrawler] via GitHub
  • Re: [PR] #2104 feat: implement MIME type whitelist filtering in ParserBolt for Tika [stormcrawler] via GitHub
  • Re: [PR] #2104 feat: implement MIME type whitelist filtering in ParserBolt for Tika [stormcrawler] via GitHub
  • Re: [PR] #2104 feat: implement MIME type whitelist filtering in ParserBolt for Tika [stormcrawler] via GitHub
  • Re: [PR] #2104 feat: implement MIME type whitelist filtering in ParserBolt for Tika [stormcrawler] via GitHub
• [PR] build(deps): bump org.jsoup:jsoup from 1.23.1 to 1.23.2 [stormcrawler] via GitHub
  • Re: [PR] build(deps): bump org.jsoup:jsoup from 1.23.1 to 1.23.2 [stormcrawler] via GitHub
• [PR] build(deps): bump software.amazon.awssdk:bom from 2.54.1 to 2.54.6 [stormcrawler] via GitHub
  • Re: [PR] build(deps): bump software.amazon.awssdk:bom from 2.54.1 to 2.54.6 [stormcrawler] via GitHub
• [PR] ParseResult: non-mutating lookup, explicit getOrCreate() and no empty documents emitted [stormcrawler] via GitHub
  • Re: [PR] ParseResult: non-mutating lookup, explicit getOrCreate() and no empty documents emitted [stormcrawler] via GitHub
  • Re: [PR] ParseResult: non-mutating lookup, explicit getOrCreate() and no empty documents emitted [stormcrawler] via GitHub
  • Re: [PR] ParseResult: non-mutating lookup, explicit getOrCreate() and no empty documents emitted [stormcrawler] via GitHub
  • Re: [PR] ParseResult: non-mutating lookup, explicit getOrCreate() and no empty documents emitted [stormcrawler] via GitHub
  • Re: [PR] ParseResult: non-mutating lookup, explicit getOrCreate() and no empty documents emitted [stormcrawler] via GitHub
  • Re: [PR] ParseResult: non-mutating lookup, explicit getOrCreate() and no empty documents emitted [stormcrawler] via GitHub
  • Re: [PR] ParseResult: non-mutating lookup, explicit getOrCreate() and no empty documents emitted [stormcrawler] via GitHub
• [PR] CommaSeparatedToMultivaluedMetadata: replace quadratic token append with a single bulk write [stormcrawler] via GitHub
  • Re: [PR] CommaSeparatedToMultivaluedMetadata: replace quadratic token append with a single bulk write [stormcrawler] via GitHub
  • Re: [PR] CommaSeparatedToMultivaluedMetadata: replace quadratic token append with a single bulk write [stormcrawler] via GitHub
  • Re: [PR] CommaSeparatedToMultivaluedMetadata: replace quadratic token append with a single bulk write [stormcrawler] via GitHub
  • Re: [PR] CommaSeparatedToMultivaluedMetadata: replace quadratic token append with a single bulk write [stormcrawler] via GitHub
  • Re: [PR] CommaSeparatedToMultivaluedMetadata: replace quadratic token append with a single bulk write [stormcrawler] via GitHub
• [PR] Make the WARC digest algorithm configurable [stormcrawler] via GitHub
  • Re: [PR] Make the WARC digest algorithm configurable [stormcrawler] via GitHub
  • Re: [PR] Make the WARC digest algorithm configurable [stormcrawler] via GitHub
  • Re: [PR] Make the WARC digest algorithm configurable [stormcrawler] via GitHub
  • Re: [PR] Make the WARC digest algorithm configurable [stormcrawler] via GitHub
  • Re: [PR] Make the WARC digest algorithm configurable [stormcrawler] via GitHub
  • Re: [PR] Make the WARC digest algorithm configurable [stormcrawler] via GitHub
  • Re: [PR] Make the WARC digest algorithm configurable [stormcrawler] via GitHub
  • Re: [PR] Make the WARC digest algorithm configurable [stormcrawler] via GitHub
  • Re: [PR] Make the WARC digest algorithm configurable [stormcrawler] via GitHub
  • Re: [PR] Make the WARC digest algorithm configurable [stormcrawler] via GitHub
  • Re: [PR] Make the WARC digest algorithm configurable [stormcrawler] via GitHub
  • Re: [PR] Make the WARC digest algorithm configurable [stormcrawler] via GitHub
  • Re: [PR] Make the WARC digest algorithm configurable [stormcrawler] via GitHub
  • Re: [PR] Make the WARC digest algorithm configurable [stormcrawler] via GitHub
• [PR] Escape CR and LF in WARC metadata records and resource Content-Type [stormcrawler] via GitHub
  • Re: [PR] Escape CR and LF in WARC metadata records and resource Content-Type [stormcrawler] via GitHub
  • Re: [PR] Escape CR and LF in WARC metadata records and resource Content-Type [stormcrawler] via GitHub
  • Re: [PR] Escape CR and LF in WARC metadata records and resource Content-Type [stormcrawler] via GitHub
  • Re: [PR] Escape CR and LF in WARC metadata records and resource Content-Type [stormcrawler] via GitHub
  • Re: [PR] Escape CR and LF in WARC metadata records and resource Content-Type [stormcrawler] via GitHub
  • Re: [PR] Escape CR and LF in WARC metadata records and resource Content-Type [stormcrawler] via GitHub
  • Re: [PR] Escape CR and LF in WARC metadata records and resource Content-Type [stormcrawler] via GitHub
• [I] Persistence spouts emit stored URLs without checking the scheme [stormcrawler] via GitHub
  • Re: [I] Persistence spouts emit stored URLs without checking the scheme [stormcrawler] via GitHub
  • Re: [I] Persistence spouts emit stored URLs without checking the scheme [stormcrawler] via GitHub
• [I] Gzipped sitemap files are expanded with no size bound, and SiteMapParserBolt reads Content-Type under a key that is never present [stormcrawler] via GitHub
• [I] SQL IndexerBolt interpolates metadata labels into the statement as unquoted identifiers [stormcrawler] via GitHub
  • Re: [I] SQL IndexerBolt interpolates metadata labels into the statement as unquoted identifiers [stormcrawler] via GitHub
• [I] OpenSearch addresses default to http and Basic credentials are registered for any host [stormcrawler] via GitHub
• [I] Playwright protocol applies neither the IP filter nor the URL filters to the requests it makes [stormcrawler] via GitHub
• [I] SCProxy.toString() rebuilds the proxy URL with user and password, and the protocol logs it [stormcrawler] via GitHub
  • Re: [I] SCProxy.toString() rebuilds the proxy URL with user and password, and the protocol logs it [stormcrawler] via GitHub
• [I] CommaSeparatedToMultivaluedMetadata appends one token at a time, which is quadratic [stormcrawler] via GitHub
  • Re: [I] CommaSeparatedToMultivaluedMetadata appends one token at a time, which is quadratic [stormcrawler] via GitHub
  • Re: [I] CommaSeparatedToMultivaluedMetadata appends one token at a time, which is quadratic [stormcrawler] via GitHub
• [I] ParseResult.get() inserts an entry for a URL that was never parsed [stormcrawler] via GitHub
  • Re: [I] ParseResult.get() inserts an entry for a URL that was never parsed [stormcrawler] via GitHub
  • Re: [I] ParseResult.get() inserts an entry for a URL that was never parsed [stormcrawler] via GitHub
• [I] Tika mimetype whitelist is checked against the HTTP header while the parser is chosen from the bytes [stormcrawler] via GitHub
  • Re: [I] Tika mimetype whitelist is checked against the HTTP header while the parser is chosen from the bytes [stormcrawler] via GitHub
  • Re: [I] Tika mimetype whitelist is checked against the HTTP header while the parser is chosen from the bytes [stormcrawler] via GitHub
• [I] Response headers named metrics.* are fed straight into the fetcher metric registry, and a non-numeric one fails the fetch [stormcrawler] via GitHub
• [I] SiteMapParserBolt sniffs every document, parses sitemaps non-strict, and turns a failed sitemap parse into a terminal ERROR [stormcrawler] via GitHub
  • Re: [I] SiteMapParserBolt sniffs every document, parses sitemaps non-strict, and turns a failed sitemap parse into a terminal ERROR [stormcrawler] via GitHub
  • Re: [I] SiteMapParserBolt sniffs every document, parses sitemaps non-strict, and turns a failed sitemap parse into a terminal ERROR [stormcrawler] via GitHub
• [I] Response headers can populate the reserved verbatim request and response metadata keys [stormcrawler] via GitHub
• [I] WARC digest algorithm is hard coded to SHA-1 with no way to choose another [stormcrawler] via GitHub
  • Re: [I] WARC digest algorithm is hard coded to SHA-1 with no way to choose another [stormcrawler] via GitHub
  • Re: [I] WARC digest algorithm is hard coded to SHA-1 with no way to choose another [stormcrawler] via GitHub
  • Re: [I] WARC digest algorithm is hard coded to SHA-1 with no way to choose another [stormcrawler] via GitHub
• [I] WARC metadata records write metadata values into warc-fields without escaping CR LF [stormcrawler] via GitHub
  • Re: [I] WARC metadata records write metadata values into warc-fields without escaping CR LF [stormcrawler] via GitHub
  • Re: [I] WARC metadata records write metadata values into warc-fields without escaping CR LF [stormcrawler] via GitHub
  • Re: [I] WARC metadata records write metadata values into warc-fields without escaping CR LF [stormcrawler] via GitHub
• [I] HostURLFilter memoizes the source host in plain fields shared by all fetcher threads [stormcrawler] via GitHub
  • Re: [I] HostURLFilter memoizes the source host in plain fields shared by all fetcher threads [stormcrawler] via GitHub
  • Re: [I] HostURLFilter memoizes the source host in plain fields shared by all fetcher threads [stormcrawler] via GitHub
• [I] URLFrontier channels are always plaintext and the module has no TLS option [stormcrawler] via GitHub
  • Re: [I] URLFrontier channels are always plaintext and the module has no TLS option [stormcrawler] via GitHub
• [I] A response header named set-header becomes a request header directive [stormcrawler] via GitHub
• [I] CharsetIdentification retries the meta charset lookup ten bytes at a time by recursing, and the call sits outside JSoupParserBolt's catch block [stormcrawler] via GitHub
• [I] The file scheme is enabled by default and FileProtocol reads any path the worker can open [stormcrawler] via GitHub
  • Re: [I] The file scheme is enabled by default and FileProtocol reads any path the worker can open [stormcrawler] via GitHub

• Earlier messages
• Later messages