Utkarsh Vashishtha created FLINK-40776:
------------------------------------------
Summary: PARSE_URL compiles the query-parameter pattern on every
row
Key: FLINK-40776
URL: https://issues.apache.org/jira/browse/FLINK-40776
Project: Flink
Issue Type: Improvement
Components: Table SQL / Runtime
Reporter: Utkarsh Vashishtha
{{SqlFunctionUtils.parseUrl(urlStr, partToExtract, key)}} compiles a new
{{Pattern}} on every invocation:
{code:java}
Pattern p = Pattern.compile("(&|^)" + Pattern.quote(key) + "=([^&]*)");
{code}
The pattern depends only on {{{}key{}}}. The URL parse immediately above
already goes through {{{}URL_CACHE{}}}.
h3. Fix
Route the pattern through a {{ThreadLocalCache}} keyed on {{{}key{}}}, matching
{{URL_CACHE}} in the same class:
{code:java}
private static final ThreadLocalCache<String, Pattern>
QUERY_PARAM_PATTERN_CACHE =
ThreadLocalCache.of(k -> Pattern.compile("(&|^)" + Pattern.quote(k) +
"=([^&]*)"));
{code}
This fix already runs in our Flink fork and performance benchmarks show an
improvement ranging till 4x, depending on key being static (most common /
highest improvement) v/s dynamic (dependent on cardinality).
Found while investigating per-row regex costs in a production Flink SQL
deployment.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
