[
https://issues.apache.org/jira/browse/NIFI-15634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18124969#comment-18124969
]
Sejal Gupta commented on NIFI-15634:
------------------------------------
Hi [~gauravkadaskar] , submitted Pull Request [#11781|
https://github.com/apache/nifi/pull/11781] to address this issue.
Summary of Verification:
Tested with a MySQL database using general_log to inspect executed queries
during runtime. As shown in the attached screenshot, metadata discovery now
correctly executes `SELECT * FROM large_table WHERE 1 = 0` prior to executing
the actual data extraction query `SELECT * FROM large_table`. This prevents
full table scans and high heap memory usage during schema inspection.
> Full query execution during metadata discovery for Generic/MySQL in
> QueryDatabaseTableRecord causes high heap usage.
> --------------------------------------------------------------------------------------------------------------------
>
> Key: NIFI-15634
> URL: https://issues.apache.org/jira/browse/NIFI-15634
> Project: Apache NiFi
> Issue Type: Bug
> Affects Versions: 2.7.2
> Reporter: gaurav kadaskar
> Priority: Major
> Labels: heap, querydatabasetablerecord
> Attachments: Screenshot from 2026-10-08 15-18-11.png
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> h3. Problem
> When Database Type is set to Generic or MySQL in QueryDatabaseTableRecord,
> getDatabaseDialectService() returns DatabaseAdapterDatabaseDialectService.
> During metadata discovery, NiFi executes the full user query instead of using
> a
> metadata-only query. Since no guard condition is applied, the query scans the
> entire dataset, which leads to high heap usage and JVM memory pressure for
> large result sets.
> h3. Previous Behavior
> Earlier NiFi versions generated a metadata-safe query by appending:
> {code:sql}
> WHERE 1=0
> {code}
> This allowed schema retrieval without reading actual data.
> h3. Expected Behavior
> Metadata discovery should use a metadata-only query and must not execute the
> full user query.
> h3. Current Behavior
> Full user query is executed to obtain metadata.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
