[
https://issues.apache.org/jira/browse/IGNITE-7526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16416350#comment-16416350
]
Denis Magda commented on IGNITE-7526:
-------------------------------------
[~vozerov], according to the summary the optimizations will be applied to the
client applications (who are reducers in 95% of the time). However, don't we
need to do the same on the mappers side? I do remember the reportings when
servers went down just because someone executed "SELECT * FROM table;" query.
> SQL: Introduce memory region for reducer merge results with disk offload
> ------------------------------------------------------------------------
>
> Key: IGNITE-7526
> URL: https://issues.apache.org/jira/browse/IGNITE-7526
> Project: Ignite
> Issue Type: Task
> Components: sql
> Reporter: Vladimir Ozerov
> Priority: Major
>
> Currently all results received from map nodes are stored inside reducer's
> heap memory. What is worse, in case of complex queries, such as having sorts
> or groupings, need to collect all results from mappers first before final
> processing could be applied. In case of big results set (or intermediate
> results) this could easily lead to OOME on reducer.
> To mitigate this we should introduce special memory area where intermediate
> results could be stored. All final processing should be stored in the same
> area as well. This area should be of limited size and should be able to
> offload results to disk in case of overflow.
> We could start with our B+Tree and free list and store results in some K-V
> form.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
