SaintBacchus opened a new issue, #9724: URL: https://github.com/apache/incubator-doris/issues/9724
### Search before asking - [X] I had searched in the [issues](https://github.com/apache/incubator-doris/issues?q=is%3Aissue) and found no similar issues. ### Description Now the FE ui only can show the last 100 query in the web page(with the request `/rest/v1/query_profile`).  100 may be not enough for the real bussiness scenario. But increase this hard code number will cause more memory use in the fe java process. ### Solution So a nature idea is to store it in the persist storage system and browser the fe ui by reading the query profile data from storage. The storage system will choose the InMemory to keep the same behaviour with now and DorisNative which is suitable for **frequent insert** and **not frequent point loop up** and **timed range scan**. In my solution, I will dive thie impl into 4 Step: - [ ] Abstract an interface named `ProfileStorage` with `write` and `read` method and impl the `InMemoryProfileStorage` which will keep the same behaviour with now. - [ ] Impl the `DorisNativeProfileStorage` to storage the records in a doris olap table. - [ ] Improve the insert performance in `DorisNativeProfileStorage` by use wal to submit all request in batch - [ ] Modify the Web UI and separate the running and finished job to avoid update semantic for performance. ### Are you willing to submit PR? - [X] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
