We found that for some tasks, the JM memory continued to increase. I set the parameter of state.backend.fs.memory-threshold to 0, and the JM memory would no longer increase, but many small files might be written in this way. Does the community have any optimization plan for this area?
- Problems with the state.backend.fs.memory-threshold parameter rui chen
