Hi Ozone devs,
Recently our team have noticed a problem in Ozone. When a large number of
clients are writing
data to Ozone simultaneously, they will face ContainerNotOpenException very
often.
The root cause is, when client allocates a block from OM, the space is not
reserved in the container.
Many clients may got blocks from a same container, which does not have space to
hold all of them.
Finally when the container becomes full, the ContainerNotOpenException will be
thrown.
Based on my understanding, I have created a proposal to reserve space for
allocated blocks.
https://docs.google.com/document/d/1NjhBhtto8cuGXnYhYToOLJyD5kzn65UVG5-scGhKaw0/
Please leave comments if I missed some points or if you have a better idea.
PS: I have another idea which is less complicated but is incompatible with the
current design.
The basic idea is writing blocks to a temp storage and assembling containers in
the background.
Regards,
Kaijie Chen
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@ozone.apache.org
For additional commands, e-mail: dev-h...@ozone.apache.org
