Hi Tao, We are planning to get a PIP out in this quarter. Regards On Mon, Jul 22, 2024 at 1:35 PM Tao Jiuming <dao...@apache.org> wrote:
> Hi, > > How's the proposal going? > > On 2024/01/19 06:28:14 Girish Sharma wrote: > > Hello everyone, > > > > A a true cloud native platform, which supports scale up and scale down, I > > feel like there is a need to be able to reduce partition count in pulsar > to > > truly achieve a scale down after events like sales (akin to black friday, > > etc) or huge temporary publish burst due to backfill. > > > > I looked through the archives (upto 2021) and did not find any prior > > discussion on the same topic. > > > > I have given this an initial thought to figure out what would it need to > > support such a feature in the lowest footprint possible. I am attaching > the > > document explaining the need, requirements and initial high level details > > [0]. What I would like is to understand if the community also finds this > > feature helpful and does the approach described in the document have some > > fatal flaw? Summarizing the approach here as well: > > > > - Introduce an ability to convert a normal topic object into a > read-only > > topic via admin api and an additional partitioned-topic metadata > property > > (just like shadow source, etc) > > - Add logic to block produce but allow new consumers and dispatch call > > based on this flag > > - Add logic in GC to clean out read only topics when all of their > > ledgers expire (TTL/retention) > > > > Goal is that there is no data movement involved and no impact on existing > > partitions during this scale down. > > > > Looking forward to the discussion. > > > > [0] > > > https://docs.google.com/document/d/1sbGQSwDihQftIRsxAXg5Zm4uxKQ0kRk9HadKYRFTswI/edit?usp=sharing > > > > Regards > > -- > > Girish Sharma > > > -- Girish Sharma
