Fault Tolerance Testing for Pulsar

2020-01-05 Thread Richard Yu
Hi all, I was just wondering on what forms of testing (i.e. fault injection) that had been performed on Pulsar. I'm interested because usually some certain frameworks (i.e. Jepsen) has been used to break distributed services on purpose. Therefore, I thought it would be good to know if Pulsar had

PIP-31: Add support for transactional messaging

2019-02-27 Thread Richard Yu
Hi all, I would like to create a PIP for issue #2664 on Github. The details of the PIP are below. I hope we could discuss this thoroughly. Cheers, Richard PIP-31: Add support for transactional messaging Motivation: Pulsar currently could improve upon their system of sending packets of data by i

Re: PIP-31: Add support for transactional messaging

2019-02-28 Thread Richard Yu
some > thoughts as well. If you can put up a google doc (make it world-editable), > Ivan can probably dump his thoughts there and we can finalize the > discussion and break down into tasks. So the whole community can actually > work together at collaborating this. > > Thanks, > Siji

Re: PIP-31: Add support for transactional messaging

2019-03-01 Thread Richard Yu
. On Thu, Feb 28, 2019 at 4:29 PM Richard Yu wrote: > Hi, > > Per request, I've created a doc so we could get some more input in an > organized manner: > > https://docs.google.com/document/d/1mSUsJvPgThnWizeQqljKU5244BMEiYHabA6OP6QEHmQ/edit?usp=sharing > > And

Re: PIP-31: Add support for transactional messaging

2019-03-03 Thread Richard Yu
gt;>> the > > >>>>> txn ledger or actually copying the data to data ledger (depending > on > > >> the > > >>>>> size of data accumulate in the transaction). > > >>>>> 4) when abort, delete the txn ledger. No other addition

PIP-32: Auto-scaling up and down of partitions

2019-04-16 Thread Richard Yu
Hello all, I wish to introduce a new auto-scaling feature to Pulsar. The respective issue for this feature could be found in the design doc. Here is the link for those who are interested: https://docs.google.com/document/d/1vdHFbmDY5DodMMQqxBZLYpzfoEPh1ENP-kmEkJHYjq8/edit?usp=sharing Cheers, Ric