Hi Xiangying, If the change applies to users, we should start with a proposal to provide the background, the goal and the related API changs.
Thanks, Penghui On Sun, Feb 26, 2023 at 11:38 PM Asaf Mesika <asaf.mes...@gmail.com> wrote: > On Fri, Feb 24, 2023 at 5:33 AM Xiangying Meng <xiangy...@apache.org> > wrote: > > > Hi Asaf, > > > How are those stats exposed to the user? > > IMO, these stats should be exposed to the user. > > The users can use it to check if the transaction snapshot is > > running correctly. > > > > > My question was "how" - prometheus, Pulsar REST API? > > I'm asking because this stats doesn't look very "metrics" in term of > structure / naming > > > Thanks, > > Xiangying > > > > On Wed, Feb 22, 2023 at 9:06 PM Asaf Mesika <asaf.mes...@gmail.com> > wrote: > > > > > How are those stats exposed to the user? > > > > > > > > > On Mon, Feb 20, 2023 at 6:01 AM Xiangying Meng <xiangy...@apache.org> > > > wrote: > > > > > > > Hi, Community, > > > > We plan to add an internal class to `TransactionBufferStats` to > record > > > the > > > > snapshot status uniformly. > > > > As we all know, the current transaction buffer(TB) filters the > messages > > > > sent using the aborted transaction by storing the aborted ID in TB. > > > > Then TB will periodically store these aborted txn IDs in a bookie > entry > > > in > > > > the form of snapshots so that TB can recover faster when recovering. > > > > But as more and more people use transactions, we found that in some > > > extreme > > > > cases, a bookie entry may not be able to store all aborted > transaction > > > IDs. > > > > So in PIP196 <https://github.com/apache/pulsar/issues/16913>, we > > > > implemented the multiple-snapshot function. > > > > As the transaction buffer snapshot mechanism becomes increasingly > > > complex, > > > > the only information related to the transaction snapshot is > > > > `lastSnapshotTimestamps`; That is not enough, we need to add more > info > > to > > > > record the snapshot stats. > > > > So I suggest adding an internal class SnapshotStats to > > > > TransactionBufferStats to record the snapshot status uniformly. > > > > > > > > The modification could be : > > > > ```java > > > > public class TransactionBufferStats { > > > > ... > > > > public long lastSnapshotTimestamps; > > > > ... > > > > } > > > > ``` > > > > ```java > > > > public class TransactionBufferStats { > > > > ... > > > > //public long lastSnapshotTimestamps; > > > > ... > > > > public SnapshotStats snapshotStats; > > > > > > > > public static class SnapshotStats { > > > > public long segmentsSize; > > > > > > > > public long unsealedAbortTxnIDs; > > > > > > > > > > > > public long lastSnapshotTimestamps; > > > > } > > > > } > > > > > > > > ``` > > > > Thanks. > > > > Xiangying > > > > > > > > > >
