On Fri, Feb 24, 2023 at 5:33 AM Xiangying Meng <xiangy...@apache.org> wrote:
> Hi Asaf, > > How are those stats exposed to the user? > IMO, these stats should be exposed to the user. > The users can use it to check if the transaction snapshot is > running correctly. > > My question was "how" - prometheus, Pulsar REST API? I'm asking because this stats doesn't look very "metrics" in term of structure / naming > Thanks, > Xiangying > > On Wed, Feb 22, 2023 at 9:06 PM Asaf Mesika <asaf.mes...@gmail.com> wrote: > > > How are those stats exposed to the user? > > > > > > On Mon, Feb 20, 2023 at 6:01 AM Xiangying Meng <xiangy...@apache.org> > > wrote: > > > > > Hi, Community, > > > We plan to add an internal class to `TransactionBufferStats` to record > > the > > > snapshot status uniformly. > > > As we all know, the current transaction buffer(TB) filters the messages > > > sent using the aborted transaction by storing the aborted ID in TB. > > > Then TB will periodically store these aborted txn IDs in a bookie entry > > in > > > the form of snapshots so that TB can recover faster when recovering. > > > But as more and more people use transactions, we found that in some > > extreme > > > cases, a bookie entry may not be able to store all aborted transaction > > IDs. > > > So in PIP196 <https://github.com/apache/pulsar/issues/16913>, we > > > implemented the multiple-snapshot function. > > > As the transaction buffer snapshot mechanism becomes increasingly > > complex, > > > the only information related to the transaction snapshot is > > > `lastSnapshotTimestamps`; That is not enough, we need to add more info > to > > > record the snapshot stats. > > > So I suggest adding an internal class SnapshotStats to > > > TransactionBufferStats to record the snapshot status uniformly. > > > > > > The modification could be : > > > ```java > > > public class TransactionBufferStats { > > > ... > > > public long lastSnapshotTimestamps; > > > ... > > > } > > > ``` > > > ```java > > > public class TransactionBufferStats { > > > ... > > > //public long lastSnapshotTimestamps; > > > ... > > > public SnapshotStats snapshotStats; > > > > > > public static class SnapshotStats { > > > public long segmentsSize; > > > > > > public long unsealedAbortTxnIDs; > > > > > > > > > public long lastSnapshotTimestamps; > > > } > > > } > > > > > > ``` > > > Thanks. > > > Xiangying > > > > > >
