Hi Penghui, sorry I missed the discussion of this PIP on the mailing list. Have you considered including the following time-related properties in the topic stats? • Topic loaded time: The time when the topic was loaded on the broker. This complements the semantics of the byte/message counters, which are maintained in memory. These counters reset after a topic reload or broker restart. • Subscription creation time: Indicates when the subscription was originally created. • Subscription loaded time: The time when the subscription was loaded on the broker. Since BytesOutCounter and MsgOutCounter are also in-memory metrics, this would help clarify their scope. By default, this value could be defined as MAX(topic-loaded-time, subscription-creation-time).
Regards, Yike ________________________________ From: Haiting Jiang <jianghait...@gmail.com> Sent: Monday, June 30, 2025 19:46 To: dev@pulsar.apache.org <dev@pulsar.apache.org> Subject: Re: [DISCUSS] PIP-431: Add Creation and Last Publish Timestamps to Topic Stats +1 It looks great to have these topic attributes. Thanks, Haiting On Fri, Jun 27, 2025 at 3:03 AM PengHui Li <peng...@apache.org> wrote: > > Hi All, > > I have drafted a propoal to enhance topic observability by adding > topicCreationTimeStamp and lastPublishTimeStamp to the topic stats API. > > *Motivation:* Currently, it is difficult to programmatically determine a > topic's age or its most recent message activity. This hinders essential > operational tasks such as identifying and cleaning up stale topics, > performing compliance audits, and gaining quick troubleshooting insights. > Existing metrics like lastLedgerCreatedTimestamp or rate-based metrics are > not suitable for these lifecycle management use cases. > > *Proposed Solution:* This PIP proposes adding two new fields to the > TopicStats object: > > 1. > > *topicCreationTimeStamp*: This value will be retrieved directly from the > ctime attribute of the topic's underlying metadata node ( > org.apache.pulsar.metadata.api.Stat). This approach is efficient and > provides an accurate creation timestamp for all topics, including those > created before this feature's deployment. > 2. > > *lastPublishTimeStamp*: This will be maintained as a high-performance, > in-memory long value on the owning broker, updated on every successful > message publish. To ensure durability across broker restarts or topic > reloads, the value will be recovered from the last entry in the managed > ledger when the user gets topic internal stats. > > These changes are designed to be fully backward-compatible and have a > negligible performance impact on the message-publishing path. > > The full proposal is available for review here: > https://jpn01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fpulsar%2Fpull%2F24469&data=05%7C02%7C%7Cfed4311a183c4fdd06cb08ddb7cbd505%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C638868808259224582%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=7MOdmk4RQq9dM%2FUlBS%2FlFI2Iw2ka8TNcSTW2ofarCTs%3D&reserved=0<https://github.com/apache/pulsar/pull/24469> > > I welcome any feedback, questions, or suggestions you may have. > > Thanks, > > Penghui
