Bookkeeper's ledger is a distributed WAL with random reads support.
ledger.readAsync(long firstEntry, long lastEntry) works if firstEntry ==
lastEntry
With that one can pipeline reads of as many entries in parallel as needed,
no BK API changes needed for such a niche case.
Reading backwards may ba
Considering topic as a timeseries store it enables us to read forward &
backward stored messages using limited since/until windows.
On 3/15/23 05:34, Haiting Jiang wrote:
Here, the goal is to read a topic from a known messageId to previous messages
until a constraint depending on read messages.
> Here, the goal is to read a topic from a known messageId to previous messages
> until a constraint depending on read messages.
I get your point, but can you give a more detailed use case?
Thanks,
Haiting
On Tue, Mar 14, 2023 at 6:40 PM Alexandre DUVAL
wrote:
>
> Hi,
>
> The way you suggest wo
Hi,
The way you suggest works when we know how many messages we want to
read. It can be a nice feature too, but this is not my case here. Here,
the goal is to read a topic from a known messageId to previous messages
until a constraint depending on read messages. It looks
hasPreviousMessage()
Hi Kannar,
+1 to find the position first and then read like normal as mentioned
by Yong and Michael.
Another problem of reading reverse is that it would break all the
read ahead techniques in the storage and result in very poor
performance.
> This would work but it will need something to store
> Have you looked at the seek implementation to see if it would be
feasible to extend the implementation and add a method to "seekBefore"
a message id in the way you described?
I think it's not very feasible for this case. Seeking before can lead
to consumer reconnection, which can cause significa
Kannar,
Why not find the stop position first, then read the message
until a given position?
Does the stop position change dynamically? You only know
it once you meet it?
Yong
On Wed, 8 Mar 2023 at 21:37, Alexandre DUVAL
wrote:
> Hi Michael,
>
> This would work but it will need something to s
Hi Michael,
This would work but it will need something to store every messages read
to reverse them before answer which can be heavy in RAM usages. The key
point of the future is to read message by message from a MessageId to
past with stop read possible conditions.
Best,
Kannar
On 3/7/23
> The goal is to start from a known MessageId and read the N message
> before this MessageId.
Have you looked at the seek implementation to see if it would be
feasible to extend the implementation and add a method to "seekBefore"
a message id in the way you described? I haven't considered all of t
Hi Yong,
The goal is to start from a known MessageId and read the N message
before this MessageId.
Best,
Kannar
On 3/7/23 01:53, Yong Zhang wrote:
Hi Kannar,
Just interested in what exactly your case.
Why do you need to read messages in a reversed order? What is your case?
Best,
Yong
O
Hi Kannar,
Just interested in what exactly your case.
Why do you need to read messages in a reversed order? What is your case?
Best,
Yong
On Mon, 6 Mar 2023 at 23:37, Alexandre DUVAL wrote:
> Hi,
>
> I'm wondering if it is possible to introduce a new feature on Pulsar
> which will enable user
Hi,
I'm wondering if it is possible to introduce a new feature on Pulsar
which will enable users to read topic from a defined MessageId to
previous messages until the begin of the topic.
I tried to use Pulsar SQL but it requires so much RAM even for little
queries (due to Presto design).
C
12 matches
Mail list logo
