Any other suggestion? JV, Sijie? My current idea for bp14 is to have a client driven scan.
Enrico Il ven 13 apr 2018, 10:22 Enrico Olivelli <eolive...@gmail.com> ha scritto: > > > Il gio 12 apr 2018, 19:59 Sam Just <sj...@salesforce.com> ha scritto: > >> IIRC, InterleavedLedgerStorage has for each ledger an index file >> mapping the entries to entry logger offsets, you could probably scan >> that directly (particularly if you included a lower ?bound -- probably >> the client's current idea of the LAC). >> -Sam >> > > Thank you Sam, > I think that the major point is about introducing a standard interface to > be implemented by LedgerStorage, now we already have 3 implementations. > > Options are: > 1) let the client drive the scan > 2) add a scan() method to LedgerStorage > 3) add a getMaxEntry() to LedgerStorage > > For bp14 I just need 1) or 3) but 1) is also a protocol issue and so it is > more expensive for long term maintenance > > Enrico > >> >> On Thu, Apr 12, 2018 at 12:31 AM, Enrico Olivelli <eolive...@gmail.com> >> wrote: >> > Hi BookKeepers, >> > during implementation of BP-14 I am facing a problem so I am asking for >> > suggestions. >> > >> > My need is to be able to know the list of all entries stored on a >> > LedgerStorage given a ledgerId. >> > >> > Scanning from 0 to LedgerStorage#getLastAddConfirmed() does not seem to >> > work because we have to deal with WriteAdvHandle, so there can be >> temporary >> > "gaps" in the sequence of entries. >> > >> > I can have a writer which writes entries 0,1,5,6,7. Its LAC will be at >> most >> > 1 as entries 2,3,4 are not written yet. >> > I need on the bookie to able to know that entries 0, 1, 5, 6, 7 are >> stored >> > on LedgerStorage. >> > >> > I cannot issue a scan from 0 to Long.MAX_VALUE, my current 'solution' >> it to >> > make the client (writer) send the 'maximum entry id' and perform a scan >> > from 0 to maxEntryId. >> > In the example the writer will send a forceLedger RPC with maxEntryId = >> 7. >> > >> > This is need only for recoveries are bookie restart because I have to >> > reconstruct the knowledge about which entries have been persisted >> durably >> > on the Bookie. >> > >> > I am not very expert about LedgerStorage implementations, and I don't >> know >> > if it would be feasible to have such 'scan all entries' method. >> > >> > This is the code I am talking about >> > >> https://github.com/apache/bookkeeper/pull/1317/files#diff-3b81b1c90d1f51017627b3c032676168R1210 >> > >> > Any help is really appreciated >> > Enrico >> > -- > > > -- Enrico Olivelli > -- -- Enrico Olivelli
