Normally, wavelet updates and commit notices go out live (or with minimal
delay) so temporarily queueing wavelet updates isn't an issue. But if a
network connection gets congested, or the remote server goes down for a
while then the server cannot queue updates indefinitely. At some point it
needs to drop unsent updates from a domain's outbound queue. If this happens
the one thing that cannot get dropped is the latest commit notice for each
wavelet. A remote server can always recover lost updates with
a history request. But in order for it to know that there is history it
needs, it must receive a new update or commit notice. In order for this to
happen, the local server MUST keep trying to sent commit notices until it
succeeds. And, in order for the local server to know that there are commit
notices it needs to send, the commit notices must be persisted and only
removed when they are acknowledged by the remote server. If the state of
commit notice acknowledgement are not persisted, then that state will be
lost if the local server restartes and it will have no way of knowing which
commit notices need to be to be sent to which domains.
This highlights two problems. One, the federation protocol doesn't include a
commit notice acknowledgment message. And, two, the server doesn't keep
track of commit notice acknowledgement state.
In the case of the federation protocol, only the XMPP variant would need an
actual acknowledgement message, the HTTP variant has
inherent acknowledgment built in via the POST response message (though the
spec should probably state that the server shouldn't reply to commit notices
until the commit notice has been persisted by the receiver).
Regarding commit notice state, the server needs to, for each wavelet, record
for each domain the last committed version that the domain
should receive and the last committed version acknowledged by the domain.
The cost to record and update this information should be low, and the cost
to discover this information should be scalable. So, for example, if the
FileDeltasStore where modified so that it records per domain commit notice
state in the same dir as the wavelet, then it would not be scalable as the
server would need to iterate over every wavelet at startup in order to
discover which commit notices still need to be sent.
>From an API point of view, the DeltaAccess interface needs something like:
void acknowledgeCommitNotice(String domain, HashedVersion
committedVersion) throws PersistenceException;
List<String> getDomainsWithUnacknowledgedCommitNotices() throws
PersistenceException;
HashedVersion getLatestCommitNoticeForDomain(String domain) throws
PersistenceException;
And the DeltaStore interface needs something like:
ExceptionalIterator<WaveletName, PersistenceException>
getWaveletsWithUnacknowledgedCommitNoticesForDomain(String domain)
throws PersistenceException;
A side issue related to commit notices is that other internal WiaB
components (the wavelet indexer for example) need to process deltas from the
wavebus, but also need to discover at start-up which wavelet deltas have not
yet been processed. When there are few waves the cost to iterate over every
wavelet is fairly small. But it's not practical to iterate over every
wavelet, at start-up, when there a 100K+ wavelets.
So a more generic commit notice store interface is needed for use by
internal components:
public interface CommitLogStore {
interface CommitNotice {
WaveletName getWaveletName();
HashedVersion getCommittedVersion();
}
/**
* Called by DeltaStore.DeltaAccess.append() prior to returning.
* This method should prevent concurrent access for a single wavelet, but
support concurrent access for separate wavelets.
*/
void recordCommitNotice(WaveletName waveletName, HashedVersion
commitedVersion) throws PersistenceException;
/**
* This will return an iterator that will iterate over all
non-acknowledged commit notices.
* If the name specified has never acknowledged a commit notice, then the
iterator will iterate of every commit notice recorded.
*/
ExceptionalInterface<CommitNotice, PersistenceException>
getWaveletsWithUnacknowledgedCommitsForName(String name) throws
PersistenceException;
/**
* A single commit notice is acknowledged. If the version specified is
less than the greatest version recorded
* then the next call to getWaveletsWithUnacknowledgedCommitsForName will
return this wavelet again.
*/
void acknowledgeCommitNotice(String name, WaveletName waveletName,
HashedVersion commitedVersion);
}
I'm advocating two separate mechanisms for managing commit notices because
of the slightly different semantics. Internal components will see every
delta for every wavelet. Remote domains will only see deltas for certain
wavelets and only while some user form that domain is a participant on the
wavelet.
-Tad
