Yes, it logically can if there have been as many transactions (its a very very large number to reach though).
Long.MAX_VALUE is (2^63 - 1) or 9223372036854775807. I hacked up my local NN's txids manually to go very large (close to max) and decided to try out if this causes any harm. I basically bumped up the freshly formatted starting txid to 9223372036854775805 (and ensured image references the same): ➜ current ls VERSION fsimage_9223372036854775805.md5 fsimage_9223372036854775805 seen_txid ➜ current cat seen_txid 9223372036854775805 NameNode started up as expected. 13/06/25 18:30:08 INFO namenode.FSImage: Image file of size 129 loaded in 0 seconds. 13/06/25 18:30:08 INFO namenode.FSImage: Loaded image for txid 9223372036854775805 from /temp-space/tmp-default/dfs-cdh4/name/current/fsimage_9223372036854775805 13/06/25 18:30:08 INFO namenode.FSEditLog: Starting log segment at 9223372036854775806 I could create a bunch of files and do regular ops (counting to much after the long max increments). I created over 100 files, just to make it go well over the Long.MAX_VALUE. Quitting NameNode and restarting fails though, with the following error: 13/06/25 18:31:08 FATAL namenode.NameNode: Exception in namenode join java.io.IOException: Gap in transactions. Expected to be able to read up until at least txid 9223372036854775806 but unable to find any edit logs containing txid -9223372036854775808 So it looks like it cannot currently handle an overflow. I've filed https://issues.apache.org/jira/browse/HDFS-4936 to discuss this. I don't think this is of immediate concern though, so we should be able to address it in future (unless there's parts of the code which already are preventing reaching this number in the first place - please do correct me if there is such a part). On Tue, Jun 25, 2013 at 3:09 PM, Azuryy Yu <azury...@gmail.com> wrote: > Hi dear All, > > It's long type for the txid currently, > > FSImage.java: > > boolean loadFSImage(FSNamesystem target, MetaRecoveryContext recovery) > throws IOException{ > > editLog.setNextTxId(lastAppliedTxId + 1L); > } > > Is it possible that (lastAppliedTxId + 1L) exceed Long.MAX_VALUE ? -- Harsh J
