Storm users: I am just getting my feet wet with Storm, and have something of a noobie question that I haven't found an answer to yet. I am wondering if there is a canonical idiom/pattern for avoid duplicate tuples when using parallelism in Spouts such that multiple Spout instances are accessing the same data source.
An example of what I mean: If I have a FileReaderSpout that reads from some file, and parallelism set to "3", is there a clean way to make sure that each Spout instance returns a unique set of tuples relative to the input data? In the old days, when I was writing MPI code, we had a process ID in each process that we could use to partition data. Is there any analogous mechanism in Storm? Or a better way altogether? Or is it best to partition the data "upstream" of Storm and avoid this at the Spout level? Thanks, Phil --- This message optimized for indexing by NSA PRISM
