I haven't found a good description on this setting and the costs in setting it too high. Hope somebody can explain.
I have about a year's worth of data partitioned by date. Using 10 nodes and setting xcievers to 5000, I can only save into 100 or so partitions. As a result, I have to do 4 rounds of saving data into the underlying partitioned table (in s3). That's pretty slow. Should I just set xcievers to 1M or will hadoop crash a result? Is each xciever really a separate thread? When will the spelling be corrected? :) Thanks a bunch!
