Jimmy, What's the exact command that produced this trace? Are you saying that the 16-second wait in your trace what times out in your CREATE TABLE statements?
Jim Witschey Software Engineer in Test | jim.witsc...@datastax.com On Sun, Apr 19, 2015 at 7:13 PM, Jimmy Lin <y2klyf+w...@gmail.com> wrote: > hi, > we have some unit tests that run parallel that will create tmp keyspace, and > tables and then drop them after tests are done. > > From time to time, our create table statement run into "All hosts(s) for > query failed... Timeout during read" (from datastax driver) error. > > We later turn on tracing, and record something in the following. > See below between "===" , Native_Transport-Request thread and MigrationStage > thread, there was like 16 seconds doing something. > > Any idea what that 16 seconds Cassandra was doing? We can work around that > but increasing our datastax driver timeout value, but wondering if there is > actually better way to solve this? > > thanks > > > > ---------------- tracing ---------- > > > 5872bf70-e6e2-11e4-823d-93572f3db015 | 58730d97-e6e2-11e4-823d-93572f3db015 > | > Key cache hit for sstable 95588 | 127.0.0.1 | 1592 | > Native-Transport-Requests:102 > 5872bf70-e6e2-11e4-823d-93572f3db015 | 58730d98-e6e2-11e4-823d-93572f3db015 > | Seeking > to partition beginning in data file | 127.0.0.1 | 1593 | > Native-Transport-Requests:102 > 5872bf70-e6e2-11e4-823d-93572f3db015 | 58730d99-e6e2-11e4-823d-93572f3db015 > | Merging > data from memtables and 3 sstables | 127.0.0.1 | 1595 | > Native-Transport-Requests:102 > > ===================== > 5872bf70-e6e2-11e4-823d-93572f3db015 | 58730d9a-e6e2-11e4-823d-93572f3db015 > | > Read 3 live and 0 tombstoned cells | 127.0.0.1 | 1610 | > Native-Transport-Requests:102 > 5872bf70-e6e2-11e4-823d-93572f3db015 | 62364a40-e6e2-11e4-823d-93572f3db015 > | Executing seq scan across 1 sstables for > (min(-9223372036854775808), min(-9223372036854775808)] | 127.0.0.1 | > 16381594 | MigrationStage:1 > ===================== > > 5872bf70-e6e2-11e4-823d-93572f3db015 | 62364a41-e6e2-11e4-823d-93572f3db015 > | Seeking > to partition beginning in data file | 127.0.0.1 | 16381782 | > MigrationStage:1 > 5872bf70-e6e2-11e4-823d-93572f3db015 | 62364a42-e6e2-11e4-823d-93572f3db015 > | > Read 0 live and 0 tombstoned cells | 127.0.0.1 | 16381787 | > MigrationStage:1 > 5872bf70-e6e2-11e4-823d-93572f3db015 | 62364a43-e6e2-11e4-823d-93572f3db015 > | Seeking > to partition beginning in data file | 127.0.0.1 | 16381789 | > MigrationStage:1 > 5872bf70-e6e2-11e4-823d-93572f3db015 | 62364a44-e6e2-11e4-823d-93572f3db015 > | > Read 0 live and 0 tombstoned cells | 127.0.0.1 | 16381791 | > MigrationStage:1 > 5872bf70-e6e2-11e4-823d-93572f3db015 | 62364a45-e6e2-11e4-823d-93572f3db015 > | Seeking > to partition beginning in data file | 127.0.0.1 | 16381792 | > MigrationStage:1 > 5872bf70-e6e2-11e4-823d-93572f3db015 | 62364a46-e6e2-11e4-823d-93572f3db015 > | > Read 0 live and 0 tombstoned cells | 127.0.0.1 | 16381794 | > MigrationStage:1 > . > . > . >
