Sean, here it is : CREATE TABLE Keyspave.tblname ( user_id bigint, session_id text, application_guid text, last_access_time timestamp, login_time timestamp, status int, terminated_by text, update_time timestamp, PRIMARY KEY (user_id, session_id) ) WITH CLUSTERING ORDER BY (session_id ASC)
also they see timeouts with limit 11 as well, so is it better to remove with limit option ? or whats best to query such schema? On Fri, Oct 12, 2018 at 11:05 AM Durity, Sean R <sean_r_dur...@homedepot.com> wrote: > Cross-partition = multiple partitions > > > > Simple example: > > Create table customer ( > > Customerid int, > > Name text, > > Lastvisit date, > > Phone text, > > Primary key (customerid) ); > > > > Query > > Select customerid from customer limit 5000; > > > > The query is asking for 5000 different partitions to be selected across > the cluster. This is a very EXPENSIVE query for Cassandra, especially as > the number of nodes goes up. Typically, you want to query a single > partition. Read timeouts are usually caused by queries that are selecting > many partitions or a very large partition. That is why a schema for the > involved table could help. > > > > > > Sean Durity > > > > *From:* Abdul Patel <abd786...@gmail.com> > *Sent:* Friday, October 12, 2018 10:04 AM > *To:* user@cassandra.apache.org > *Subject:* [EXTERNAL] Re: Tracing in cassandra > > > > Cpuld you elaborate cross partition query? > > On Friday, October 12, 2018, Durity, Sean R <sean_r_dur...@homedepot.com> > wrote: > > I suspect you are doing a cross-partition query, which will not scale well > (as you can see). What is the schema for the table involved? > > > > > > Sean Durity > > > > *From:* Abdul Patel <abd786...@gmail.com> > *Sent:* Thursday, October 11, 2018 5:54 PM > *To:* a...@instaclustr.com > *Cc:* user@cassandra.apache.org > *Subject:* [EXTERNAL] Re: Tracing in cassandra > > > > Query : > > SELECT * FROM keysoace.tablenameWHERE user_id = 390797583 LIMIT 5000; > > -Error: ReadTimeout: Error from server: code=1200 [Coordinator node timed > out waiting for replica nodes' responses] message="Operation timed out - > received only 0 responses." info={'received_responses': 0, > 'required_responses': 1, 'consistency': 'ONE'} > > > > e70ac650-cd9e-11e8-8e99-15807bff4dfd | > e70bd7c0-cd9e-11e8-8e99-15807bff4dfd > | > Parsing SELECT * FROM keysoace.tablenameWHERE user_id = 390797583 LIMIT > 5000; | 10.54.145.32 | 4020 | > Native-Transport-Requests-3 > > e70ac650-cd9e-11e8-8e99-15807bff4dfd | > e70bfed0-cd9e-11e8-8e99-15807bff4dfd > | > Preparing statement | > 10.54.145.32 | 5065 | > Native-Transport-Requests-3 > > e70ac650-cd9e-11e8-8e99-15807bff4dfd | > e70c25e0-cd9e-11e8-8e99-15807bff4dfd | > > Executing > single-partition query on roles | 10.54.145.32 | 6171 > | ReadStage-2 > > e70ac650-cd9e-11e8-8e99-15807bff4dfd | > e70c4cf0-cd9e-11e8-8e99-15807bff4dfd > | > Acquiring sstable references | 10.54.145.32 | 6362 > | ReadStage-2 > > e70ac650-cd9e-11e8-8e99-15807bff4dfd | > e70c4cf1-cd9e-11e8-8e99-15807bff4dfd > | > Skipped 0/2 non-slice-intersecting sstables, included 0 due to tombstones | > 10.54.145.32 | 6641 | > ReadStage-2 > > e70ac650-cd9e-11e8-8e99-15807bff4dfd | > e70c4cf2-cd9e-11e8-8e99-15807bff4dfd > | > Key cache hit for sstable 346 | 10.54.145.32 | 6955 > | ReadStage-2 > > e70ac650-cd9e-11e8-8e99-15807bff4dfd | > e70c4cf3-cd9e-11e8-8e99-15807bff4dfd > | > Bloom filter allows skipping sstable 347 | 10.54.145.32 > | 7202 | ReadStage-2 > > e70ac650-cd9e-11e8-8e99-15807bff4dfd | > e70c7400-cd9e-11e8-8e99-15807bff4dfd > | > Merged data > from memtables and 2 sstables | 10.54.145.32 | 7386 > | ReadStage-2 > > e70ac650-cd9e-11e8-8e99-15807bff4dfd | > e70c7401-cd9e-11e8-8e99-15807bff4dfd > | > Read 1 live and 0 tombstone cells | 10.54.145.32 | 7519 > | ReadStage-2 > > e70ac650-cd9e-11e8-8e99-15807bff4dfd | > e70c7402-cd9e-11e8-8e99-15807bff4dfd > | > Executing single-partition query on roles | 10.54.145.32 | 7826 > | ReadStage-4 > > e70ac650-cd9e-11e8-8e99-15807bff4dfd | > e70c7403-cd9e-11e8-8e99-15807bff4dfd > | > Acquiring sstable references | 10.54.145.32 | 7924 > | ReadStage-4 > > e70ac650-cd9e-11e8-8e99-15807bff4dfd | > e70c7404-cd9e-11e8-8e99-15807bff4dfd > | > Skipped 0/2 non-slice-intersecting sstables, included 0 due to tombstones | > 10.54.145.32 | 8060 | > ReadStage-4 > > e70ac650-cd9e-11e8-8e99-15807bff4dfd | > e70c7405-cd9e-11e8-8e99-15807bff4dfd > | > Key cache hit for > sstable 346 | 10.54.145.32 | 8137 > | ReadStage-4 > > e70ac650-cd9e-11e8-8e99-15807bff4dfd | > e70c7406-cd9e-11e8-8e99-15807bff4dfd | > > Bloom > filter allows skipping sstable 347 | 10.54.145.32 | 8187 > | ReadStage-4 > > e70ac650-cd9e-11e8-8e99-15807bff4dfd | > e70c7407-cd9e-11e8-8e99-15807bff4dfd > | > Merged data from memtables and 2 sstables | 10.54.145.32 | 8318 > | ReadStage-4 > > e70ac650-cd9e-11e8-8e99-15807bff4dfd | > e70c9b10-cd9e-11e8-8e99-15807bff4dfd > | > Read 1 live and 0 tombstone cells | 10.54.145.32 | 8941 > | ReadStage-4 > > e70ac650-cd9e-11e8-8e99-15807bff4dfd | > e70cc220-cd9e-11e8-8e99-15807bff4dfd > | > Read-repair DC_LOCAL | 10.54.145.32 | 9468 | > Native-Transport-Requests-3 > > e70ac650-cd9e-11e8-8e99-15807bff4dfd | > e70cc221-cd9e-11e8-8e99-15807bff4dfd > | > reading data from /10.54.145.31 > <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.31&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=VFRGkFAneO-HMP4eVg8FrXVgiDy0bsL9Q2MS83r85Qw&e=> > | 10.54.145.32 | 9754 | > Native-Transport-Requests-3 > > e70ac650-cd9e-11e8-8e99-15807bff4dfd | > e70cc222-cd9e-11e8-8e99-15807bff4dfd | > > Executing > single-partition query on session | 10.54.145.32 | 10310 > | ReadStage-4 > > e70ac650-cd9e-11e8-8e99-15807bff4dfd | > e70ce930-cd9e-11e8-8e99-15807bff4dfd > | > reading digest from /10.54.101.56 > <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.101.56&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=KoAVTaVcog1srnn8npMVq8wpXmZq95ibi870ELyivGM&e=> > | 10.54.145.32 | 10343 | > Native-Transport-Requests-3 > > e70ac650-cd9e-11e8-8e99-15807bff4dfd | > e70ce931-cd9e-11e8-8e99-15807bff4dfd > | > Acquiring sstable references | 10.54.145.32 | 10356 > | ReadStage-4 > > e70ac650-cd9e-11e8-8e99-15807bff4dfd | > e70ce932-cd9e-11e8-8e99-15807bff4dfd > | > Key cache hit for sstable 68642 | 10.54.145.32 | 10445 > | ReadStage-4 > > e70ac650-cd9e-11e8-8e99-15807bff4dfd | > e70ce933-cd9e-11e8-8e99-15807bff4dfd > | > reading digest from /10.54.101.57 > <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.101.57&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=zYDbByQA6i7TWxl_e6119WE_qT7auXhn8qscmGcG40A&e=> > | 10.54.145.32 | 10480 | > Native-Transport-Requests-3 > > e70ac650-cd9e-11e8-8e99-15807bff4dfd | > e70ce934-cd9e-11e8-8e99-15807bff4dfd | > Bloom > filter allows skipping sstable 68397 | 10.54.145.32 | 10496 > | ReadStage-4 > > e70ac650-cd9e-11e8-8e99-15807bff4dfd | > e70ce935-cd9e-11e8-8e99-15807bff4dfd > | > Skipped 0/3 non-slice-intersecting sstables, included 0 due to tombstones | > 10.54.145.32 | 10547 | > ReadStage-4 > > e70ac650-cd9e-11e8-8e99-15807bff4dfd | > e70ce936-cd9e-11e8-8e99-15807bff4dfd > | > Sending READ message to /10.54.145.31 > <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.31&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=VFRGkFAneO-HMP4eVg8FrXVgiDy0bsL9Q2MS83r85Qw&e=> > | 10.54.145.32 | 10575 | > MessagingService-Outgoing-/10.54.145.31-Small > > e70ac650-cd9e-11e8-8e99-15807bff4dfd | > e70ce937-cd9e-11e8-8e99-15807bff4dfd > | > Key cache hit for sstable 68655 | 10.54.145.32 | 10631 > | ReadStage-4 > > e70ac650-cd9e-11e8-8e99-15807bff4dfd | > e70ce938-cd9e-11e8-8e99-15807bff4dfd > | > Sending READ message to /10.54.101.56 > <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.101.56&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=KoAVTaVcog1srnn8npMVq8wpXmZq95ibi870ELyivGM&e=> > | 10.54.145.32 | 10695 | > MessagingService-Outgoing-/10.54.101.56-Small > > e70ac650-cd9e-11e8-8e99-15807bff4dfd | > e70ce939-cd9e-11e8-8e99-15807bff4dfd > | > Sending READ > message to /10.54.101.57 > <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.101.57&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=zYDbByQA6i7TWxl_e6119WE_qT7auXhn8qscmGcG40A&e=> > | 10.54.145.32 | 10823 | > MessagingService-Outgoing-/10.54.101.57-Small > > e70ac650-cd9e-11e8-8e99-15807bff4dfd | > e70ce93a-cd9e-11e8-8e99-15807bff4dfd | > > Merged > data from memtables and 3 sstables | 10.54.145.32 | 10863 > | ReadStage-4 > > e70ac650-cd9e-11e8-8e99-15807bff4dfd | > e70ce93b-cd9e-11e8-8e99-15807bff4dfd > | > Read 13 live and 0 tombstone cells | 10.54.145.32 | 10914 > | ReadStage-4 > > e70ac650-cd9e-11e8-8e99-15807bff4dfd | > e70dac80-cd9e-11e8-9ad7-9f2cb164b2ba > | > READ message received from /10.54.145.32 > <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=> > | 10.54.145.31 | 883 | MessagingService-Incoming-/ > 10.54.145.32 > <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=> > > e70ac650-cd9e-11e8-8e99-15807bff4dfd | > e70dac80-cd9e-11e8-ba8b-bbf19da89f90 > | > READ message received from /10.54.145.32 > <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=> > | 10.54.101.57 | 803 | MessagingService-Incoming-/ > 10.54.145.32 > <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=> > > e70ac650-cd9e-11e8-8e99-15807bff4dfd | > e70dd390-cd9e-11e8-bed7-f13ab36bfa4c > | > READ message received from / > 10.54.145.32 > <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=> > | 10.54.101.56 | 961 | MessagingService-Incoming-/ > 10.54.145.32 > <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=> > > e70ac650-cd9e-11e8-8e99-15807bff4dfd | > e70dfaa0-cd9e-11e8-9ad7-9f2cb164b2ba | > > Executing > single-partition query on session | 10.54.145.31 | 3208 > | ReadStage-3 > > e70ac650-cd9e-11e8-8e99-15807bff4dfd | > e70dfaa0-cd9e-11e8-ba8b-bbf19da89f90 > | > Executing single-partition query on session | 10.54.101.57 | 3258 > | ReadStage-3 > > e70ac650-cd9e-11e8-8e99-15807bff4dfd | > e70dfaa1-cd9e-11e8-9ad7-9f2cb164b2ba > | > Acquiring sstable references | 10.54.145.31 | 3483 > | ReadStage-3 > > e70ac650-cd9e-11e8-8e99-15807bff4dfd | > e70dfaa1-cd9e-11e8-ba8b-bbf19da89f90 > | > Acquiring sstable references | 10.54.101.57 | 3549 > | ReadStage-3 > > e70ac650-cd9e-11e8-8e99-15807bff4dfd | > e70e21b0-cd9e-11e8-9ad7-9f2cb164b2ba > | > Key cache hit for sstable 74713 | > 10.54.145.31 | 3732 | > ReadStage-3 > > e70ac650-cd9e-11e8-8e99-15807bff4dfd | > e70e21b0-cd9e-11e8-ba8b-bbf19da89f90 | > > Key > cache hit for sstable 85409 | 10.54.101.57 | 3978 > | ReadStage-3 > > e70ac650-cd9e-11e8-8e99-15807bff4dfd | > e70e21b0-cd9e-11e8-bed7-f13ab36bfa4c > | > Executing single-partition query on session | 10.54.101.56 | 3497 > | ReadStage-3 > > e70ac650-cd9e-11e8-8e99-15807bff4dfd | > e70e21b1-cd9e-11e8-9ad7-9f2cb164b2ba > | > Key cache hit for sstable 74445 | 10.54.145.31 | 3893 > | ReadStage-3 > > e70ac650-cd9e-11e8-8e99-15807bff4dfd | > e70e21b1-cd9e-11e8-ba8b-bbf19da89f90 > | > Bloom filter allows skipping sstable 85201 | 10.54.101.57 | 4193 > | ReadStage-3 > > e70ac650-cd9e-11e8-8e99-15807bff4dfd | > e70e21b2-cd9e-11e8-9ad7-9f2cb164b2ba > | > Bloom filter allows skipping sstable 74383 | 10.54.145.31 | > 3975 | ReadStage-3 > > e70ac650-cd9e-11e8-8e99-15807bff4dfd | > e70e21b2-cd9e-11e8-ba8b-bbf19da89f90 > | > Skipped 0/3 non-slice-intersecting sstables, included > 0 due to tombstones | 10.54.101.57 | 4354 > | ReadStage-3 > > e70ac650-cd9e-11e8-8e99-15807bff4dfd | > e70e21b3-cd9e-11e8-9ad7-9f2cb164b2ba | > Skipped > 0/3 non-slice-intersecting sstables, included 0 due to tombstones | > 10.54.145.31 | 4142 | > ReadStage-3 > > e70ac650-cd9e-11e8-8e99-15807bff4dfd | > e70e21b3-cd9e-11e8-ba8b-bbf19da89f90 > | > Key cache hit for sstable 85418 | 10.54.101.57 | 4512 > | ReadStage-3 > > e70ac650-cd9e-11e8-8e99-15807bff4dfd | > e70e48c0-cd9e-11e8-ba8b-bbf19da89f90 > | > Merged data from memtables and 3 sstables | 10.54.101.57 | 4989 > | ReadStage-3 > > e70ac650-cd9e-11e8-8e99-15807bff4dfd | > e70e48c0-cd9e-11e8-bed7-f13ab36bfa4c > | > Acquiring sstable references | 10.54.101.56 | 3751 > | ReadStage-3 > > e70ac650-cd9e-11e8-8e99-15807bff4dfd | > e70e48c1-cd9e-11e8-ba8b-bbf19da89f90 > | > Read 13 live and 0 tombstone > cells | 10.54.101.57 | 5336 | > ReadStage-3 > > e70ac650-cd9e-11e8-8e99-15807bff4dfd | > e70e48c1-cd9e-11e8-bed7-f13ab36bfa4c | > > Key > cache hit for sstable 101147 | 10.54.101.56 | 4091 > | ReadStage-3 > > e70ac650-cd9e-11e8-8e99-15807bff4dfd | > e70e48c2-cd9e-11e8-ba8b-bbf19da89f90 > | > Read 13 live and 0 tombstone cells | 10.54.101.57 | 5586 > | ReadStage-3 > > e70ac650-cd9e-11e8-8e99-15807bff4dfd | > e70e6fd0-cd9e-11e8-ba8b-bbf19da89f90 > | > Enqueuing response to /10.54.145.32 > <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=> > | 10.54.101.57 | 5745 | > ReadStage-3 > > e70ac650-cd9e-11e8-8e99-15807bff4dfd | > e70e6fd1-cd9e-11e8-ba8b-bbf19da89f90 > | > Sending REQUEST_RESPONSE message to /10.54.145.32 > <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=> > | 10.54.101.57 | 6473 | > MessagingService-Outgoing-/10.54.145.32-Small > > e70ac650-cd9e-11e8-8e99-15807bff4dfd | > e70e96e0-cd9e-11e8-8e99-15807bff4dfd > | > REQUEST_RESPONSE message received from /10.54.101.57 > <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.101.57&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=zYDbByQA6i7TWxl_e6119WE_qT7auXhn8qscmGcG40A&e=> > | 10.54.145.32 | 57 | MessagingService-Incoming-/ > 10.54.101.57 > <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.101.57&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=zYDbByQA6i7TWxl_e6119WE_qT7auXhn8qscmGcG40A&e=> > > e70ac650-cd9e-11e8-8e99-15807bff4dfd | > e70e96e0-cd9e-11e8-9ad7-9f2cb164b2ba | > Merged > data from memtables and 3 sstables | 10.54.145.31 | 6600 > | ReadStage-3 > > e70ac650-cd9e-11e8-8e99-15807bff4dfd | > e70e96e0-cd9e-11e8-bed7-f13ab36bfa4c > | > Key cache hit for sstable 100896 | 10.54.101.56 | 6496 > | ReadStage-3 > > e70ac650-cd9e-11e8-8e99-15807bff4dfd | > e70e96e1-cd9e-11e8-8e99-15807bff4dfd > | > Processing response from /10.54.101.57 > <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.101.57&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=zYDbByQA6i7TWxl_e6119WE_qT7auXhn8qscmGcG40A&e=> > | 10.54.145.32 | 587 | > RequestResponseStage-1 > > e70ac650-cd9e-11e8-8e99-15807bff4dfd | > e70e96e1-cd9e-11e8-9ad7-9f2cb164b2ba > | > Read 13 live and 0 tombstone cells | 10.54.145.31 | 6892 > | ReadStage-3 > > e70ac650-cd9e-11e8-8e99-15807bff4dfd | > e70e96e2-cd9e-11e8-9ad7-9f2cb164b2ba > | > Enqueuing response to /10.54.145.32 > <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=> > | 10.54.145.31 | 7038 | > ReadStage-3 > > e70ac650-cd9e-11e8-8e99-15807bff4dfd | > e70ebdf0-cd9e-11e8-8e99-15807bff4dfd > | > REQUEST_RESPONSE message > received from /10.54.145.31 > <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.31&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=VFRGkFAneO-HMP4eVg8FrXVgiDy0bsL9Q2MS83r85Qw&e=> > | 10.54.145.32 | 15 | MessagingService-Incoming-/ > 10.54.145.31 > <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.31&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=VFRGkFAneO-HMP4eVg8FrXVgiDy0bsL9Q2MS83r85Qw&e=> > > e70ac650-cd9e-11e8-8e99-15807bff4dfd | > e70ebdf0-cd9e-11e8-9ad7-9f2cb164b2ba | > > Sending > REQUEST_RESPONSE message to /10.54.145.32 > <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=> > | 10.54.145.31 | 7641 | > MessagingService-Outgoing-/10.54.145.32-Small > > e70ac650-cd9e-11e8-8e99-15807bff4dfd | > e70ebdf0-cd9e-11e8-bed7-f13ab36bfa4c > | > Bloom filter allows skipping sstable 100887 | 10.54.101.56 | 6652 > | ReadStage-3 > > e70ac650-cd9e-11e8-8e99-15807bff4dfd | > e70ebdf1-cd9e-11e8-8e99-15807bff4dfd > | > Processing response from /10.54.145.31 > <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.31&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=VFRGkFAneO-HMP4eVg8FrXVgiDy0bsL9Q2MS83r85Qw&e=> > | 10.54.145.32 | 95 | > RequestResponseStage-1 > > e70ac650-cd9e-11e8-8e99-15807bff4dfd | > e70ebdf1-cd9e-11e8-bed7-f13ab36bfa4c > | > Skipped 0/4 non-slice-intersecting sstables, included 0 due to tombstones | > 10.54.101.56 | 6828 | > ReadStage-3 > > e70ac650-cd9e-11e8-8e99-15807bff4dfd | > e70ebdf2-cd9e-11e8-bed7-f13ab36bfa4c > | > Key cache hit for sstable > 101149 | 10.54.101.56 | 7005 | > ReadStage-3 > > e70ac650-cd9e-11e8-8e99-15807bff4dfd | > e70f3320-cd9e-11e8-bed7-f13ab36bfa4c | > > Merged > data from memtables and 4 sstables | 10.54.101.56 | 9897 > | ReadStage-3 > > e70ac650-cd9e-11e8-8e99-15807bff4dfd | > e70f8140-cd9e-11e8-bed7-f13ab36bfa4c > | > Read 13 live and 0 tombstone cells | 10.54.101.56 | 11969 > | ReadStage-3 > > e70ac650-cd9e-11e8-8e99-15807bff4dfd | > e70f8141-cd9e-11e8-bed7-f13ab36bfa4c > | > Read 13 live and 0 tombstone cells | 10.54.101.56 | 12067 > | ReadStage-3 > > e70ac650-cd9e-11e8-8e99-15807bff4dfd | > e70f8142-cd9e-11e8-bed7-f13ab36bfa4c > | > Enqueuing response to /10.54.145.32 > <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=> > | 10.54.101.56 | 12361 | > ReadStage-3 > > e70ac650-cd9e-11e8-8e99-15807bff4dfd | > e70fcf60-cd9e-11e8-bed7-f13ab36bfa4c > | > Sending REQUEST_RESPONSE message to /10.54.145.32 > <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=> > | 10.54.101.56 | 14161 | > MessagingService-Outgoing-/10.54.145.32-Small > > e70ac650-cd9e-11e8-8e99-15807bff4dfd | > e70ff670-cd9e-11e8-8e99-15807bff4dfd | > REQUEST_RESPONSE > message received from /10.54.101.56 > <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.101.56&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=KoAVTaVcog1srnn8npMVq8wpXmZq95ibi870ELyivGM&e=> > | 10.54.145.32 | 51 | MessagingService-Incoming-/ > 10.54.101.56 > <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.101.56&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=KoAVTaVcog1srnn8npMVq8wpXmZq95ibi870ELyivGM&e=> > > e70ac650-cd9e-11e8-8e99-15807bff4dfd | > e70ff671-cd9e-11e8-8e99-15807bff4dfd | > >