CASSANDRA-10959 <https://issues.apache.org/jira/browse/CASSANDRA-10959> made
the control connection timeout be the same as the connect timeout. This
patch is available since 2.2.5, 3.0.3, 3.3.
On Fri, Oct 14, 2016 at 11:45 AM, joseph gao <gaojf.bok...@gmail.com> wrote:
> I've found the problem, in cqlsh file, find all Cluster 's construct
> method like ' conn = Cluster(xxx,xxx). At the end, add
> parameter control_connection_timeout=float(_SOME_MS_VALUE_). As below:
> conn = Cluster(contact_points=(self.hostname,), port=self.port,
> cql_version=self.conn.cql_version,
> protocol_version=DEFAULT_PROTOCOL_VERSION,
> auth_provider=auth_provider,
> ssl_options=self.conn.ssl_options,
> load_balancing_policy=WhiteListRoundRobinPolicy([
> self.hostname]),
> control_connection_timeout=float(36000))
> this will fix the problem, thanks all.
>
> 2016-09-20 17:23 GMT+08:00 George Sigletos <sigle...@textkernel.nl>:
>
>> Apologies. The entire error stack:
>>
>> ERROR [SharedPool-Worker-5] 2016-09-20 11:23:20,039 ErrorMessage.java:251
>> - Unexpected exception during request
>> com.google.common.util.concurrent.UncheckedExecutionException:
>> java.lang.RuntimeException:
>> org.apache.cassandra.exceptions.ReadTimeoutException:
>> Operation timed out - received only 2 responses.
>> at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2201)
>> ~[guava-16.0.jar:na]
>> at com.google.common.cache.LocalCache.get(LocalCache.java:3934)
>> ~[guava-16.0.jar:na]
>> at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3938)
>> ~[guava-16.0.jar:na]
>> at
>> com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4821)
>> ~[guava-16.0.jar:na]
>> at
>> org.apache.cassandra.auth.PermissionsCache.getPermissions(PermissionsCache.java:72)
>> ~[apache-cassandra-2.1.14.jar:2.1.14]
>> at org.apache.cassandra.auth.Auth.getPermissions(Auth.java:75)
>> ~[apache-cassandra-2.1.14.jar:2.1.14]
>> at
>> org.apache.cassandra.service.ClientState.authorize(ClientState.java:353)
>> ~[apache-cassandra-2.1.14.jar:2.1.14]
>> at
>> org.apache.cassandra.service.ClientState.ensureHasPermission(ClientState.java:251)
>> ~[apache-cassandra-2.1.14.jar:2.1.14]
>> at
>> org.apache.cassandra.service.ClientState.hasAccess(ClientState.java:245)
>> ~[apache-cassandra-2.1.14.jar:2.1.14]
>> at
>> org.apache.cassandra.service.ClientState.hasColumnFamilyAccess(ClientState.java:229)
>> ~[apache-cassandra-2.1.14.jar:2.1.14]
>> at
>> org.apache.cassandra.cql3.statements.SelectStatement.checkAccess(SelectStatement.java:195)
>> ~[apache-cassandra-2.1.14.jar:2.1.14]
>> at
>> org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:235)
>> ~[apache-cassandra-2.1.14.jar:2.1.14]
>> at
>> org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:260)
>> ~[apache-cassandra-2.1.14.jar:2.1.14]
>> at
>> org.apache.cassandra.transport.messages.QueryMessage.execute(QueryMessage.java:122)
>> ~[apache-cassandra-2.1.14.jar:2.1.14]
>> at
>> org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:439)
>> [apache-cassandra-2.1.14.jar:2.1.14]
>> at
>> org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:335)
>> [apache-cassandra-2.1.14.jar:2.1.14]
>> at io.netty.channel.SimpleChannelInboundHandler.channelRead(Sim
>> pleChannelInboundHandler.java:105) [netty-all-4.0.23.Final.jar:4.
>> 0.23.Final]
>> at io.netty.channel.AbstractChannelHandlerContext.invokeChannel
>> Read(AbstractChannelHandlerContext.java:333)
>> [netty-all-4.0.23.Final.jar:4.0.23.Final]
>> at io.netty.channel.AbstractChannelHandlerContext.access$700(Ab
>> stractChannelHandlerContext.java:32) [netty-all-4.0.23.Final.jar:4.
>> 0.23.Final]
>> at io.netty.channel.AbstractChannelHandlerContext$8.run(Abstrac
>> tChannelHandlerContext.java:324) [netty-all-4.0.23.Final.jar:4.
>> 0.23.Final]
>> at java.util.concurrent.Executors$RunnableAdapter.call(Unknown
>> Source) [na:1.7.0_79]
>> at org.apache.cassandra.concurrent.AbstractTracingAwareExecutor
>> Service$FutureTask.run(AbstractTracingAwareExecutorService.java:164)
>> [apache-cassandra-2.1.14.jar:2.1.14]
>> at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105)
>> [apache-cassandra-2.1.14.jar:2.1.14]
>> at java.lang.Thread.run(Unknown Source) [na:1.7.0_79]
>> Caused by: java.lang.RuntimeException:
>> org.apache.cassandra.exceptions.ReadTimeoutException:
>> Operation timed out - received only 2 responses.
>> at org.apache.cassandra.auth.Auth.selectUser(Auth.java:276)
>> ~[apache-cassandra-2.1.14.jar:2.1.14]
>> at org.apache.cassandra.auth.Auth.isSuperuser(Auth.java:97)
>> ~[apache-cassandra-2.1.14.jar:2.1.14]
>> at
>> org.apache.cassandra.auth.AuthenticatedUser.isSuper(AuthenticatedUser.java:50)
>> ~[apache-cassandra-2.1.14.jar:2.1.14]
>> at
>> org.apache.cassandra.auth.CassandraAuthorizer.authorize(CassandraAuthorizer.java:67)
>> ~[apache-cassandra-2.1.14.jar:2.1.14]
>> at
>> org.apache.cassandra.auth.PermissionsCache$1.load(PermissionsCache.java:124)
>> ~[apache-cassandra-2.1.14.jar:2.1.14]
>> at
>> org.apache.cassandra.auth.PermissionsCache$1.load(PermissionsCache.java:121)
>> ~[apache-cassandra-2.1.14.jar:2.1.14]
>> at
>> com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3524)
>> ~[guava-16.0.jar:na]
>> at
>> com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2317)
>> ~[guava-16.0.jar:na]
>> at
>> com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2280)
>> ~[guava-16.0.jar:na]
>> at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2195)
>> ~[guava-16.0.jar:na]
>> ... 23 common frames omitted
>>
>> On Tue, Sep 20, 2016 at 11:22 AM, George Sigletos <sigle...@textkernel.nl
>> > wrote:
>>
>>> This appears in the system log:
>>>
>>> Caused by: java.lang.RuntimeException:
>>> org.apache.cassandra.exceptions.ReadTimeoutException:
>>> Operation timed out - received only 2 responses.
>>> at org.apache.cassandra.auth.Auth.selectUser(Auth.java:276)
>>> ~[apache-cassandra-2.1.14.jar:2.1.14]
>>> at org.apache.cassandra.auth.Auth.isSuperuser(Auth.java:97)
>>> ~[apache-cassandra-2.1.14.jar:2.1.14]
>>> at
>>> org.apache.cassandra.auth.AuthenticatedUser.isSuper(AuthenticatedUser.java:50)
>>> ~[apache-cassandra-2.1.14.jar:2.1.14]
>>> at
>>> org.apache.cassandra.auth.CassandraAuthorizer.authorize(CassandraAuthorizer.java:67)
>>> ~[apache-cassandra-2.1.14.jar:2.1.14]
>>> at
>>> org.apache.cassandra.auth.PermissionsCache$1.load(PermissionsCache.java:124)
>>> ~[apache-cassandra-2.1.14.jar:2.1.14]
>>> at
>>> org.apache.cassandra.auth.PermissionsCache$1.load(PermissionsCache.java:121)
>>> ~[apache-cassandra-2.1.14.jar:2.1.14]
>>> at
>>> com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3524)
>>> ~[guava-16.0.jar:na]
>>> at
>>> com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2317)
>>> ~[guava-16.0.jar:na]
>>> at
>>> com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2280)
>>> ~[guava-16.0.jar:na]
>>> at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2195)
>>> ~[guava-16.0.jar:na]
>>>
>>>
>>> On Tue, Sep 20, 2016 at 11:12 AM, George Sigletos <
>>> sigle...@textkernel.nl> wrote:
>>>
>>>> I am also getting the same error:
>>>> cqlsh <my host> -u cassandra -p cassandra
>>>>
>>>> Connection error: ('Unable to connect to any servers', {'<my host>':
>>>> OperationTimedOut('errors=Timed out creating connection (5 seconds),
>>>> last_host=None',)})
>>>>
>>>> But it is not consistent. Sometimes I manage to connect. It is random.
>>>> Using 2.1.14
>>>>
>>>> On Tue, Jun 14, 2016 at 4:29 AM, joseph gao <gaojf.bok...@gmail.com>
>>>> wrote:
>>>>
>>>>> hi, Patric, [image: 内嵌图片 1],
>>>>> netstat -lepunt looks like above
>>>>>
>>>>> 2016-05-27 23:16 GMT+08:00 Patrick McFadin <pmcfa...@gmail.com>:
>>>>>
>>>>>> Can you do a netstat -lepunt and show the output? If Cassandra is
>>>>>> running you aren't trying to connect to the ip/port it's bound to.
>>>>>>
>>>>>> Patrick
>>>>>>
>>>>>>
>>>>>> On Monday, May 23, 2016, joseph gao <gaojf.bok...@gmail.com> wrote:
>>>>>>
>>>>>>> I used to think it's firewall/network issues too. So I make ufw to
>>>>>>> be inactive. I really don't what's the reason.
>>>>>>>
>>>>>>> 2016-05-09 19:01 GMT+08:00 kurt Greaves <k...@instaclustr.com>:
>>>>>>>
>>>>>>>> Don't be fooled, despite saying tcp6 and :::*, it still listens on
>>>>>>>> IPv4. As far as I'm aware this happens on all 2.1 Cassandra nodes, and
>>>>>>>> may
>>>>>>>> just be an oddity of netstat. It would be unrelated to your connection
>>>>>>>> timeout issues, that's most likely related to firewall/network issues.
>>>>>>>>
>>>>>>>> On 9 May 2016 at 09:59, joseph gao <gaojf.bok...@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> It doesn't work ,still using ipv6 [image: 内嵌图片 1]
>>>>>>>>>
>>>>>>>>> And I already set [image: 内嵌图片 2]
>>>>>>>>>
>>>>>>>>> Now I'm using 4.1.1 using 9160 port instead of 5.x.x。
>>>>>>>>>
>>>>>>>>> Hopefully this could be resolved, Thanks!
>>>>>>>>>
>>>>>>>>> 2016-03-30 22:13 GMT+08:00 Alain RODRIGUEZ <arodr...@gmail.com>:
>>>>>>>>>
>>>>>>>>>> Hi Joseph,
>>>>>>>>>>
>>>>>>>>>> why cassandra using tcp6 for 9042 port like :
>>>>>>>>>>> tcp6 0 0 0.0.0.0:9042 :::*
>>>>>>>>>>> LISTEN
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> if I remember correctly, in 2.1 and higher, cqlsh uses native
>>>>>>>>>> transport, port 9042 (instead of thrift port 9160) and your clients
>>>>>>>>>> (if
>>>>>>>>>> any) are also probably using native transport (port 9042). So yes,
>>>>>>>>>> this
>>>>>>>>>> could be an issue indeed.
>>>>>>>>>>
>>>>>>>>>> You should have something like:
>>>>>>>>>>
>>>>>>>>>> tcp 0 0 1.2.3.4:9042 :::*
>>>>>>>>>> LISTEN
>>>>>>>>>>
>>>>>>>>>> You are using IPv6 and no rpc address. Try setting it to the
>>>>>>>>>> listen address and using IPv4.
>>>>>>>>>>
>>>>>>>>>> C*heers,
>>>>>>>>>>
>>>>>>>>>> -----------------------
>>>>>>>>>>
>>>>>>>>>> Alain Rodriguez - al...@thelastpickle.com
>>>>>>>>>>
>>>>>>>>>> France
>>>>>>>>>>
>>>>>>>>>> The Last Pickle - Apache Cassandra Consulting
>>>>>>>>>>
>>>>>>>>>> http://www.thelastpickle.com
>>>>>>>>>>
>>>>>>>>>> 2016-03-30 6:09 GMT+02:00 joseph gao <gaojf.bok...@gmail.com>:
>>>>>>>>>>
>>>>>>>>>>> why cassandra using tcp6 for 9042 port like :
>>>>>>>>>>> tcp6 0 0 0.0.0.0:9042 :::*
>>>>>>>>>>> LISTEN
>>>>>>>>>>> would this be the problem
>>>>>>>>>>>
>>>>>>>>>>> 2016-03-30 11:34 GMT+08:00 joseph gao <gaojf.bok...@gmail.com>:
>>>>>>>>>>>
>>>>>>>>>>>> still have not fixed it . cqlsh: error: no such option:
>>>>>>>>>>>> --connect-timeout
>>>>>>>>>>>> cqlsh version 5.0.1
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> 2016-03-25 16:46 GMT+08:00 Alain RODRIGUEZ <arodr...@gmail.com>
>>>>>>>>>>>> :
>>>>>>>>>>>>
>>>>>>>>>>>>> Hi Joseph.
>>>>>>>>>>>>>
>>>>>>>>>>>>> As I can't reproduce here, I believe you are having network
>>>>>>>>>>>>> issue of some kind.
>>>>>>>>>>>>>
>>>>>>>>>>>>> MacBook-Pro:~ alain$ cqlsh --version
>>>>>>>>>>>>> cqlsh 5.0.1
>>>>>>>>>>>>> MacBook-Pro:~ alain$ echo 'DESCRIBE KEYSPACES;' | cqlsh
>>>>>>>>>>>>> --connect-timeout=5 --request-timeout=10
>>>>>>>>>>>>> system_traces system
>>>>>>>>>>>>> MacBook-Pro:~ alain$
>>>>>>>>>>>>>
>>>>>>>>>>>>> It's been a few days, did you manage to fix it ?
>>>>>>>>>>>>>
>>>>>>>>>>>>> C*heers,
>>>>>>>>>>>>> -----------------------
>>>>>>>>>>>>> Alain Rodriguez - al...@thelastpickle.com
>>>>>>>>>>>>> France
>>>>>>>>>>>>>
>>>>>>>>>>>>> The Last Pickle - Apache Cassandra Consulting
>>>>>>>>>>>>> http://www.thelastpickle.com
>>>>>>>>>>>>>
>>>>>>>>>>>>> 2016-03-21 9:59 GMT+01:00 joseph gao <gaojf.bok...@gmail.com>:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> cqlsh version 5.0.1. nodetool tpstats looks good, log looks
>>>>>>>>>>>>>> good. And I used specified port 9042. And it immediately returns
>>>>>>>>>>>>>> fail (less
>>>>>>>>>>>>>> than 3 seconds). By the way where should I use
>>>>>>>>>>>>>> '--connect-timeout', cqlsh
>>>>>>>>>>>>>> seems don't have such parameters.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> 2016-03-18 17:29 GMT+08:00 Alain RODRIGUEZ <
>>>>>>>>>>>>>> arodr...@gmail.com>:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Is the node fully healthy or rejecting some requests ?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> What are the outputs for "grep -i "ERROR"
>>>>>>>>>>>>>>> /var/log/cassandra/system.log" and "nodetool tpstats"?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Any error? Any pending / blocked or dropped messages?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Also did you try using distinct ports (9160 for thrift, 9042
>>>>>>>>>>>>>>> for native) - out of curiosity, not sure this will help.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> What is your version of cqlsh "cqlsh --version" ?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> doesn't work most times. But some time it just work fine
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Do you fill like this is due to a timeout (query being too
>>>>>>>>>>>>>>> big, cluster being to busy)? Try setting this higher:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> --connect-timeout=CONNECT_TIMEOUT
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Specify the connection timeout in
>>>>>>>>>>>>>>> seconds (default: 5 seconds).
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> --request-timeout=REQUEST_TIMEOUT
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Specify the default request timeout
>>>>>>>>>>>>>>> in seconds (default: 10 seconds).
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> C*heers,
>>>>>>>>>>>>>>> -----------------------
>>>>>>>>>>>>>>> Alain Rodriguez - al...@thelastpickle.com
>>>>>>>>>>>>>>> France
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> The Last Pickle - Apache Cassandra Consulting
>>>>>>>>>>>>>>> http://www.thelastpickle.com
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> 2016-03-18 4:49 GMT+01:00 joseph gao <gaojf.bok...@gmail.com
>>>>>>>>>>>>>>> >:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Of course yes.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> 2016-03-17 22:35 GMT+08:00 Vishwas Gupta <
>>>>>>>>>>>>>>>> vishwas.gu...@snapdeal.com>:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Have you started the Cassandra service?
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> sh cassandra
>>>>>>>>>>>>>>>>> On 17-Mar-2016 7:59 pm, "Alain RODRIGUEZ" <
>>>>>>>>>>>>>>>>> arodr...@gmail.com> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Hi, did you try with the address of the node rather than
>>>>>>>>>>>>>>>>>> 127.0.0.1
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Is the transport protocol used by cqlsh (not sure if it
>>>>>>>>>>>>>>>>>> is thrift or binary - native in 2.1) active ? What is the
>>>>>>>>>>>>>>>>>> "nodetool info"
>>>>>>>>>>>>>>>>>> output ?
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> C*heers,
>>>>>>>>>>>>>>>>>> -----------------------
>>>>>>>>>>>>>>>>>> Alain Rodriguez - al...@thelastpickle.com
>>>>>>>>>>>>>>>>>> France
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> The Last Pickle - Apache Cassandra Consulting
>>>>>>>>>>>>>>>>>> http://www.thelastpickle.com
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> 2016-03-17 14:26 GMT+01:00 joseph gao <
>>>>>>>>>>>>>>>>>> gaojf.bok...@gmail.com>:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> hi, all
>>>>>>>>>>>>>>>>>>> cassandra version 2.1.7
>>>>>>>>>>>>>>>>>>> When I use cqlsh to connect cassandra, something is wrong
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Connection error: ( Unable to connect to any servers',
>>>>>>>>>>>>>>>>>>> {'127.0.0.1': OperationTimedOut('errors=None,
>>>>>>>>>>>>>>>>>>> last_host=None,)})
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> This happens lots of times, but sometime it works just
>>>>>>>>>>>>>>>>>>> fine. Anybody knows why?
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>> ------
>>>>>>>>>>>>>>>>>>> Joseph Gao
>>>>>>>>>>>>>>>>>>> PhoneNum:15210513582
>>>>>>>>>>>>>>>>>>> QQ: 409343351
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>> ------
>>>>>>>>>>>>>>>> Joseph Gao
>>>>>>>>>>>>>>>> PhoneNum:15210513582
>>>>>>>>>>>>>>>> QQ: 409343351
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> --
>>>>>>>>>>>>>> ------
>>>>>>>>>>>>>> Joseph Gao
>>>>>>>>>>>>>> PhoneNum:15210513582
>>>>>>>>>>>>>> QQ: 409343351
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> --
>>>>>>>>>>>> ------
>>>>>>>>>>>> Joseph Gao
>>>>>>>>>>>> PhoneNum:15210513582
>>>>>>>>>>>> QQ: 409343351
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> ------
>>>>>>>>>>> Joseph Gao
>>>>>>>>>>> PhoneNum:15210513582
>>>>>>>>>>> QQ: 409343351
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> ------
>>>>>>>>> Joseph Gao
>>>>>>>>> PhoneNum:15210513582
>>>>>>>>> QQ: 409343351
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Kurt Greaves
>>>>>>>> k...@instaclustr.com
>>>>>>>> www.instaclustr.com
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> ------
>>>>>>> Joseph Gao
>>>>>>> PhoneNum:15210513582
>>>>>>> QQ: 409343351
>>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> ------
>>>>> Joseph Gao
>>>>> PhoneNum:15210513582
>>>>> QQ: 409343351
>>>>>
>>>>
>>>>
>>>
>>
>
>
> --
> ------
> Joseph Gao
> PhoneNum:15210513582
> QQ: 409343351
>
--
Stefania Alborghetti
|+852 6114 9265| stefania.alborghe...@datastax.com
