On Wed, Oct 16, 2013 at 05:11:46AM +0000, Girish Shilamkar wrote: > Hello, > > Sometimes while running automation testcases tests just fail with following > error: > > ConnectionError: HTTPConnectionPool(host='10.147.59.65', port=8080): > Max retries exceeded with url: /client/api? > apiKey=lcgWQ9XrjyIyPjsl7Y0BGI2udSRMs6S1w_42lWkwpCSfLWjkgoWvCakWwUCJ3TqfgMmuHWT4DXv5Lgoz-KCuSQ&egressdefaultpolicy=true&serviceproviderlist%5B2%5D. > provider=VirtualRouter&serviceproviderlist%5B5%5D.service=Dhcp&displaytext=Network+offering-VR+services-8FM4XI&specifyVlan=False&serviceproviderlist%5B4%5D. > provider=VirtualRouter&serviceproviderlist%5B1%5D.provider=VirtualRouter&availability=Optional&conservemode=True&servicecapabilitylist%5B1%5D. > capabilitytype=SupportedSourceNatTypes&serviceproviderlist%5B3%5D.service=Dns&serviceproviderlist%5B5%5D.provider=VirtualRouter&response=json&serviceproviderlist%5B0%5D. > provider=VirtualRouter&servicecapabilitylist%5B0%5D.capabilitytype=RedundantRouter&serviceproviderlist%5B8%5D.provider=VirtualRouter&serviceproviderlist%5B1%5D. > service=Lb&servicecapabilitylist%5B0%5D.service=SourceNat&serviceproviderlist%5B4%5D. > service=Firewall&supportedservices=Dhcp%2CDns%2CSourceNat%2CPortForwarding%2CVpn%2CFirewall%2CLb%2CUserData%2CStaticNat&traffictype=GUEST&servicecapabilitylist%5B1%5D. > service=SourceNat&serviceproviderlist%5B6%5D.provider=VirtualRouter&serviceproviderlist%5B8%5D.service=StaticNat&serviceproviderlist%5B3%5D.provider=VirtualRouter&name=Network+ > offering-VR+services-OLND5Q&guestiptype=Isolated&serviceproviderlist%5B7%5D.provider=VirtualRouter&servicecapabilitylist%5B1%5D. > capabilityvalue=peraccount&serviceproviderlist%5B7%5D.service=SourceNat&serviceproviderlist%5B2%5D.service=PortForwarding&servicecapabilitylist%5B0%5D. > capabilityvalue=true&serviceproviderlist%5B6%5D.service=Vpn&command=createNetworkOffering&signature=dFBPTQoc%2B%2B42%2FhDVyQQOFB87c7k%3D&serviceproviderlist%5B0%5D.service=UserData > (Caused by <class 'httplib.BadStatusLine'>: '') >
We actually don't use the submitCmdsAndWait to send the requests to the mgmt server. We just send it through the apiclient and cloudConnection.py takes over. I've seen the BadStatusLine issue with httplib on our test systems too. It is pretty random. At first I thought it was the throttling of the API but we don't enable that setting. I'm still looking for why this happens ... and how to repro it. > These errors are intermittent. While investigating this problem I found this > in marvin.cloudstackTestClient > > def submitCmdsAndWait(self, cmds, workers=1): > '''FixME, httplib has issue if more than one thread submitted''' > if self.asyncJobMgr is None: > self.asyncJobMgr = asyncJobMgr.asyncJobMgr(self.apiClient, > self.dbConnection) > return self.asyncJobMgr.submitCmdsAndWait(cmds, workers) > > My theory is if the response from management server slow, multiple thread are > submitted and we hit this issue. > > Please advise. > > Regards, > Girish > -- Prasanna., ------------------------ Powered by BigRock.com
