On 05/22/2017 08:28 AM, Ed Warnicke wrote:
Do you have any insight on this?

Ed
I am aware of it but I don't have any insight at the moment.
I am setting up an environment where I can run csit locally to do some local verification.
Also, I am searching the Centos and RHEL lists for known python issues.

On Mon, May 22, 2017 at 1:29 AM, Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco) <ksek...@cisco.com <mailto:ksek...@cisco.com>> wrote:


    Hi,

    the centos python crash is known, but we're unsure about the root
    cause.
    Building newer python from source on centos vm makes the crashes
    go away
    so we're assuming that the (older) python itself might be the culprit,
    since we haven't seen these on ubuntu at all.

    Regarding the second crash - I'm not sure whether this is 'make test'
    fault or not.

    If somebody could translate from java/hudson/... to english and/or
    provide logs, then I could take a look..

    Thanks,
    Klement

    Quoting Kinsella, Ray (2017-05-22 10:21:21)
    > Hi folks,
    >
    > Not sure if it's just me, but some CI tests have suddenly start
    failing
    > for me. Is it just me or a wider problem?
    >
    > Ray K
    >
    >
    > CENTOS
    > https://jenkins.fd.io/job/vpp-verify-master-centos7/5568/
    <https://jenkins.fd.io/job/vpp-verify-master-centos7/5568/>
    >
    > 19:52:10 IP Multicast Signabash: line 1: 21723 Segmentation fault
    > (core dumped) python run_tests.py -d
    > /w/workspace/vpp-verify-master-centos7/test
    > 19:54:03 make[2]: *** [test] Error 139
    > 19:54:03 make[2]: Leaving directory
    > `/w/workspace/vpp-verify-master-centos7/test'
    > 19:54:03 make[1]: *** [test] Error 2
    > 19:54:03 make[1]: Leaving directory
    `/w/workspace/vpp-verify-master-centos7'
    > 19:54:03 make: *** [verify] Error 2
    > 19:54:04 Build step 'Execute shell' marked build as failure
    >
    >
    > UBUNTU
    >
    https://jenkins.fd.io/job/vpp-verify-master-ubuntu1604/5573/console
    <https://jenkins.fd.io/job/vpp-verify-master-ubuntu1604/5573/console>
    >
    > 19:37:57 IP NULL route
    >           OK
    > 19:37:57
    > ============================================================FATAL:
    > command execution failed
    > 19:48:45 java.io.EOFException
    > 19:48:45        at
    >
    
java.io.ObjectInputStream$PeekInputStream.readFully(ObjectInputStream.java:2638)
    > 19:48:45        at
    >
    
java.io.ObjectInputStream$BlockDataInputStream.readShort(ObjectInputStream.java:3113)
    > 19:48:45        at
    >
    java.io.ObjectInputStream.readStreamHeader(ObjectInputStream.java:853)
    > 19:48:45        at
    java.io.ObjectInputStream.<init>(ObjectInputStream.java:349)
    > 19:48:45        at
    >
    hudson.remoting.ObjectInputStreamEx.<init>(ObjectInputStreamEx.java:48)
    > 19:48:45        at
    >
    
hudson.remoting.AbstractSynchronousByteArrayCommandTransport.read(AbstractSynchronousByteArrayCommandTransport.java:34)
    > 19:48:45        at
    >
    
hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:59)
    > 19:48:45 Caused: java.io.IOException: Unexpected termination of
    the channel
    > 19:48:45        at
    >
    
hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:73)
    > 19:48:45 Caused: java.io.IOException: Backing channel
    > 'ubuntu1604-basebuild-4c-4g-5113' is disconnected.
    > 19:48:45        at
    >
    
hudson.remoting.RemoteInvocationHandler.channelOrFail(RemoteInvocationHandler.java:192)
    > 19:48:45        at
    >
    
hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:257)
    > 19:48:45        at com.sun.proxy.$Proxy87.isAlive(Unknown Source)
    > 19:48:45        at
    > hudson.Launcher$RemoteLauncher$ProcImpl.isAlive(Launcher.java:1043)
    > 19:48:45        at
    > hudson.Launcher$RemoteLauncher$ProcImpl.join(Launcher.java:1035)
    > 19:48:45        at
    > hudson.tasks.CommandInterpreter.join(CommandInterpreter.java:155)
    > 19:48:45        at
    > hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:109)
    > 19:48:45        at
    > hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:66)
    > 19:48:45        at
    > hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
    > 19:48:45        at
    >
    
hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:779)
    > 19:48:45        at
    hudson.model.Build$BuildExecution.build(Build.java:206)
    > 19:48:45        at
    hudson.model.Build$BuildExecution.doRun(Build.java:163)
    > 19:48:45        at
    >
    
hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:534)
    > 19:48:45        at hudson.model.Run.execute(Run.java:1728)
    > 19:48:45        at
    hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
    > 19:48:45        at
    > hudson.model.ResourceController.execute(ResourceController.java:98)
    > 19:48:45        at hudson.model.Executor.run(Executor.java:405)
    > 19:48:45 Build step 'Execute shell' marked build as failure
    > 19:48:45 FATAL: channel is already closed
    > 19:48:45 java.io.EOFException
    > 19:48:45        at
    >
    
java.io.ObjectInputStream$PeekInputStream.readFully(ObjectInputStream.java:2638)
    > 19:48:45        at
    >
    
java.io.ObjectInputStream$BlockDataInputStream.readShort(ObjectInputStream.java:3113)
    > 19:48:45        at
    >
    java.io.ObjectInputStream.readStreamHeader(ObjectInputStream.java:853)
    > 19:48:45        at
    java.io.ObjectInputStream.<init>(ObjectInputStream.java:349)
    > 19:48:45        at
    >
    hudson.remoting.ObjectInputStreamEx.<init>(ObjectInputStreamEx.java:48)
    > 19:48:45        at
    >
    
hudson.remoting.AbstractSynchronousByteArrayCommandTransport.read(AbstractSynchronousByteArrayCommandTransport.java:34)
    > 19:48:45        at
    >
    
hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:59)
    > 19:48:45 Caused: java.io.IOException: Unexpected termination of
    the channel
    > 19:48:45        at
    >
    
hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:73)
    > 19:48:45 Caused: hudson.remoting.ChannelClosedException: channel is
    > already closed
    > 19:48:45        at hudson.remoting.Channel.send(Channel.java:605)
    > 19:48:45        at hudson.remoting.Request.call(Request.java:130)
    > 19:48:45        at hudson.remoting.Channel.call(Channel.java:829)
    > 19:48:45        at
    hudson.Launcher$RemoteLauncher.launch(Launcher.java:959)
    > 19:48:45        at
    hudson.Launcher$ProcStarter.start(Launcher.java:384)
    > 19:48:45        at
    >
    
com.cloudbees.jenkins.plugins.sshagent.exec.ExecRemoteAgent.stop(ExecRemoteAgent.java:126)
    > 19:48:45        at
    >
    
com.cloudbees.jenkins.plugins.sshagent.SSHAgentBuildWrapper$SSHAgentEnvironment.tearDown(SSHAgentBuildWrapper.java:417)
    > 19:48:45        at
    hudson.model.Build$BuildExecution.doRun(Build.java:174)
    > 19:48:45        at
    >
    
hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:534)
    > 19:48:45        at hudson.model.Run.execute(Run.java:1728)
    > 19:48:45        at
    hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
    > 19:48:45        at
    > hudson.model.ResourceController.execute(ResourceController.java:98)
    > 19:48:45        at hudson.model.Executor.run(Executor.java:405)
    > 19:48:45 ERROR: Step ‘Archive the artifacts’ failed: no
    workspace for
    > vpp-verify-master-ubuntu1604 #5573
    > 19:48:45 [PostBuildScript] - Execution post build scripts.
    > 19:48:45 ERROR: Build step failed with exception
    > 19:48:45 java.lang.NullPointerException: no workspace from node
    >
    
jenkins.plugins.openstack.compute.JCloudsSlave[ubuntu1604-basebuild-4c-4g-5113]
    > which is computer
    > jenkins.plugins.openstack.compute.JCloudsComputer@80c4db and has
    channel
    > null
    > 19:48:45        at
    > hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:88)
    > 19:48:45        at
    > hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:66)
    > 19:48:45        at
    >
    
org.jenkinsci.plugins.postbuildscript.PostBuildScript.processBuildSteps(PostBuildScript.java:204)
    > 19:48:45        at
    >
    
org.jenkinsci.plugins.postbuildscript.PostBuildScript.processScripts(PostBuildScript.java:143)
    > 19:48:45        at
    >
    
org.jenkinsci.plugins.postbuildscript.PostBuildScript._perform(PostBuildScript.java:105)
    > 19:48:45        at
    >
    
org.jenkinsci.plugins.postbuildscript.PostBuildScript.perform(PostBuildScript.java:85)
    > 19:48:45        at
    > hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
    > 19:48:45        at
    >
    
hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:779)
    > 19:48:45        at
    >
    
hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:720)
    > 19:48:45        at
    hudson.model.Build$BuildExecution.post2(Build.java:186)
    > 19:48:45        at
    >
    
hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:665)
    > 19:48:45        at hudson.model.Run.execute(Run.java:1753)
    > 19:48:45        at
    hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
    > 19:48:45        at
    > hudson.model.ResourceController.execute(ResourceController.java:98)
    > 19:48:45        at hudson.model.Executor.run(Executor.java:405)
    > 19:48:45 Build step 'Execute a set of scripts' marked build as
    failure
    >
    >
    >
    > _______________________________________________
    > vpp-dev mailing list
    > vpp-dev@lists.fd.io <mailto:vpp-dev@lists.fd.io>
    > https://lists.fd.io/mailman/listinfo/vpp-dev
    <https://lists.fd.io/mailman/listinfo/vpp-dev>
    _______________________________________________
    csit-dev mailing list
    csit-...@lists.fd.io <mailto:csit-...@lists.fd.io>
    https://lists.fd.io/mailman/listinfo/csit-dev
    <https://lists.fd.io/mailman/listinfo/csit-dev>



--
*Thomas F Herbert*
Fast Data Planes
Office of Technology
*Red Hat*
_______________________________________________
vpp-dev mailing list
vpp-dev@lists.fd.io
https://lists.fd.io/mailman/listinfo/vpp-dev
  • [vpp-dev] CI ... Kinsella, Ray
    • Re: [vpp... Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco)
      • Re: ... Ed Warnicke
        • ... Damjan Marion (damarion)
        • ... Thomas F Herbert
          • ... Burt Silverman
            • ... Burt Silverman
              • ... Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco)

Reply via email to