Quick update: I just had the pipeline fail NOT during a long wait but while Ansible was setting a variable only a few seconds after the playbook was started, so the issue does NOT appear to have to do with long wait times.
On Thursday, May 13, 2021 at 7:15:38 AM UTC-5 Richard Lund wrote: > Hi. > > We use Jenkins to build our apps and then start AWX playbooks to deploy > the built app to Openshift. For that, we use the Ansible Tower plugin in > Jenkins and access it through pipeline code. > > Randomly, we get what looks like communication issues between AWX and > Jenkins. > > The pipeline will fail with the error: > > *"ERROR: Failed to get job status from Tower"* > > Often it will add a *503* error with the same message. > > I have checked all the Jenkins logs I can get to, but I can't find more > details on this. > > AWX is deployed in Openshift as Stateful Set, and I have scanned through > awx:celery and awx:web to try and track this down, but the only mention of > a 503 is this: > > [Ansible-Tower] Building GET request to https://awx/api/v2/jobs/57602/ > [Ansible-Tower] Forcing cert trust > [Ansible-Tower] Request completed with (503) > [Ansible-Tower] Deleting oAuth token 15396 for awx > [Ansible-Tower] Forcing cert trust > [Ansible-Tower] Calling for oAuth token delete at > https://awx/api/v2/tokens/15396/ > [Ansible-Tower] Request completed with (200) > > This seems to happen the most while AWX is waiting for Openshift to finish > deployment. The job ALWAYS finishes normally - yet Jenkins receives this > error and fails the pipeline even though everything else worked. It also > seems to happen on apps that take the longest to deploy - maybe 10-15 > minutes. > > We had this issue last year, then it went away, now it came back, and it > is affecting our work when pipelines are marked as failed and we have to > constantly double-check if the job actually finished or not. > > I have restarted Jenkins and AWX, and I have updated the Ansible Tower > plugin to the newest version - no change. > > AWX is on version 9.0.0.0 (installed in Openshift) with Ansible 2.8.5. > > Jenkins is on version 2.235.1 (also installed in Openshift). > > Ansible Tower Plugin is on version 0.16.0. > > I'd appreciate any help or pointers you can provide that can help me track > this down. > > Thanks! > > -- You received this message because you are subscribed to the Google Groups "Jenkins Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/1426fb08-03b1-49f2-a7a4-6f1d445b4320n%40googlegroups.com.
