GitHub Actions: Reverse DNS lookup for current hostname is broken, causes network issues
Hi all,
Reverse DNS lookups for the current hostname is broken on ubuntu-latest
GitHub Actions Runner VMs.
The reverse lookup returns different IP addresses which aren't addresses of
the current host.
This can cause various networking issues.
The workaround is to add an entry for the host name to /etc/hosts.
For example, by adding this action in the workflow's build job:
- name: Add the current IP address, long hostname and short hostname record
to /etc/hosts file
run: |
echo -e "$(ip addr show eth0 | grep "inet\b" | awk '{print $2}' | cut
-d/ -f1)\t$(hostname -f) $(hostname -s)" | sudo tee -a /etc/hosts
More information:
Apache Pulsar issue:
https://github.com/apache/pulsar/issues/10232
Apache Pulsar fix (currently in-progress):
https://github.com/apache/pulsar/pull/10233
I did a quick search on GitHub Community site and found these problem
reports:
-
https://github.community/t/github-action-and-oserror-errno-99-cannot-assign-requested-address/173973
-
https://github.community/t/volatile-local-ip-addresses-during-action-run/174362
- https://github.community/t/github-actions-timeout/174533
I hope this helps.
BR,
Lari
Re: Better stability with docker authenticated jenkins agents
On 07/04/2021 07:43, Mick Semb Wever wrote:
>>
>> My understanding is that pulls of all images from the apache/* namespace
>> are not subject to rate limiting. Thus, the recommendation to move
>> everything you need inside of it.
>>
> As >95% of our CI docker commands are pulls from apache/ images,
> if rate-limiting is the cause of this (note that nowhere did we see the
> toomanyrequests response error),
> then we still need to authenticate docker to get the rate-limiting
> exception for those apache/ images,
> as you mention here¹. Has that changed?
My mistake, I mis-remembered why we wanted this change. Moving images to
the apache organisation only prevents them from being auto-deleted by
Docker Hub's scrubbing process, which started in the middle of last year.
Your proposal to have a Docker Hub account seems reasonable, but since
this is something projects can solve themselves, not critical path.
We'll probably store the Docker Hub creds in Jenkins, then reference
that in the build, which is what we do for other creds we need today. As
we use declarative pipeline, that's something like:
docker {
image "${DOCKER_IMAGE}"
label 'docker'
args "${DOCKER_ARGS}"
registryCredentialsId "${DOCKER_CREDS"}
}
Then it is easy to use our own Jenkins-stored creds
(https://www.jenkins.io/doc/book/using/using-credentials/), or Infra can
give us a pair to use instead.
> Maybe I'm circling, but doesn't this then support the need that we should
> have jenkins agents docker authenticated somehow?
>
> [1]
> https://lists.apache.org/thread.html/rede9074dd499ae10dcb501dedcdec43fe9cbb5c646a2c38b19946f85%40%3Cbuilds.apache.org%3E
Re: Better stability with docker authenticated jenkins agents
> Then it is easy to use our own Jenkins-stored creds > (https://urldefense.proofpoint.com/v2/url?u=https-3A__www.jenkins.io_doc_book_using_using-2Dcredentials_&d=DwIFaQ&c=adz96Xi0w1RHqtPMowiL2g&r=42Z7FyMoAS1DbvgKNjU8zxi7xTPVAGalPzk7bfmRVgw&m=Oa7qm3rmqlXgK8wUUTEnADpIgZT18tE6W-ghG1QUspU&s=vRKVrs-iZa60eTjfvdc1aggzj_-CyObd2xs70PdN1rs&e= > ), or Infra can > give us a pair to use instead. > Ahh! Do we have rights to store credentials in the ASF jenkins? I was under the impression INFRA had to do this for us. This goes to the crux of the problem raised: the original request on INFRA, i.e. to have "credentials stored in an appropriate manner". Gavin, is it an acceptable approach to have projects create their own dockerhub accounts and to store those credentials in jenkins themselves as a way to solve the rate-limiting of apache/* namespace images?
Re: Better stability with docker authenticated jenkins agents
On 14/04/2021 13:05, Mick Semb Wever wrote: >> Then it is easy to use our own Jenkins-stored creds >> (https://urldefense.proofpoint.com/v2/url?u=https-3A__www.jenkins.io_doc_book_using_using-2Dcredentials_&d=DwIFaQ&c=adz96Xi0w1RHqtPMowiL2g&r=42Z7FyMoAS1DbvgKNjU8zxi7xTPVAGalPzk7bfmRVgw&m=Oa7qm3rmqlXgK8wUUTEnADpIgZT18tE6W-ghG1QUspU&s=vRKVrs-iZa60eTjfvdc1aggzj_-CyObd2xs70PdN1rs&e= >> ), or Infra can >> give us a pair to use instead. >> > > > Ahh! Do we have rights to store credentials in the ASF jenkins? I was > under the impression INFRA had to do this for us. Yes, you can, especially for Cassandra where you have your own separate build master. Projects are on the honour system not to use creds they're not supposed to have access to, which is at least one major reason why CouchDB requested and received their own CloudBees/Jenkins build master. The separate build master structure should prevent anyone outside of your PMC-approved group from creating any jobs on your build server that might use those credentials inappropriately. -Joan
Re: Better stability with docker authenticated jenkins agents
Folders can store credentials as well if you're on the main Jenkins instance here. On Wed, 14 Apr 2021 at 12:19, Joan Touzet wrote: > > On 14/04/2021 13:05, Mick Semb Wever wrote: > >> Then it is easy to use our own Jenkins-stored creds > >> (https://urldefense.proofpoint.com/v2/url?u=https-3A__www.jenkins.io_doc_book_using_using-2Dcredentials_&d=DwIFaQ&c=adz96Xi0w1RHqtPMowiL2g&r=42Z7FyMoAS1DbvgKNjU8zxi7xTPVAGalPzk7bfmRVgw&m=Oa7qm3rmqlXgK8wUUTEnADpIgZT18tE6W-ghG1QUspU&s=vRKVrs-iZa60eTjfvdc1aggzj_-CyObd2xs70PdN1rs&e= > >> ), or Infra can > >> give us a pair to use instead. > >> > > > > > > Ahh! Do we have rights to store credentials in the ASF jenkins? I was > > under the impression INFRA had to do this for us. > > Yes, you can, especially for Cassandra where you have your own separate > build master. > > Projects are on the honour system not to use creds they're not supposed > to have access to, which is at least one major reason why CouchDB > requested and received their own CloudBees/Jenkins build master. The > separate build master structure should prevent anyone outside of your > PMC-approved group from creating any jobs on your build server that > might use those credentials inappropriately. > > -Joan
