Public bug reported:
Hi there,
i have found my kubernetes worker in "NotReady" state with message "PLEG is not
healthy: pleg was last seen active" because dns resolution was not available
anymore because "systemd-resolved" service was not able to restart after DHCP
was not available for some seconds. The "systemd-resolved" service remain in
fail state "start-limit-hit" after DHCP come back.
ubuntu version: "Ubuntu 18.04.5 LTS"
systemd version: "237-3ubuntu10.42"
Dec 18 16:33:26 mykubeworker dhclient[3963]: No DHCPOFFERS received.
Dec 18 16:33:26 mykubeworker dhclient[3963]: No working leases in persistent
database - sleeping.
Dec 18 16:33:26 mykubeworker systemd[1]: Stopping Network Name Resolution...
Dec 18 16:33:26 mykubeworker systemd[1]: Stopped Network Name Resolution.
Dec 18 16:33:26 mykubeworker systemd[1]: Starting Network Name Resolution...
Dec 18 16:33:27 mykubeworker systemd-resolved[12446]: Positive Trust Anchors:
Dec 18 16:33:27 mykubeworker systemd-resolved[12446]: . IN DS 19036 8 2
49aac11d7b6f6446702e54a1607371607a1a41855200fd2ce1cdde32f24e8fb5
Dec 18 16:33:27 mykubeworker systemd-resolved[12446]: . IN DS 20326 8 2
e06d44b80b8f1d39a95c0b0d7c65d08458e880409bbc683457104237c7f8ec8d
Dec 18 16:33:27 mykubeworker systemd-resolved[12446]: Negative trust anchors:
10.in-addr.arpa 16.172.in-addr.arpa 17.172.in-addr.arpa 18.172.in-addr.arpa
19.172.in-addr.arpa 20.172.in-addr.arpa 21.172.in-addr.arpa 22.172.in-addr.arpa
23.172.in-addr.arpa 24.172.in-addr.arpa 25.172.in-addr.arpa 26.172.in-addr.arpa
27.172.in-addr.arpa 28.172.in-addr.arpa 29.172.in-addr.arpa 30.172.in-addr.arpa
31.172.in-addr.arpa 168.192.in-addr.arpa d.f.ip6.arpa corp home internal
intranet lan local private test
Dec 18 16:33:27 mykubeworker systemd-resolved[12446]: Using system hostname
'mykubeworker'.
Dec 18 16:33:27 mykubeworker systemd[1]: Started Network Name Resolution.
Dec 18 16:33:27 mykubeworker falco: 16:33:27.143027083: Error Filesystem
modification detected (process=systemd-resolve [/lib/systemd/systemd-resolved])
command=/lib/systemd/systemd-resolved parent=systemd workingDir=/
user=systemd-resolve event=mkdir syscallParams=[ mode=0 ])
Dec 18 16:33:29 mykubeworker dhclient[3963]: DHCPDISCOVER on cali67b6d1b92f1 to
255.255.255.255 port 67 interval 20 (xid=0x3613aa29)
Dec 18 16:33:30 mykubeworker kubelet[3803]: I1218 16:33:30.194552 3803
log.go:172] http: TLS handshake error from 127.0.0.1:60608: EOF
Dec 18 16:33:30 mykubeworker dhclient[3963]: DHCPDISCOVER on calic82ce5602e6 to
255.255.255.255 port 67 interval 11 (xid=0xcd2df676)
Dec 18 16:33:32 mykubeworker dhclient[3963]: DHCPDISCOVER on cali23e2ed5333b to
255.255.255.255 port 67 interval 12 (xid=0x88732c20)
Dec 18 16:33:33 mykubeworker dhclient[3963]: DHCPDISCOVER on calia2ecd0e6842 to
255.255.255.255 port 67 interval 16 (xid=0x495a3734)
Dec 18 16:33:33 mykubeworker dhclient[3963]: DHCPDISCOVER on calia4cded2fdce to
255.255.255.255 port 67 interval 12 (xid=0x21021616)
Dec 18 16:33:34 mykubeworker dhclient[3963]: DHCPDISCOVER on cali62cbcbcfc4e to
255.255.255.255 port 67 interval 12 (xid=0x9c551b5f)
Dec 18 16:33:34 mykubeworker dhclient[3963]: DHCPDISCOVER on cali7a1f8cb0cb7 to
255.255.255.255 port 67 interval 14 (xid=0x2775512f)
Dec 18 16:33:38 mykubeworker dhclient[3963]: DHCPDISCOVER on cali993775d71fc to
255.255.255.255 port 67 interval 13 (xid=0x7b648445)
Dec 18 16:33:39 mykubeworker dhclient[3963]: DHCPDISCOVER on califd5f6897e93 to
255.255.255.255 port 67 interval 9 (xid=0x29f94242)
Dec 18 16:33:40 mykubeworker kubelet[3803]: I1218 16:33:40.195260 3803
log.go:172] http: TLS handshake error from 127.0.0.1:60744: EOF
Dec 18 16:33:40 mykubeworker dhclient[3963]: DHCPDISCOVER on calie82c4a2ad1a to
255.255.255.255 port 67 interval 20 (xid=0x8d37493e)
Dec 18 16:33:41 mykubeworker dhclient[3963]: DHCPDISCOVER on caliee1def00d5f to
255.255.255.255 port 67 interval 11 (xid=0x9d48c529)
Dec 18 16:33:41 mykubeworker dhclient[3963]: DHCPDISCOVER on calic82ce5602e6 to
255.255.255.255 port 67 interval 10 (xid=0xcd2df676)
Dec 18 16:33:42 mykubeworker dhclient[3963]: DHCPDISCOVER on cali3445044a923 to
255.255.255.255 port 67 interval 8 (xid=0xa904895f)
Dec 18 16:33:43 mykubeworker dhclient[3963]: No DHCPOFFERS received.
Dec 18 16:33:43 mykubeworker dhclient[3963]: No working leases in persistent
database - sleeping.
Dec 18 16:33:43 mykubeworker systemd[1]: Stopping Network Name Resolution...
Dec 18 16:33:43 mykubeworker systemd[1]: Stopped Network Name Resolution.
Dec 18 16:33:43 mykubeworker systemd[1]: Starting Network Name Resolution...
Dec 18 16:33:43 mykubeworker systemd-resolved[12661]: Positive Trust Anchors:
Dec 18 16:33:43 mykubeworker systemd-resolved[12661]: . IN DS 19036 8 2
49aac11d7b6f6446702e54a1607371607a1a41855200fd2ce1cdde32f24e8fb5
Dec 18 16:33:43 mykubeworker systemd-resolved[12661]: . IN DS 20326 8 2
e06d44b80b8f1d39a95c0b0d7c65d08458e880409bbc683457104237c7f8ec8d
Dec 18 16:33:43 mykubeworker systemd-resolved[12661]: Negative trust anchors:
10.in-addr.arpa 16.172.in-addr.arpa 17.172.in-addr.arpa 18.172.in-addr.arpa
19.172.in-addr.arpa 20.172.in-addr.arpa 21.172.in-addr.arpa 22.172.in-addr.arpa
23.172.in-addr.arpa 24.172.in-addr.arpa 25.172.in-addr.arpa 26.172.in-addr.arpa
27.172.in-addr.arpa 28.172.in-addr.arpa 29.172.in-addr.arpa 30.172.in-addr.arpa
31.172.in-addr.arpa 168.192.in-addr.arpa d.f.ip6.arpa corp home internal
intranet lan local private test
Dec 18 16:33:43 mykubeworker systemd-resolved[12661]: Using system hostname
'mykubeworker'.
Dec 18 16:33:43 mykubeworker systemd[1]: Started Network Name Resolution.
Dec 18 16:33:43 mykubeworker falco: 16:33:43.764889570: Error Filesystem
modification detected (process=systemd-resolve [/lib/systemd/systemd-resolved])
command=/lib/systemd/systemd-resolved parent=systemd workingDir=/
user=systemd-resolve event=mkdir syscallParams=[ mode=0 ])
Dec 18 16:33:44 mykubeworker dhclient[3963]: DHCPDISCOVER on cali23e2ed5333b to
255.255.255.255 port 67 interval 12 (xid=0x88732c20)
Dec 18 16:33:45 mykubeworker dhclient[3963]: DHCPDISCOVER on calia4cded2fdce to
255.255.255.255 port 67 interval 8 (xid=0x21021616)
Dec 18 16:33:47 mykubeworker dhclient[3963]: No DHCPOFFERS received.
Dec 18 16:33:47 mykubeworker dhclient[3963]: No working leases in persistent
database - sleeping.
Dec 18 16:33:47 mykubeworker systemd[1]: Stopping Network Name Resolution...
Dec 18 16:33:47 mykubeworker systemd[1]: Stopped Network Name Resolution.
Dec 18 16:33:47 mykubeworker systemd[1]: Starting Network Name Resolution...
Dec 18 16:33:47 mykubeworker systemd-resolved[12694]: Positive Trust Anchors:
Dec 18 16:33:47 mykubeworker systemd-resolved[12694]: . IN DS 19036 8 2
49aac11d7b6f6446702e54a1607371607a1a41855200fd2ce1cdde32f24e8fb5
Dec 18 16:33:47 mykubeworker systemd-resolved[12694]: . IN DS 20326 8 2
e06d44b80b8f1d39a95c0b0d7c65d08458e880409bbc683457104237c7f8ec8d
Dec 18 16:33:47 mykubeworker systemd-resolved[12694]: Negative trust anchors:
10.in-addr.arpa 16.172.in-addr.arpa 17.172.in-addr.arpa 18.172.in-addr.arpa
19.172.in-addr.arpa 20.172.in-addr.arpa 21.172.in-addr.arpa 22.172.in-addr.arpa
23.172.in-addr.arpa 24.172.in-addr.arpa 25.172.in-addr.arpa 26.172.in-addr.arpa
27.172.in-addr.arpa 28.172.in-addr.arpa 29.172.in-addr.arpa 30.172.in-addr.arpa
31.172.in-addr.arpa 168.192.in-addr.arpa d.f.ip6.arpa corp home internal
intranet lan local private test
Dec 18 16:33:47 mykubeworker systemd-resolved[12694]: Using system hostname
'mykubeworker'.
Dec 18 16:33:47 mykubeworker falco: 16:33:47.305057099: Error Filesystem
modification detected (process=systemd-resolve [/lib/systemd/systemd-resolved])
command=/lib/systemd/systemd-resolved parent=systemd workingDir=/
user=systemd-resolve event=mkdir syscallParams=[ mode=0 ])
Dec 18 16:33:47 mykubeworker systemd[1]: Started Network Name Resolution.
Dec 18 16:33:48 mykubeworker dhclient[3963]: DHCPDISCOVER on califd5f6897e93 to
255.255.255.255 port 67 interval 3 (xid=0x29f94242)
Dec 18 16:33:48 mykubeworker dhclient[3963]: DHCPDISCOVER on cali7a1f8cb0cb7 to
255.255.255.255 port 67 interval 1 (xid=0x2775512f)
Dec 18 16:33:49 mykubeworker dhclient[3963]: No DHCPOFFERS received.
Dec 18 16:33:49 mykubeworker dhclient[3963]: No working leases in persistent
database - sleeping.
Dec 18 16:33:49 mykubeworker systemd[1]: Stopping Network Name Resolution...
Dec 18 16:33:49 mykubeworker systemd[1]: Stopped Network Name Resolution.
Dec 18 16:33:49 mykubeworker systemd[1]: Starting Network Name Resolution...
Dec 18 16:33:49 mykubeworker systemd-resolved[12724]: Positive Trust Anchors:
Dec 18 16:33:49 mykubeworker systemd-resolved[12724]: . IN DS 19036 8 2
49aac11d7b6f6446702e54a1607371607a1a41855200fd2ce1cdde32f24e8fb5
Dec 18 16:33:49 mykubeworker systemd-resolved[12724]: . IN DS 20326 8 2
e06d44b80b8f1d39a95c0b0d7c65d08458e880409bbc683457104237c7f8ec8d
Dec 18 16:33:49 mykubeworker systemd-resolved[12724]: Negative trust anchors:
10.in-addr.arpa 16.172.in-addr.arpa 17.172.in-addr.arpa 18.172.in-addr.arpa
19.172.in-addr.arpa 20.172.in-addr.arpa 21.172.in-addr.arpa 22.172.in-addr.arpa
23.172.in-addr.arpa 24.172.in-addr.arpa 25.172.in-addr.arpa 26.172.in-addr.arpa
27.172.in-addr.arpa 28.172.in-addr.arpa 29.172.in-addr.arpa 30.172.in-addr.arpa
31.172.in-addr.arpa 168.192.in-addr.arpa d.f.ip6.arpa corp home internal
intranet lan local private test
Dec 18 16:33:49 mykubeworker systemd-resolved[12724]: Using system hostname
'mykubeworker'.
Dec 18 16:33:49 mykubeworker systemd[1]: Started Network Name Resolution.
Dec 18 16:33:49 mykubeworker dhclient[3963]: No DHCPOFFERS received.
Dec 18 16:33:49 mykubeworker dhclient[3963]: No working leases in persistent
database - sleeping.
Dec 18 16:33:49 mykubeworker systemd[1]: Stopping Network Name Resolution...
Dec 18 16:33:49 mykubeworker falco: 16:33:49.473565891: Error Filesystem
modification detected (process=systemd-resolve [/lib/systemd/systemd-resolved])
command=/lib/systemd/systemd-resolved parent=systemd workingDir=/
user=systemd-resolve event=mkdir syscallParams=[ mode=0 ])
Dec 18 16:33:49 mykubeworker systemd[1]: Stopped Network Name Resolution.
Dec 18 16:33:49 mykubeworker systemd[1]: Starting Network Name Resolution...
Dec 18 16:33:49 mykubeworker systemd-resolved[12759]: Positive Trust Anchors:
Dec 18 16:33:49 mykubeworker systemd-resolved[12759]: . IN DS 19036 8 2
49aac11d7b6f6446702e54a1607371607a1a41855200fd2ce1cdde32f24e8fb5
Dec 18 16:33:49 mykubeworker systemd-resolved[12759]: . IN DS 20326 8 2
e06d44b80b8f1d39a95c0b0d7c65d08458e880409bbc683457104237c7f8ec8d
Dec 18 16:33:49 mykubeworker systemd-resolved[12759]: Negative trust anchors:
10.in-addr.arpa 16.172.in-addr.arpa 17.172.in-addr.arpa 18.172.in-addr.arpa
19.172.in-addr.arpa 20.172.in-addr.arpa 21.172.in-addr.arpa 22.172.in-addr.arpa
23.172.in-addr.arpa 24.172.in-addr.arpa 25.172.in-addr.arpa 26.172.in-addr.arpa
27.172.in-addr.arpa 28.172.in-addr.arpa 29.172.in-addr.arpa 30.172.in-addr.arpa
31.172.in-addr.arpa 168.192.in-addr.arpa d.f.ip6.arpa corp home internal
intranet lan local private test
Dec 18 16:33:49 mykubeworker systemd-resolved[12759]: Using system hostname
'mykubeworker'.
Dec 18 16:33:49 mykubeworker systemd[1]: Started Network Name Resolution.
Dec 18 16:33:49 mykubeworker falco: 16:33:49.788638784: Error Filesystem
modification detected (process=systemd-resolve [/lib/systemd/systemd-resolved])
command=/lib/systemd/systemd-resolved parent=systemd workingDir=/
user=systemd-resolve event=mkdir syscallParams=[ mode=0 ])
Dec 18 16:33:49 mykubeworker dhclient[3963]: DHCPDISCOVER on cali67b6d1b92f1 to
255.255.255.255 port 67 interval 12 (xid=0x3613aa29)
Dec 18 16:33:50 mykubeworker dhclient[3963]: DHCPDISCOVER on cali3445044a923 to
255.255.255.255 port 67 interval 10 (xid=0xa904895f)
Dec 18 16:33:50 mykubeworker kubelet[3803]: I1218 16:33:50.196879 3803
log.go:172] http: TLS handshake error from 127.0.0.1:60874: EOF
Dec 18 16:33:51 mykubeworker dhclient[3963]: No DHCPOFFERS received.
Dec 18 16:33:51 mykubeworker dhclient[3963]: No working leases in persistent
database - sleeping.
Dec 18 16:33:51 mykubeworker systemd[1]: Stopping Network Name Resolution...
Dec 18 16:33:51 mykubeworker systemd[1]: Stopped Network Name Resolution.
Dec 18 16:33:51 mykubeworker systemd[1]: Starting Network Name Resolution...
Dec 18 16:33:51 mykubeworker systemd-resolved[12792]: Positive Trust Anchors:
Dec 18 16:33:51 mykubeworker systemd-resolved[12792]: . IN DS 19036 8 2
49aac11d7b6f6446702e54a1607371607a1a41855200fd2ce1cdde32f24e8fb5
Dec 18 16:33:51 mykubeworker systemd-resolved[12792]: . IN DS 20326 8 2
e06d44b80b8f1d39a95c0b0d7c65d08458e880409bbc683457104237c7f8ec8d
Dec 18 16:33:51 mykubeworker systemd-resolved[12792]: Negative trust anchors:
10.in-addr.arpa 16.172.in-addr.arpa 17.172.in-addr.arpa 18.172.in-addr.arpa
19.172.in-addr.arpa 20.172.in-addr.arpa 21.172.in-addr.arpa 22.172.in-addr.arpa
23.172.in-addr.arpa 24.172.in-addr.arpa 25.172.in-addr.arpa 26.172.in-addr.arpa
27.172.in-addr.arpa 28.172.in-addr.arpa 29.172.in-addr.arpa 30.172.in-addr.arpa
31.172.in-addr.arpa 168.192.in-addr.arpa d.f.ip6.arpa corp home internal
intranet lan local private test
Dec 18 16:33:51 mykubeworker systemd-resolved[12792]: Using system hostname
'mykubeworker'.
Dec 18 16:33:51 mykubeworker systemd[1]: Started Network Name Resolution.
Dec 18 16:33:51 mykubeworker falco: 16:33:51.695704199: Error Filesystem
modification detected (process=systemd-resolve [/lib/systemd/systemd-resolved])
command=/lib/systemd/systemd-resolved parent=systemd workingDir=/
user=systemd-resolve event=mkdir syscallParams=[ mode=0 ])
Dec 18 16:33:51 mykubeworker dhclient[3963]: No DHCPOFFERS received.
Dec 18 16:33:51 mykubeworker dhclient[3963]: No working leases in persistent
database - sleeping.
Dec 18 16:33:51 mykubeworker systemd[1]: Stopping Network Name Resolution...
Dec 18 16:33:51 mykubeworker systemd[1]: Stopped Network Name Resolution.
Dec 18 16:33:51 mykubeworker systemd[1]: systemd-resolved.service: Start
request repeated too quickly.
Dec 18 16:33:51 mykubeworker systemd[1]: systemd-resolved.service: Failed with
result 'start-limit-hit'.
Dec 18 16:33:51 mykubeworker systemd[1]: Failed to start Network Name
Resolution.
** Affects: systemd (Ubuntu)
Importance: Undecided
Status: New
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1908898
Title:
systemd-resolved service reach start-limit-hit if dhcp is not
reachable of a while
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1908898/+subscriptions
--
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
