Attached are iptables from master/hosts. I think they look normal. No firewall (as far as I know).
Heath Professor Graduate Coordinator Chemical and Biological Engineering http://che.eng.ua.edu<http://che.eng.ua.edu/> University of Alabama 3448 SEC, Box 870203<x-apple-data-detectors://1/0> Tuscaloosa, AL 35487<x-apple-data-detectors://1/0> (205) 348-1733<tel:(205)%20348-1733> (phone) (205) 561-7450 (cell) (205) 348-7558<tel:(205)%20348-7558> (fax) htur...@eng.ua.edu<mailto:htur...@eng.ua.edu> http://turnerresearchgroup.ua.edu From: slurm-users [mailto:slurm-users-boun...@lists.schedmd.com] On Behalf Of Fulcomer, Samuel Sent: Monday, May 21, 2018 10:15 AM To: Slurm User Community List <slurm-users@lists.schedmd.com> Subject: Re: [slurm-users] network/communication failure Is there a firewall turned on? What does "iptables -L -v" report on the three hosts? On Mon, May 21, 2018 at 11:05 AM, Turner, Heath <htur...@eng.ua.edu<mailto:htur...@eng.ua.edu>> wrote: If anyone has advice, I would really appreciate... I am running (just installed) slurm-11.17.6, with a master + 2 hosts. It works locally on the master (controller + execution). However, I cannot establish communication from master [triumph01] with the 2 hosts [triumph02,triumph03]. Here is some more info: 1. munge is running, and munge verification tests all pass. 2. system clocks are in sync on master/hosts. 3. identical slurm.conf files are on master/hosts. 4. configuration of resources (memory/cpus/etc) are correct and have been confirmed on all machines (all hardware is identical). 5. I have attached: a) slurm.conf b) log file from master slurmctld c) log file from host slurmd Any ideas about what to try next? Heath Turner Professor Graduate Coordinator Chemical and Biological Engineering http://che.eng.ua.edu University of Alabama 3448 SEC, Box 870203 Tuscaloosa, AL 35487 (205) 348-1733 (phone) (205) 561-7450 (cell) (205) 348-7558 (fax) htur...@eng.ua.edu<mailto:htur...@eng.ua.edu> http://turnerresearchgroup.ua.edu
Chain INPUT (policy DROP 0 packets, 0 bytes) pkts bytes target prot opt in out source destination 281K 929M ACCEPT all -- lo any anywhere anywhere 771M 392G ACCEPT all -- any any anywhere anywhere ctstate ESTABLISHED 0 0 ACCEPT icmp -- any any anywhere anywhere ctstate RELATED 1015K 98M input_ext all -- any any anywhere anywhere 0 0 LOG all -- any any anywhere anywhere limit: avg 3/min burst 5 LOG level warning tcp-options ip-options prefix "SFW2-IN-ILL-TARGET " 0 0 DROP all -- any any anywhere anywhere Chain FORWARD (policy DROP 0 packets, 0 bytes) pkts bytes target prot opt in out source destination 0 0 LOG all -- any any anywhere anywhere limit: avg 3/min burst 5 LOG level warning tcp-options ip-options prefix "SFW2-FWD-ILL-ROUTING " Chain OUTPUT (policy ACCEPT 243M packets, 1885G bytes) pkts bytes target prot opt in out source destination 281K 929M ACCEPT all -- any lo anywhere anywhere Chain forward_ext (0 references) pkts bytes target prot opt in out source destination Chain input_ext (1 references) pkts bytes target prot opt in out source destination 207K 72M DROP all -- any any anywhere anywhere PKTTYPE = broadcast 0 0 ACCEPT icmp -- any any anywhere anywhere icmp source-quench 2 168 ACCEPT icmp -- any any anywhere anywhere icmp echo-request 30 1800 LOG tcp -- any any anywhere anywhere limit: avg 3/min burst 5 tcp dpt:ssh flags:FIN,SYN,RST,ACK/SYN LOG level warning tcp-options ip-options prefix "SFW2-INext-ACC-TCP " 30 1800 ACCEPT tcp -- any any anywhere anywhere tcp dpt:ssh 798K 26M DROP all -- any any anywhere anywhere PKTTYPE = multicast 0 0 DROP all -- any any anywhere anywhere PKTTYPE = broadcast 9604 576K LOG tcp -- any any anywhere anywhere limit: avg 3/min burst 5 tcp flags:FIN,SYN,RST,ACK/SYN LOG level warning tcp-options ip-options prefix "SFW2-INext-DROP-DEFLT " 0 0 LOG icmp -- any any anywhere anywhere limit: avg 3/min burst 5 LOG level warning tcp-options ip-options prefix "SFW2-INext-DROP-DEFLT " 174 14616 LOG udp -- any any anywhere anywhere limit: avg 3/min burst 5 ctstate NEW LOG level warning tcp-options ip-options prefix "SFW2-INext-DROP-DEFLT " 9793 591K DROP all -- any any anywhere anywhere Chain reject_func (0 references) pkts bytes target prot opt in out source destination 0 0 REJECT tcp -- any any anywhere anywhere reject-with tcp-reset 0 0 REJECT udp -- any any anywhere anywhere reject-with icmp-port-unreachable 0 0 REJECT all -- any any anywhere anywhere reject-with icmp-proto-unreachable
Chain INPUT (policy DROP 0 packets, 0 bytes) pkts bytes target prot opt in out source destination 457K 1334M ACCEPT all -- lo any anywhere anywhere 1108M 662G ACCEPT all -- any any anywhere anywhere ctstate ESTABLISHED 0 0 ACCEPT icmp -- any any anywhere anywhere ctstate RELATED 1320K 116M input_ext all -- any any anywhere anywhere 0 0 LOG all -- any any anywhere anywhere limit: avg 3/min burst 5 LOG level warning tcp-options ip-options prefix "SFW2-IN-ILL-TARGET " 0 0 DROP all -- any any anywhere anywhere Chain FORWARD (policy DROP 0 packets, 0 bytes) pkts bytes target prot opt in out source destination 0 0 LOG all -- any any anywhere anywhere limit: avg 3/min burst 5 LOG level warning tcp-options ip-options prefix "SFW2-FWD-ILL-ROUTING " Chain OUTPUT (policy ACCEPT 424M packets, 2600G bytes) pkts bytes target prot opt in out source destination 457K 1334M ACCEPT all -- any lo anywhere anywhere Chain forward_ext (0 references) pkts bytes target prot opt in out source destination Chain input_ext (1 references) pkts bytes target prot opt in out source destination 207K 72M DROP all -- any any anywhere anywhere PKTTYPE = broadcast 0 0 ACCEPT icmp -- any any anywhere anywhere icmp source-quench 3 252 ACCEPT icmp -- any any anywhere anywhere icmp echo-request 117 7020 LOG tcp -- any any anywhere anywhere limit: avg 3/min burst 5 tcp dpt:ssh flags:FIN,SYN,RST,ACK/SYN LOG level warning tcp-options ip-options prefix "SFW2-INext-ACC-TCP " 131 7860 ACCEPT tcp -- any any anywhere anywhere tcp dpt:ssh 798K 26M DROP all -- any any anywhere anywhere PKTTYPE = multicast 0 0 DROP all -- any any anywhere anywhere PKTTYPE = broadcast 23953 1437K LOG tcp -- any any anywhere anywhere limit: avg 3/min burst 5 tcp flags:FIN,SYN,RST,ACK/SYN LOG level warning tcp-options ip-options prefix "SFW2-INext-DROP-DEFLT " 0 0 LOG icmp -- any any anywhere anywhere limit: avg 3/min burst 5 LOG level warning tcp-options ip-options prefix "SFW2-INext-DROP-DEFLT " 174 14616 LOG udp -- any any anywhere anywhere limit: avg 3/min burst 5 ctstate NEW LOG level warning tcp-options ip-options prefix "SFW2-INext-DROP-DEFLT " 314K 19M DROP all -- any any anywhere anywhere Chain reject_func (0 references) pkts bytes target prot opt in out source destination 0 0 REJECT tcp -- any any anywhere anywhere reject-with tcp-reset 0 0 REJECT udp -- any any anywhere anywhere reject-with icmp-port-unreachable 0 0 REJECT all -- any any anywhere anywhere reject-with icmp-proto-unreachable