Thanks for the idea.
I did resolve this a day or two ago.

The story is; 
This server was a fairly recent replacement for an older Ubuntu setup. The new 
server as well as the old one are/were VM's - yet on different VM platforms. 
The old VM was turned off, and was marked never to start except unless manually 
started. [There were a few other things on the VM host that had yet to be 
migrated - so we didn't want it entirely off quite yet.]

The problem happened again in the last day or two - and packet captures showed 
that no packets were even arriving at the new VM.
Since there really wasn't anything that should be blocking that traffic, I 
checked the arp table on a problem client. 
The arp table showed an "incorrect" MAC address for the current BIND server. 
[The MAC in the arp table didn't match the MAC for the new VM.]

While I didn't have the MAC address for the "old" deactivated server handy, it 
was the first obvious problem/solution to check.
Sure enough, after connection to the VM hypervisor, I could see that the "old" 
BIND vm was active.

I killed it, and service returned to normal.

So, the "solution" was pretty routine.
What made it more "interesting" and perhaps odd is how seemingly randomly the 
problem would crop up. 
And it would only impact some clients, not all. There was no pattern that 
seemed to explain why some got the current/correct BIND server and others 
didn't. [The arp poisoning certainly wasn't anywhere nearly universal.]
And why was it so infrequent - it would go many days between issues.
I have to assume the bad VM had been up for some time, at least since the 
problems started.
There are quite a number of odd-ish other things too, but not worth detailing.

Probably it's just one of those "undefined" situations where you can't 
anticipate some predictable order to what happens when you screw it up. Rather 
than burn additional time trying to grok what was going on - it's simply best 
to say "don't do that - bad things happen, though I can't say what bad things 
will happen and in which logical order. They just will - so DON'T DO THAT!"

[And yeah, I obviously knew all about not doing that. But it happened anyway, 
in spite of specific steps to prevent it. I'm still not sure why.]

In the end, it's a somewhat complicated story with a very obvious cause - but 
it wasn't so clear at the outset.

TLDR version; 
Don't run your old and new bind servers on the same IP address - ether by 
accident or intentionally. Bad stuff will happen! 
It might be really odd, or it might be plain as day -  but in either case it 
won't be good! :)

Thanks all for the suggestions! 
Here's hoping I don't need to ask for BIND assistance for another 20 years! :)

-Greg



I just randomly spotted this post, and thought I would toss in 2ยข

How many nics and how many it's are on the servers?  Are the failing clients on 
the same subnet as the server?

--
_______________________________________________
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe 
from this list

bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users

Reply via email to