It will serve time if it receives enough valid packets, possibly from other sources. If you use the default setup, it uses multiple peers so you should be ok.
Well, in this case, my box is the server (time.cdmon.com) and these kind of packets are sendend by the ntp clients.
Btw, more recent versions of ntpd just do not log these messagesanymore.
;) -- Thanks, Jordi Espasa Clofent