Yeah turned out there was something going on that I've never seen before. Two network cards were fighting over the same IP address. They were assigned different addresses but from the ISP equipment's side the IP address was changing interfaces.

I removed the 2nd network card and the problem stopped.