Hi Erik
There is a small possibility that you have a hardware problem.
10 years ago I bought a DECT telephone that interfered with my cable modem if the base station was placed within 50cm of the modem. Got up to 8% packet loss when testing with "ping -f" to the gateway IP.
Simply moving the base station to the other side of my desk fixed the problem.
Did you recently buy something that emits a lot of radio waves? Do you frequently pose your mobile phone near your modem?
No, not really. The modem is far away from mobile phones or even from the DECT station. I fear that it's not a modem problem, more a routing problem. When I connect to IPs which are bad in connectivity through my VPN rather than directly, all is good. I don't think the modem has a problem here (Layer 2 bridge doesn't do routing).
Cheers