Today I learned that Linux (or at least certainly Ubuntu) will enable power saving mode on a wifi device by default. This resulted in me getting ~300ms pings to my machine on my LAN and very laggy ssh sessions. You can turn it off permanently as described here (or temporarily as described in the question post):

https://unix.stackexchange.com/questions/269661/how-to-turn-off-wireless-power-management-permanently/315400#315400