X11 is more of a display server. Wayland is more like a display server protocol. In the context of wayland, the compositors that implement the wayland protocol are indeed "display servers".
Luckily we don't really need to care about any of that. Well, not unless we want to work on one. 😅