@marcan I share these frustrations. I really want to like it have have it as a viable chat platform but the jank is real. Element on iOS has weird issues especially with long syncs, and Element X is supposed to solve this with "sliding sync" but in practice it's even less usable because chats don't sort by recent messages and you can't decrypt old messages ("coming soon"). The Cinny client is the most usable one I've found so far but it also has weird issues especially around joining new Spaces and rooms within those spaces.
It probably doesn't helm that the main people behind Matrix seem to be working more on features and their server implementation rewrite rather than improving stability of what exists. I'm hoping Conduit, a third party server software, can make up some ground.