A fedi-tip or whatever it's called for people that follow a fuck-ton of people by accident like me:
Make a "friends" list of the people you're closest to and mostly follow that instead (add it as a tab in Tusky, or a column in the advanced web view, or whatever your app can do), and dip into the regular following list for a few mins while you're waiting for the kettle to boil or something. Much nicer and more meaningful connections, and best of both worlds really x
Also get out of the habit of scrolling for hours, I dip in for a few minutes here and there through the day and if I miss something I miss something, no big deal x