@matt hmm, it might be easier to have a structured conversation investigation perf issues in a forum where it's easier to share log files and the like. Is there a github issue tracking this already?

@matt

It might not be a quick fix, but spinning off older entries into their own quasi-archive tables could help, especially if they're ones that are rarely accessed. You'd need a routing mechanism somewhere to detect when to jump from the main table to the archives.

And do you have something like Redis in place for caching common / recent queries? That could help for material that's frequently accessed but doesn't change often.

@matt if you haven't already, turn on slow query logging, it will expose the problem queries and you can tune indexes from there. You didn't mention the DB engine you are using, but MySQL can be an absolute nightmare to do this, I do it all the time.

@matt Unfortuantely I know nothing about your stack or your code (other than it being MySQL-backed). However, here's the general troubleshooting lists of things that make a database-backed web application slow:

1. Too many queries: app is pulling lots of things from the DB constantly that could be cached, or doing joins in the app code

2. Unoptimized requests: some DB requests need better queries/better data structure/indexes

...