@zachleat thanks for writing this Zach!

On a project a while ago we ran into a lot of headaches trying to compare Lighthouse scores over time to track performance changes.

Even the difference between running mobile lighthouse tests on a new M1 Macbook and an older Intel-powered Macbook was huge. We ended up having to change our strategy and rely more on tools like webpagetest.org.

I wish there was more info out there about how device variability impacts these scores!