But I tried switching to iCloud custom domain purely and it worked with it, then I switched back and it kept working until I edited the SPF record.

I'm rate limited right now to keep experimenting, though.