 #MastoAdmin malarkey follows... So I went through the upgrade process to bring my masto instance up to v3.3.0 last night and I've been fighting ever since with a weird problem. I cannot use it with the web front end anymore! Whenever I try to go to the web page I get a "fake 404" page that says "File /poco does not exist" and does a directory listing of my root dir! Oddly enough everything else works (API and streaming and admin URLs, and I am federation and can use Tusky on a mobile fine!) 14 comments
 @msh I am not familiar with anything named "poco" in Mastodon. It must be your nginx config or something. @Gargron until I tried upgrading my instance I wasn't aware of anything called poco either! There is nothing to do with poco in my nginx config so I guess I'll start looking into "or something" :blobshrug: @msh I don't think nginx does a directory listing by default either. You sure you don't have Apache or something hogging the ports? @Gargron I'm certain there is no apache installed and when I stop nginx and Mastodon services nothing is listening on the ports. The behaviour started following upgrade from 3.2.0 to 3.3.0 (I hadn't gotten to upgrading the 3.2.x but that wouldn't have anything to do with this right?) @Gargron https://coales.co is my instance and as you can see it's not right when you go there. Oddly anything not that or in any url not beginning in /web/... still works and the API still works so I am resorting to tusky and pinafore to use my instance. For example I can still go to /about or /public and do my admin and moderation at /admin @msh This is very wrong. This does not look like a nginx directory listing. And Mastodon doesn't have anything that produces that output. It's listing files in the Mastodon root directory, not public! Thankfully .env.production can't be retrieved, but still. I would start suspecting foul play at this point. Some kind of malware or hack? @msh Also check the public directory for an index.html file, it could hijack the / path like that if it existed, it's not supposed to exist. But check what/who created it, if that's what's happening. @msh By the way if it's any clue it seems like that index.html is two different pages concatenated, if you look at the source. First it's the 404 page about /poco and then a directory listing page. @Gargron you're right, I hadn't noticed the two stes of <html>...</html> before! There isn't any indication of compromise in my system and no stray index.html (the whole mastodon direrctory was blown away then live was pulled in from git again). But there is progress. I *can* go to https://.../web and it will let me in again, but that initial redirect from the root URL still does the poco thing. Maybe something is cached somewhere from the first botched upgrade? @msh Eh, could be, then delete the nginx cache directory and see what happens Fantastic! That did the trick. I'm still bothered that I haven't determined the root cause yet, though it definitely started happening immediately after I did the first upgrade attempt. Thanks for the tips! I kinda tossed you in my mentions tongue-in-cheek so it was quite beyond my expectations! There is no way any closed source product or service or anything from the Big Internet Silos would EVER step up like that. |
Gregory's Server
OK I nuked my masto install Directory completely and reinstalled and restored and it's STILL borked! What is this poco and why is my web front end complaining about it being missing?!
@Gargron ?
Anyone?
Bueller?