|
Top-level
 @bear "I saw a data scrape" by who, and are you sure it was a scrape? There's a misnomer that activitypub is push-only, this is incorrect & you will actually see a LOT of GET requests retrieving data from your server. Do you have authorized fetch enabled? It can help somewhat. Most fediverse User-Agents also advertise where they're from. 4 comments
@bear read requests in ActivityPub won't necessarily correlate to an increase in sidekiq jobs (unless it's to verify the reader's signatures, but even that is generally done in the request/response lifecycle) As for a walk of your CDN, were there any user agents or IP addresses popping in the logs to indicate a crawl? Some fediverse clients behave in varying ways (proxying requests, requesting additional data, etc), so patterns would be important here. @thisismissem Good to know, regarding no increase in Sidekiq activity. I guess I still have a lot to learn. π What I recall from that moment was that everything stood out like a spike in the CloudFront dashboards. Within a short time window it looked like every asset was touched. I remember it being hard to determine who the client was, but that it looked like one client. @bear there are some fediverse software projects that'll try to backfill posts when they become aware of a user, perhaps it's this gone awry and is a bug instead of a bad actor? Though, also, Jortage is gaining popularity, which is an alternative to storing all the content yourself. |
Gregory's Server
@thisismissem Within the first week of operating an instance, I suddenly saw a walk of all of the CDN assets, including cached media from other instances. At first I assumed it was some kind of index, but I was able to corroborate this action by reviewing nginx logs. I did not see an uptick in Sidekiq.
As a new admin, I wasn't sure how to gather more information to understand what would cause a 1:1 serving of GB of media from my CDN I had just set up.