 People say, "The Internet is forever". This is not true. So much on the Internet has disappeared. I would state further that more has disappeared from the Internet than currently what exists. This is one reason I wish that a P2P content delivery system was the default: because not only would it deliver information faster, information would depend less on WHERE it is and more on WHAT it is -- thereby creating an avenue for redundancies. 60 comments
One reason I hope PeerTube takes off is because: 1. It's decentralized That's one step in the right direction -- but I hope other social media networks follow their lead. Actually, I kind of wish there was a universal P2P protocol that was a mixture of HTTP and BitTorrent. That alone would fix so many problems with the Internet! @anji I've used IPFS for three years, and I've yet to see broad adoption apart from crypto. If something like archive.org were stored on IPFS, then that would be a game changer. @polychrome @anji Okay, but hear me out. What if archive.org was decentralized, and every library and university in the world ran an instance? @polychrome @atomicpoet @anji it's true - you cannot trust anyone who's motives are not preservation to preserve your media. Archiving has been getting increasingly difficult and expensive over the years as the volume and diversity of media goes up, and it's expensive. I'd go one further and say optical media - not necessarily CD/DVD, though - is the way to go - formed by irreversible chemical/ mechanical processes. Tapes and disks are fine - but are erasable and so less durable. @polychrome @atomicpoet @anji also - tape heads have a finite lifetime (in hours read). Many kinds of tape machines (and this heads) which were once common are no longer manufactured: thus there is a finite supply of tape heads. There are archives in the world which have more hours of media stored in them than there are tape head hours in the world. So some of the archive is already lost - it's just we have to decide which bit we don't recover.  @polychrome @atomicpoet @anji LTO Tape drives are quite popular for archival purposes, as they don't lose data easily when not in use for a long time. @polychrome @atomicpoet @anji You can't expect non-commercial instance to be reliable if you don't support it. Support instance (mastodon, peertube, etc..) or setup/share your own and will last forever.  I plan to use the 3.5"HDs to store expanded source trees, software/distro, and the complete EXR-stream renders of my output. The EXRs are "intermediates". Regenerating them is expensive, but it is an automatic process, once the software is running. I'll use the optical M-Disc media to store source files, PNG streams, video renders, and software archives. As for the volatility of PeerTube, that's why I'm running my own instance, now. Hopefully this works. 🤞  @polychrome @atomicpoet @anji the recent threat by #Google to oust #GSuiteLegacy users who wouldn't pay the ransom gave a new perspective on the net-archives, faced as I was with somehow finding new homes for 16 years worth of life-history data for 8 users. I think we must accept that "Digital Archive" is a contradiction in terms, a transient transport from A to B. Digital 'artifacts' are a 'volatile' variable contained within a scope that will inevitably be garbage collected. @atomicpoet @anji It looks like archive.org is actually planning to use IPFS. Have to look up the source later, currently at work. @atomicpoet @anji ok, here's a follow up to this. Looks like they removed any evidence for that, the only thing I could fine was a cut version of the interview on archive.org. The whole interview isn't available anymore. @anji @atomicpoet ipfs backs a lot of Library Genesis. It's working well enough for them. I think it's just a matter of time.  @atomicpoet IPFS is one of the distribution mechanisms used by Library Genesis.   @atomicpoet Welcome to the #interpeer project, or at least where I hope it'll be in a few months. @atomicpoet For something like archiving, IPFS may actually be a good choice. The goal for us is broader, and also support real-time scenarios such as live broad-/manycasting, as well as collaborative editing. IPFS has a few features that mean it doesn't lend itself all that well to those scenarios where there are frequent updates to a resource. @atomicpoet imo that would be great not only from data preservation standpoint, but also just because it's more convenient. I can't even count how many times I had to waste time waiting for slow CDNs that are thousands of kilometers away to give me data I want. Decentralization not only makes it more persistent, but also makes content delivery much faster. But I guess at that point in time average person won't really care about that, unfortunately.  @atomicpoet   @atomicpoet I also like the aims of PeerTube, but being based on ActivityPub it seems to have some of the same limitations as Mastodon. Lack of global search, discovery, trends, etc. It would be nice to be able to easily search for and discover all content everywhere from any server… That’s part of what makes YouTube (and Twitter) so good. @anji @atomicpoet I'm 99% sure it would be possible to make something like this with the existing videos on PeerTube based on ActivityPub. I'm 0% sure you can currently easily search for and discover all content currently on YouTube, let alone all the content that's been deleted because YouTube didn't like it. @raphaelmorgan @atomicpoet If someone on your local server doesn’t follow or boost a remote post/video, it never appears and cannot be found in search… That’s a problem, I think. Not arguing that a centralized solution is better than a distributed one! But the ActivityPub follow/boost model which doesn’t proactively announce a servers content to the entire network frustrates me sometimes.  @anji @atomicpoet But here is the thing (notice that I don't understand much about fediverse yet). Peertube, in its own, is decentralized and uses P2P, so this scenario is harder to happen than Youtube or other platforms. But if a specific Peertube instance goes down, what happens with the videos stored on that specific instance? Same thing with Funkwhale instances, Pixelfed instances and other instances of fediverse webpages. @icg937 Actually, my Pixelfed instance has gone down multiple times. You could still view the content from other instances. That's the magic of ActivityPub. @atomicpoet FB deleted my notes that had info I wanted to hold onto... f-n TEXT... tiny KBs.... and no (real) warning. That pretty much did it for me. F FB  @atomicpoet Stuff on P2P systems just disappear in different ways than stuff on corporate infrastructure. It does make making copies of data easier, but there isn't an inherent pattern preventing data loss. I've seen my fair share of stale torrents of unique/rare data :) @wmd Sure, we've talked about this on the rest of the thread. The problem with P2P is persistence. However, that problem can be easily solved. @atomicpoet If you want to have information security in terms of availability, you better use your personal hardware and backup strategy. @nicolegoebel That's a good practice to have but it doesn't address the core problem I'm talking about. @atomicpoet tbh I don't think that the loss of such platforms is a real tragedy. If the data is worth sth, it will pop up again elswhere. Well, if anyone wants to move their #videos to #WordPress, send me some details and I'll give you a quote. We might need to discuss transcoding for all browsers. I mostly host #blogs with photos, but I'd like to have more video clients, that seems like the future. If you have a lot of big videos, I'd probably offer two rates, one with daily offsite #backups and one without offsite backups. For offsite backups of videos, I'd probably need a new 'rsync' script. @pj @atomicpoet  @atomicpoet On the other paw, that means you can't delete stuff. /ever./ And sometimes deleting stuff is important.  @atomicpoet What I mean when I say that the internet is forever is that once something's been put on it, you can never be 100% sure it's been removed.
@atomicpoet I'm in the process of converting my blog from Jekyll to Hugo. I'm making sure that the URL hierarchy stays the same. Going through some older posts almost none of the links work any more. Are they up in Wayback? Maybe but it's not like it's an official archive or anything. I've thought about writing something that does "sip to local" for shared posts and then if the original link disappears redirect to the local copy.
@atomicpoet @woozle the internet only remembers if it gives it joy (your embarrassments), not what should be archived (the driver you needed)
ofc p2p only works if someone recognized its value two decades ago…  Your chart is ready, and can be found here: https://www.solipsys.co.uk/Chartodon/108510551890790366.svg Things may have changed since I started compiling that, and some things may have been inaccessible. In particular, the very nature of the fediverse means some toots may never have made it to my instance, in which case I can't see them, and can't include them. The chart will eventually be deleted, so if you'd like to keep it, make sure you download a copy. @atomicpoet This is something I stumbled into just a few weeks ago, and not even with video content. @atomicpoet On the creation vs. destruction rates: Total Internet data is still growing AFAIU. The doubling rate has been about 18 months to 2 years, for decades now (and pre-dating both the Web and Internet). So unless half of all data is lost within 18--24 months, no, the total would still be growing. Though average lifespan is likely low. Internet Archive's metric is that the typical webpage on last 50 days until deleted or changed. @atomicpoet the best argument against 100% P2P hosting is the millions of dead torrents. Ideally you'd have a system where there is multiple local points of centralisation, that can also keep functioning based on direct P2P between users in case the central entity disappears. So basically peertube, but other instances should be able to archive content form other instances as long as it is shared by some user/seeder. @atomicpoet So if one instance disappears, and the video stops being centrally hosted, it should still be indexed and made accessible by other instances as long as there are people seeding it. I'm not sure if peertube does this atm |
Gregory's Server
Imagine if YouTube was shut down.
It would be a literal tragedy for the human race. Billions of hours worth of creativity would disappear in an instant.
"That would never happen!" some believe.
But it already has.
Remember Google+? All of it's gone forever.
Or remember all that media stored on MySpace? It's vanished.
We must stop depending on Big Tech to archive our data. Their mandate is to profit off our data, not preserve it.