Email or username:

Password:

Forgot your password?
Em's posts Post Back to profile
Em :official_verified:

Tiny Apocalyptic Time Tip 🌐✨

If you too feel uneasy about
the state of the world,
and you too worry about losing access to one of the greatest knowledge treasure of the internet,

Know that you can download an
offline copy of Wikipedia!

Here's how πŸ“šπŸ‘‡

1. Download the free and open source software Kiwix (this will be your reader): kiwix.org/en/applications/

2. If you want smaller versions of Wikipedia, you can download them within Kiwix.

Within the app, go to "Categories" in the menu on the left, then browse to a topic/version you want. Scroll to the bottom for Wikipedia mini, for example. Click on it then click "Download" on the right :neocat_book:

3. Once you have downloaded a database, click on "Opened" on the left > database you want to search > "Open Main Page" on the right.

4. You can use the Search field on the upper-right to find topics like on online Wikipedia! πŸ”

5. If you want the full English version of Wikipedia (110GB), you might want to download it from the torrent file instead. Install a torrent client of your choice (I use Transmission).

Then, go to this page, click on "Download - 109.89 GB" blue button on the first result (size may vary overtime), then select "Torrent file": library.kiwix.org/#lang=eng&ca

6. Once you have the torrent file, open it with your torrent client to start the download. This is BIG! Be patient! πŸ“¦

7. Once the download is completed, open your Wikipedia `.zim` file with Kiwix!

8. Magic! πŸ“–βœ¨

Extra Tip: You can download many other awesome knowledge files from the Kiwix Library! Personally, I also got the iFixit knowledge base! βš’οΈ :blobcatrainbow:

#Wikipedia #Kiwix #Offline #KnowledgeIsPower

Screenshot showing the application Kiwix with the open Wikipedia entry for Kiwix. Very meta.
22 January at 15:41 | Open on infosec.exchange
47 comments
Tyrion 🦜

@Em0nM4stodon

Thank you for this! I was having some issues (can't remember what) a month ago, and then kind of forgot I wanted to get a copy of Wikipedia. πŸ˜…

I'll try this out after lunch.

Edit: Awesome, the torrent is downloading. I just remembered that this was the issue. In the software itself, the download speed was terrible. Didn't know there was a torrent copy I could just import.

Edit 2: Something else it's worth pointing out, I didn't know this myself: Kiwix also has the ability to download a ton of other wikis and documentations aside from Wikipedia. I've downloaded a lot of programming stuff.

@Em0nM4stodon

Thank you for this! I was having some issues (can't remember what) a month ago, and then kind of forgot I wanted to get a copy of Wikipedia. πŸ˜…

I'll try this out after lunch.

Edit: Awesome, the torrent is downloading. I just remembered that this was the issue. In the software itself, the download speed was terrible. Didn't know there was a torrent copy I could just import.

22 January at 15:53 | Open on social.vivaldi.net
drybones263

@Em0nM4stodon I would have thought that the download size would be way larger for English Wikipedia.

22 January at 16:10 | Open on mastodon.social
Em :official_verified:

@blumoop Indeed! That is impressive engineering.

22 January at 16:19 | Open on infosec.exchange
anarcogordo

@Em0nM4stodon @blumoop Does it include images and media?
If it does then yes, thats impressive

22 January at 20:05 | Open on rebel.ar
Em :official_verified:
draeath

@Em0nM4stodon @blumoop @anarcogordo I'm guessing it doesn't keep edit history or other namespaces like user pages - but it would be neat if it did!

22 January at 22:25 | Open on infosec.exchange
trendless

@Em0nM4stodon makes me wonder if there are any pre LLM copies of the file floating around.. πŸ€”

22 January at 16:18 | Open on zeroes.ca
Em :official_verified:

@trendless I have a 2020 copy. Mild pandemic panic πŸ˜…

22 January at 16:21 | Open on infosec.exchange
Madeleine Morris

@Em0nM4stodon I am compelled to follow you... if only for this one response. 'Mild pandemic panic.'

I have an eerie feeling you're the kind of person who safeguards the entirety of human knowledge while the assholes of the world fiddle as it falls apart.

@trendless

22 January at 19:12 | Open on mstdn.social
Em :official_verified:
Madeleine Morris

@Em0nM4stodon @trendless

I just purchased a new external hard drive to house it. I'm having that "is this paranoia?" self-reflection, but at this time, I think it might be wise to just go with the paranoiac flow.

22 January at 20:54 | Open on mstdn.social
Alethe

@Em0nM4stodon When I read 110GB at first I thought it was just the text. Turns out that includes media. My mind is blown.

I was also inspired by your post to see if Khan Academy can be downloaded for offline use, and it turns out it's possible with a platform called Kolibri that provides educational resources for people or places without internet access:
learningequality.org/kolibri/

22 January at 17:38 | Open on tech.lgbt
Martin Rust

@alethez
Good. What about articles' edit histories and talk pages?
@Em0nM4stodon

22 January at 19:54 | Open on infosec.exchange
Alethe

@martinrust @Em0nM4stodon I couldn't find any specific information, but you can check out an online copy of Wikipedia at the Kiwix library:
library.kiwix.org/viewer#wikip

I couldn't find anything pointing to the history or talk pages so I assume they aren't saved. According to their FAQ they don't offer incremental updates either, so their versions are pretty much snapshots.

I also discovered that they are on Mastodon at @kiwix

@martinrust @Em0nM4stodon I couldn't find any specific information, but you can check out an online copy of Wikipedia at the Kiwix library:
library.kiwix.org/viewer#wikip

I couldn't find anything pointing to the history or talk pages so I assume they aren't saved. According to their FAQ they don't offer incremental updates either, so their versions are pretty much snapshots.

22 January at 20:43 | Open on tech.lgbt
Daniel DΓΌsentrieb

@Em0nM4stodon thank you, I didn't knew this existed. I'll immediately dive into this

22 January at 17:58 | Open on social.linux.pizza
Amro will grow out of it

@Em0nM4stodon I just did it, this afternoon and the second item I downloaded was the ifixit file :blobcatfingerguns:
I had to figure it out without your toot, was a bit fiddely. You explain it wel.
Great app

22 January at 18:01 | Open on todon.nl
Viral Obscurity

@Em0nM4stodon I've just added the full torrent. Might as well make good use of my 1 gig internet connection

I'd love to do this with the internet archive but that's far to big for me to store

22 January at 18:33 | Open on mstdn.social
Viss

@Em0nM4stodon are you aware of a way to keep an updated version of wikipedia locally? having a one time snapshot and only needing 110 gig is pretty rad (and i have the space, so i can trivially do this), but it would be nice to have a version that is updated, so if wikipedia ever goes offline, i'd in theory have the most recent updates and data available

22 January at 19:07 | Open on mastodon.social
Taggart :donor:

@Viss @Em0nM4stodon I am also interested in this, but recognize that information quality may in fact decrease with time at this point.

22 January at 19:08 | Open on infosec.exchange
Security Writer :verified: :donor:

@mttaggart @Viss @Em0nM4stodon I’ll third this. Just setting this up for a few site targets and I’d be interested in the delta sync and versioning (if it suddenly starts being vandalised or redacted)

22 January at 19:15 | Open on infosec.exchange
viq

@SecurityWriter @mttaggart @Viss @Em0nM4stodon I had a very quick look. There's torrents. I'd need to check whether they're for compressed or uncompressed contents - easier would be uncompressed. I'd set it up on zfs or btrfs, snapshot once one version is complete, then download updated version on top - unchanged files should stay the same, changed should detect they're different and get overwritten. Ideal would be an rsync mirror, or in some VCS

22 January at 19:23 | Open on social.hackerspace.pl
Joshua M πŸ‡¦πŸ‡Ί

@mttaggart @Viss @Em0nM4stodon its been decreasing for years.. calling Wikipedia a treasure at this point is a stretch far

22 January at 23:13 | Open on linuxrocks.online
Big George

@Viss @Em0nM4stodon would also like to know how to keep it updated.

22 January at 19:10 | Open on mastodon.africa
Third spruce tree on the left

@BigG @Viss @Em0nM4stodon Since the infrastructure is already in place to host/seed the torrent, and do the scraping/packaging, I'd suggest your better bet instead of downloading/scraping it yourself is get involved in the Kiwix project that creates that 110Gb zim - they used to be updated regularly, but the latest is the Jan 2024 one; who knows maybe that team is days away from releasing the 2025.01 zim and/or they just need some help?

22 January at 20:23 | Open on mas.to
Pokemod97
Madeleine Morris

@Em0nM4stodon @cstross

It worries me that I'm contemplating buying an external harddrive to house this on.

22 January at 19:09 | Open on mstdn.social
Crobibbly

There's also aard2. The text of English language Wikipedia (compressed file format) is about 20G. Linked from aarddict.org/
Multiple archived versions available.

@Remittancegirl @Em0nM4stodon @cstross

22 January at 19:56 | Open on mas.to
Menhera Lexi

@Em0nM4stodon@infosec.exchange Time to put my 2tb drive to use! Grabbing ifixit and Ask Ubuntu as well (desktop runs Ubuntu)

22 January at 19:17 | Open on catgirl.center
Celeste :verified_trans:

@Em0nM4stodon Since I'm building a "doomsday machine", this is exactly what I need!

The project started as a kind of fun-cyberpunk-machine but since the extreme-right in the US have made clear they want to attack Wikipedia and other open sources of information, this has become somewhat serious. I think the threats against Wikipedia are at least a little credible. So well, here I am, building a backup...

22 January at 19:21 | Open on infosec.exchange
Celeste :verified_trans:

@Em0nM4stodon When I read 110GB, I thought "just as big as some modern games"...

Downloaded!

22 January at 19:23 | Open on infosec.exchange
Em :official_verified:

@celeste_42bit Ah! Indeed indeed. And this is a game you can play for MANY hours 😁

22 January at 20:45 | Open on infosec.exchange
Bernie the Wordsmith

@Em0nM4stodon @celeste_42bit

This is a great idea! I just remembered the meme

Small Brain: Have Cyberpunk 2077 taking 100GB

Enlightened brain: Have indie games taking 100GB

Galaxy brain: Having Wikipedia taking 100GB of space to help avoiding Cyberpunk 2077 corporations becoming a reality and taking over

22 January at 20:51 | Open on masto.es
Stu

@celeste_42bit @Em0nM4stodon for some reason I never thought about it like that, but you're right. It's almost absurd, the sum total of knowledge, versus, I don't know, the Master Chief Collection.

22 January at 21:02 | Open on hachyderm.io
Pokemod97

@Em0nM4stodon They are currently working on updating the wikipedia downloader, so donations or volunteer hours would be nice. kiwix.org/en/wikipedia-offline

22 January at 19:25 | Open on social.treehouse.systems
sour

@Em0nM4stodon@infosec.exchange

do you know where the .zim gets downloded to so i can put it on a hard drive

22 January at 19:28 | Open on hol.ogra.ph
irelephant

@Em0nM4stodon@infosec.exchange I would like to add, you can archive a website into a zim with https://zimit.kiwix.org

22 January at 20:24 | Open on calckey.world
Internet Rando

@Em0nM4stodon I feel like it's worth mentioning that #internetinabox (internet-in-a-box.org) ships with #kiwix, and a variety of tools to simplify the creation and distribuiton of (optionally) offline hotspots to share these collected works as well.

I've prototyped a couple in my rPi400 and it works really well, 10/10 would recommend for redundant distributions of valuable educational resources, with a front end for non-techies.

Bonus: it's basically just a #debian distro

22 January at 20:44 | Open on mastodon.seattlematrix.org
Joe Turner

@Em0nM4stodon How do you tackle the updates? how is that handled?

22 January at 20:48 | Open on mas.to
Kevin Russell

@Em0nM4stodon

Hey World you can download the entire Wikipedia. There are versions in some other languages, and there are offline translation tools.

Bonjour monde, vous pouvez tΓ©lΓ©charger toute la WikipΓ©dia. Il y a des versions dans d'autres langues, et il y a des outils de traduction hors ligne.

Halo dunia Anda dapat download seluruh Wikipedia. Ada versi dalam beberapa bahasa lain, dan ada perangkat terjemahan offline.

#wikipedia

22 January at 21:21 | Open on mstdn.social
womp

@Em0nM4stodon This has been mentioned on the #prepping hashtag and reddits before. It's a great thing to have for any offline scenario.

I anticipate the #prep, #prepper related hashtags to be more active going forward. 🫀

22 January at 21:23 | Open on mastodon.cloud
Ember

@Em0nM4stodon Thank you SO much! I am downloading it right now. We see the orange menace already trying to rewrite history so I’m sure they’re going to have plenty of cronies trying to rewrite Wikipedia. 🫣

22 January at 21:34 | Open on mastodon.social
Kyle Kurth

@Em0nM4stodon Can it do things like NIST and CISA? It would be good to scrape all that info before it's shut down too.

22 January at 21:38 | Open on twit.social
Tom McNeely

@Em0nM4stodon
If you don't have the bandwidth for these huge downloads, you can buy flash drives with the data files on them, say, on eBay. I'd love to be able to buy them directly from the Wikimedia foundation, and give them my money instead. (I do donate.)

22 January at 21:39 | Open on subdued.social
Rich Puchalsky β©œβƒ

@Em0nM4stodon

Thanks for this: it's a good thing. Do you know of any way to get an archive copy of an earlier version of wiki from a previous time? I want to archive one from October 2022 before the first big LLMs were released.

22 January at 22:02 | Open on mastodon.social
BardMoss the Linux Guy

@Em0nM4stodon
You could also install Endless OS and have weeks worth of stuff to go through, including educational games, a very robust set of Wikipedia, dictionaries, and other useful items.

yesterday at 2:46 | Open on hostux.social
Lee Hauser

@Em0nM4stodon @pronoiac I got this a few weeks ago, and picked up a copy of Project Gutenberg while I was there.

yesterday at 5:23 | Open on mefi.social
Go Up