Lemmy v0.19.6 Release

What is Lemmy?

Lemmy is a self-hosted social link aggregation and discussion platform. It is completely free and open, and not controlled by any company. This means that there is no advertising, tracking, or secret algorithms. Content is organized into communities, so it is easy to subscribe to topics that you are interested in, and ignore others. Voting is used to bring the most interesting items to the top.

Changes

This release took a long time to complete due to a major performance problem which brought lemmy.ml to a crawl every time we tried to deploy the new version. It took a lot of testing (in production) to narrow it down to a single commit, and finally fix the problem.

The release itself contains numerous bug fixes and minor improvements:

Lemmy Enhancements

- Parallel federation sending by @phiresky in #4623
- Reduce CPU usage for generating link previews by @phiresky in #4957
- Switch from OpenSSL to rustls by @kwaa in #4901
- Increase max post url length to 2000 characters by @dessalines in #4960
- Increase max length of user bio to 1000 charactes by @dessalines #5014
- Reduce maximum comment depth to 50 by @nutomic #5009
- Resize post thumbnails by @nutomic #5107/files
- Add category to RSS feeds by @nutomic #5030
- Allow users to view their own removed/deleted communities by @dessalines in #4912
- Add backend check to enforce hierarchy of admins and mods by @dessalines in #4860
- Do pictrs transformations for proxied image urls by @dessalines in #4895
- Enable more build optimizations by @nutomic in #5168
- Calculate “controversial” ranking with exponent instead of multiply (just like Reddit) by @dullbananas in #4872
- Automatically remove tracking parameters from URLs by @dessalines #5018
- Relax timeout for sending activities by @Nothing4You in #4864

Bug Fixes

- Fix admin notification for new user registration (fixes #4916) by @Nutomic in #4925
- Allow community settings changes by remote mods @flamingo-cant-draw in #4937
- Fix problem with connecting to Postgres with TLS @FenrirUnbound in #4910
- Fix bug when commenting in local-only community by @dessalines in #4854 and @abdel-m in #4920
- Fix scheduled task to delete users with denied applications by @Nothing4You in #4907

API

- Return image dimensions and content type in API responses by @dessalines in #4704
- Adding a show_read override to GetPosts. by @dessalines in #4846
- Add show_nsfw override filter to GetPosts. by @dessalines in #4889
- Require authentication for site metadata fetch endpoint by @dessalines in #4968
- Add the ability to fetch a registration application by person_id by @dessalines in #4913
- Order community posts by published data, not id by @dullbananas in #4859
- Throw error when non-mod posts to mod-only comm or when URL is blocked by @flamingo-cant-draw in #4966
- Add option to search exclusively by post title by Carlos-Cabello #5015

Database

- Approve applications in transaction by @Nothing4You in #4970
- Use trigger to generate apub URL in insert instead of update, and fix query planner options not being set when TLS is disabled by @dullbananas in #4797

Lemmy-UI

- Fix full-size post images. by @dessalines in #2797
- Fix modlog ID filtering. by @dessalines in #2795
- Allow Arabic and Cyrillic characters when signing up or creating community by @SleeplessOne1917
- UX - Swap “Select Language” and “Cancel/Preview/Reply” button locations around in commentsReverse order of buttons in Reply TextArea
- Fix jump to content by @SleeplessOne1917
- Fixing peertube and ordinary video embeds. by @dessalines in #2676
- Changing sameSite cookie from Strict to Lax. by @dessalines in #2677
- Remove show new post notifs setting. by @dessalines in #2675
- Fix memory leak around emojis on server render by @makotech222 in #2674
- Enable spellcheck for markdown text area by @SleeplessOne1917 in #2669
- Pre release dep bump by @SleeplessOne1917 in #2661
- Add ability to fill magnet link title on post creation. by @dessalines in #2654
- Registration application view by @SleeplessOne1917 in #2651
- Add torrent help by @dessalines in #2650
- More moderation history by @dessalines in #2649
- Fix tribute related bug by @SleeplessOne1917 in #2647
- Remove min and max length from password input when using login form by @SleeplessOne1917 in #2643
- Remove trending communities card from home. by @dessalines in #2639
- Set data-bs-theme based on the presence of “dark” in theme name by @SleeplessOne1917 in #2638
- Fixing modlog filtering to allow admins and mods to filter by mod. by @dessalines in #2629
- Fix issue from logo bugfix by @SleeplessOne1917 in #2620
- Make more post params cross-postable by @SleeplessOne1917 in #2621
- Fix wonky comment action icon button alignment by @SleeplessOne1917 in #2622
- Prevent broken logo from crashing site by @SleeplessOne1917 in #2619
- Add rate limit info message. by @dessalines in #2563
- Fix getQueryString by @matc-pub in #2558

New Contributors

- @abdel-m made their first contribution in #4920
- @johnspurlock made their first contribution in #4917
- @FenrirUnbound made their first contribution in #4910
- @kwaa made their first contribution in #4901
- @Daniel15 made their first contribution in #4892

Full Changelog

- Lemmy Backend
- Lemmy-UI

Upgrade instructions

This upgrade could take as long as ~30 minutes for larger servers, due to needing to recalculate controversy ranks for all historical posts.

There are no breaking changes with this release.

Follow the upgrade instructions for ansible or docker.

If you need help with the upgrade, you can ask in our support forum or on the Matrix Chat.

Thanks to everyone

We’d like to thank our many contributors and users of Lemmy for coding, translating, testing, and helping find and fix bugs. We’re glad many people find it useful and enjoyable enough to contribute.

Special shout out to @SleeplessOne1917, @phiresky, @dullbananas, @mv-gh, @Nothing4u, @asonix, @sunaurus, @flamingo-cant-draw, and @Freakazoid182 for their many code contributions and helpful insights.

Support development

We (@dessalines and @nutomic) have been working full-time on Lemmy for over five years. This is largely thanks to support from NLnet foundation, as well as donations from individual users.

If you like using Lemmy, and want to make sure that we will always be available to work full time building it, consider donating to support its development. A recurring donation is the best way to ensure that open-source software like Lemmy can stay independent and alive, and helps us grow our little developer co-op to support more full-time developers.

- Liberapay (preferred option)
- Open Collective
- Patreon
- Cryptocurrency

Lemmy v0.18.0 Release

What is Lemmy?

Lemmy is a self-hosted social link aggregation and discussion platform. It is completely free and open, and not controlled by any company. This means that there is no advertising, tracking, or secret algorithms. Content is organized into communities, so it is easy to subscribe to topics that you are interested in, and ignore others. Voting is used to bring the most interesting items to the top.

Major Changes HTTP API instead of Websocket

Until now Lemmy-UI used websocket for all API requests. This has many disadvantages, like making the code harder to maintain, and causing live updates to the site which many users dislike. Most importantly, it requires keeping a connection open between server and client at all times, which causes increased load and makes scaling difficult. That’s why we decided to rip out websocket entirely, and switch to HTTP instead. This change was made much more urgent by the sudden influx of new users. @CannotSleep420 and @dessalines have been working hard for the past weeks to implement this change in lemmy-ui.

HTTP on its own is already more lightweight than websocket. Additionally it also allows for caching of server responses which can decrease load on the database. Here is an experimental nginx config which enables response caching. Note that Lemmy doesn’t send any cache-control headers yet, so there is a chance that private data gets cached and served to other users. Test carefully and use at your own risk.

Two-Factor Authentication

New support for two-factor authentication. Use an app like andOTP or Authenticator Pro to store a secret for your account. This secret needs to be entered every time you login. It ensures that an attacker can’t access your account with the password alone.

Custom Emojis

Instance admins can add different images as emojis which can be referenced by users when posting.

Other changes Progressive Web App

Lemmy’s web client can now be installed on browsers that support PWAs, both on desktop and mobile. It will use an instance’s icon and name for the app if they are set, making it look like a given instance is an app.

Note for desktop Firefox users: the desktop version of Firefox does not have built in support for PWAs. If you would like to use a Lemmy instance as a PWA, use use this extension.

Error Pages

Lemmy’s web client now has error pages that include resources to use if the problem persists. This should be much less jarring for users than displaying a white screen with the text “404 error message here”.

Route Changes

Pages that took arguments in the route now take query parameters instead. For example, a link to lemmy.ml’s home page with a few options used to look like this:

https://lemmy.ml/home/data_type/Post/listing_type/All/sort/Active/page/1

The new route would look like this:

https://lemmy.ml?listingType=All

Note that you now only have to specify parameters you want instead of all of them.

Searchable select redesign

The searchable selects, such as those used on the search page, have a new look and feel. No more inexplicable green selects when using the lightly themes!

Share button

Posts on the web client now have a share button on supported browsers. This can be used to share posts to other applications quickly and easily.

Lemmy-UI Overall look and feel

lemmy-ui is now upgraded to bootstrap 5, and every component is now much cleaner.

Special thanks to sleepless, alectrocute, jsit, and many others for their great work on improving and re-organizing lemmy-ui.

Database optimizations

Special thanks to johanndt, for suggesting improvements to Lemmy’s database queries. Some of these suggestions have already been implemented, and more are on the way.

Query speed is Lemmy’s main performance bottleneck, so we really appreciate any help database experts can provide.

Captchas

Captchas are not available in this version, as they need to be reimplemented in a different way. They will be back in 0.18.1, so wait with upgrading if you rely on them.

Upgrade instructions

Follow the upgrade instructions for ansible or docker.

If you need help with the upgrade, you can ask in our support forum or on the Matrix Chat.

Support development

We (@dessalines and @nutomic) have been working full-time on Lemmy for almost three years. This is largely thanks to support from NLnet foundation.

If you like using Lemmy, and want to make sure that we will always be available to work full time building it, consider donating to support its development. No one likes recurring donations, but they’ve proven to be the only way that open-source software like Lemmy can stay independent and alive.

Lemmy Release v0.14.0: Federation with Mastodon and Pleroma 🥳

Today is an exciting day for the Lemmy project.

Almost one year after first enabling federation, we now federate with other projects for the first time! According to some people's definition, this finally makes us part of the Fediverse.

It took a lot of work to make this possible, so big thanks to NLnet for funding our full time work on Lemmy, and to @lanodan and @asonix for helping to figure out how Pleroma and Mastodon federation works (it's difficult because they have almost no documentation).

Major Changes Federation code rewrite

The rewrite of the federation code started by @nutomic in August is now mostly complete. As a result, the code is much cleaner, and has tests to guarantee no breaking changes between Lemmy versions. As a side effect of this rewrite, it was now relatively easy to enable federation with other projects.

Mastodon and Pleroma users can:

View Lemmy communities, user profiles, posts and comments Follow Lemmy communities to receive new posts and comments Replies (mentions) work in both directions, including notifications

In addition, Pleroma users can exchange private messages with Lemmy users.

Note that Pleroma and Mastodon rely on a compatibility mode in Lemmy, which means that they won't receive events like Deletes or Votes. Other projects whose federation works similar to Pleroma/Mastodon will likely also federate.

Hardcoded slur filter removed

Lemmy finally has essential moderation tools (reporting, user/community blocking), so the hardcoded filter isn't necessary anymore. If you want to keep using the slur filter, copy these lines to your config file when upgrading, and adjust to your liking.

Upgrade notes

Federation with Pleroma/Mastodon works automatically, you don't need to change anything, assuming that your allowlist/blocklist configuration permits it.

Note that Mastodon and Pleroma are much, much bigger than Lemmy at this point, with a combined 3 milion users and 4500 instances, compared to 20.000 users and 35 instances for Lemmy (source). The existing mod tools in Lemmy might not be adequate to handle that at the moment.

Be aware that if you have federation enabled in the Lemmy config, Mastodon and Pleroma users can now fetch all posts and comments, to view them and share with their followers. The Lemmy blocklist/allowlist can not prevent this, it only prevents posts/comments from blocked instances to be shown on your own instance. The only solution to this problem is disabling federation, or waiting for signed fetch to be implemented.

If you want to use federation, but review new instances before federating with them, use the allowlist. You can switch from open federation to allowlist federation by pasting the output of the command below into federation.allowed_instances in the Lemmy config.

curl https://your-instance.com/api/v3/site | jq -c .federated_instances.linked

The lemmy.hjson additional_slurs field has changed its name to slur_filter.

Follow the Docker or Ansible upgrade instructions here.

Lemmy-Ansible

We've now separated our ansible install method (the preferred way to deploy Lemmy) into its own repo, lemmy-ansible. Let us know if you need help migrating existing installations over to it.

Changes API There is now a GetUnreadCount in the API to check the count of your unread messages, replies, and mentions. A full list of the API changes can be seen on this diff of lemmy-js-client: 0.13.0 -> 0.14.0-rc.1 . Lemmy Server More federation compat (#1894) Adding clippy:unwrap to husky. Fixes #1892 (#1893) Remove header guard for activitypub routes Add federation test cases for Smithereen and Mastodon Reduce stack memory usage in apub code Remove ActivityFields trait, deserialize into another struct instead Check if post or comment are deleted first. Fixes #1864 (#1867) Correctly use and document check_is_apub_id_valid() param use_strict_allowlist Convert note.content and chat_message.content to html (fixes #1871) Upgrade background_jobs to 0.9.1 #1820 (#1875) Fix husky fmt hook. (#1868) Renaming to slur_filter. Fixes #1773 (#1801) Three instance inbox bug (#1866) Remove ansible from this repo. (#1829) Rewrite collections to use new fetcher (#1861) Dont blank out post or community info. Fixes #1813 (#1841) Format config/defaults.hjson before committing (#1860) Breaking apub changes (#1859) Pleroma federation2 (#1855) Create a custom pre-commit hook, generates config/defaults.hjson (#1857) Add cargo metadata to all crates (#1853) Add both (De)Serialize to all models (#1851) Adding GetUnreadCount to the API. Fixes #1794 (#1842) Federate reports (#1830) Fix saved posts and hide read posts issue. Fixes #1839 (#1840) Dont allow posts to deleted / removed communities. Fixes #1827 (#1828) Dont swallow API errors (fixes #1834) (#1837) Fix federation of initial post/comment vote (fixes #1824) (#1835) Fix clippy warnings added in nightly (#1833) Admins can view all reports. Fixes #1810 (#1825) Adding a message_id to emails. Fixes #1807 (#1826) Generate config docs from code (#1786) Trying a background_jobs fix. #1820 (#1822) mark parent as read on reply (#1819) Move code to apub library (#1795) Adding honeypot to user and post creation. Fixes #1802 (#1803) Add database host back into config file (#1805) Lemmy UI Updating translations. Fixing unload (#487) Fix setup password. Fixes #478 (#484) Adding post comment scrolling hack. Fixes #480 #486 Navbar links (#476) Try fixing crypto node bug. Fixes #473 (#474) Use community title and user display name for dropdown. Mahanstreamer userpage (#471) Using i18next compatibility v3 (#465) Show original created time tooltip (#462) Revert version of i18next to fix plurals. Fixes #451 (#460) Fixing cross-posts showing on initial load. Fixes #457 (#464) Show bot account info. Fixes #458 (#463) Very weak password check (#461) Simplifying getunreadcount. (#455)

Lemmy Release v0.12.0 🥳 . User and Community blocking, lean federation, and a whole lot more.

Since our last release in April, we've had ~80 commits to Lemmy.

Lemmy Server Major Changes

Note: Issue links are below.

@nutomic did a major rewrite of the federation code. It is much simpler now, and reduced from 8000 lines of code to 6400. Functionality is mostly the same, but future changes will be much easier. You can now block users and communities, and their posts / comments won't show up in your feed. Removed IFramely: Lemmy can now fetch site metadata on its own. New API docs at: https://join-lemmy.org/api General Fix prod deploy script and clippy (#1724) Fix image uploads. Fixes #1725 (#1734) Adding more site setup vars. Fixes #678 (#1718) Dont append ? to url when cleaning it (#1716) User / community blocking. Fixes #426 (#1604) Swap out iframely (#1706) Adding ModTransferCommunity to modlog in API. Fixes #1437 Make sure bots aren't included in aggregate counts (#1705) Don't allow deleted users to do actions. Fixes #1656 (#1704) When banning a user, remove communities they've created (#1700) Distribute Lemmy via crates.io Simplify config using macros (#1686) Simplify lemmy_context() function (dont return errors) Blank out extra info for deleted or removed content. Fixes #1679 (#1680) Add show_new_posts_notifs setting. Fixes #1664 (#1665) Adding shortname fetching for users and communities. Fixes #1662 (#1663) Upgrading deps, running clippy fix on nightly 1.55.0 (#1638) Running clippy --fix (#1647) Make captcha case-insensitive Remove tracking params from post url (fixes #768) Fix IPv6 port setup for Nginx (#1636) Fix --cert-name for certbot. (#1631) Change join.lemmy.ml to join-lemmy.org (#1628) Upgrade pictrs. Fixes #1599 (#1600) Invalidate current logins on account deletion. Fixes #1602 (#1603) Upgrading api test deps (#1608) Fix nsfw posts showing for non-logged in users. Fixes #1614 (#1615) Add additional slurs configuration option. Closes #1464. (#1612) Updating to rust 1.51.0 (#1598) Remove brotli, zstd dependencies for faster compilation API A full list of the API changes can be seen on this diff of lemmy-js-client: 0.11.0 -> 0.12.0 . Federation Move resolving of activitypub objects to separate api endpoint (fixes #1584) Rewrite remaining activities (#1712) Migrate comment inReplyTo field to single value (ref #1454) Fix issue with protocol string in actor id generation (#1668) Lemmy UI Integrating resolve_user into search. (#377) Add lazy loading of images. Fixes #329 (#379) Adding vi, sk, mnc, and cy languages. (#378) Feature/user community block (#362) Swapping out iframely. (#374) Adding mod transfer community (#373) Remove content more (#372) Scroll to comments on post's x comments button (#312) Remove websocket connection messages. Fixes #355 Center spinner, make smaller. Fixes #203 Fix font issues. Fixes #354 Have setting to disable notifs for new posts. Fixes #132 (#345) Remove max length constraints on actors. Fixes #350 (#351) Fix captcha replay bug. Fixes #348 (#349) Removing community and user routes in favor of shortnames. Fixes #317 (#343) Don't use default subscribed for communities page. Adding Listing type to communities page, default local. #190 (#342) Fix language bug on mobile browsers Collapse sidebar on mobile. Fixes #335 (#340) Re-organized components folder. (#339) Fixing too many large spinners (#337) Moving comment link to top bar. Fixes #307 (#336) Fix/ws error messages (#334) Make spinner bigger. Fixes #203 Fix preview description html. Fixes #110 Always show previous paginator, extract paginator component. Use better comment collapse icon, and add text. Fixes #318 Fix symbols issue. Fixes #319 Don't restore scroll position on page refresh. Fixes #186 Insert triple backticks for 'code' button when multiple lines are selected. (#311) Change join.lemmy-ui.ml to join-lemmy-ui.org Adding a comment here placeholder. Fixes #301 Fix non-local community and person links. Fixes #290 Fix navbar bug. Fixes #289 Hide names of mods / admins without priveleges. Fixes #285 Adding URL search type. Fixes #286 Add a link to joinlemmy-ui on lemmy-ui.ml signup. Fixes #235 Fix duped site description. Fixes #281 Upgrade notes Servers

You may need to add this to your lemmy.hjson:

pictrs_url: "http://pictrs:8080"

If you'd like to make a DB backup before upgrading, follow this guide.

To upgrade your instance to v0.12.0, simply follow the instructions in the documentation:

Upgrade with manual Docker installation Upgrade with Ansible installation Clients / Apps A full list of the API changes can be seen on this diff of lemmy-js-client: 0.11.0 -> 0.12.0 .

Lemmy v0.10.0 Release (2021-04-05)

Changes

Since our last release in February, we've had ~150 commits to Lemmy. The biggest changes, as we'll outline below, are a split of Lemmy's user tables into federated and local tables, necessitating a v3 of Lemmy's API, federated moderation, i18n support in join.lemmy.ml, and lots of back-end cleanup.

Lemmy Server General Rewrote config implementation, finally allowing us to use newer Rust versions. Removed categories. Various refactors. API A full list of the API changes can be seen on this diff of lemmy-js-client: 0.9.9 -> 0.10.0 . Login invalidation on password change, thanks to @Mart-Bogdan Federation It is now possible to add users from other instances as community mods. Federating Matrix ID. Many changes for better compatibility with ActivityPub standard. Database Split the user_ into person and local_user tables. Strictly typed commonly used ID columns, to prevent DB errors using i32 as ids. Strictly typed URL fields, thanks to ajyoon. Created default DB forms, now used in all the unit tests. Lemmy UI Now using utf-8 emojis. Support for all the above changes to Lemmy. Typescript-safe i18n strings, thanks to @shilangyu. Added expandable post text (click on open book icon). Prettier cross-posting, which does smart quoting. Bugfixes for restoring scroll position on post page, custom site favicons, and autocomplete for login fields. Lemmy Docs Gazconroy built an Async API spec for Lemmy, that now serves as our main API docs. join.lemmy.ml Rewrote in inferno isomorphic, added i18n support via weblate. Added a section on the support page thanking contributors. Changed some page urls / titles Upgrade notes

Important: there are multiple breaking changes:

Configuration via environment variables is not supported anymore, you must have all your config in the lemmy.hjson file ( except for LEMMY_CONFIG_LOCATION ). The config format for allowed_instances and blocked_instances has changed, and you need to adjust your config file manually: before: allowed_instances: ds9.lemmy.ml,enterprise.lemmy.ml now: allowed_instances: ["ds9.lemmy.ml", "enterprise.lemmy.ml"] , and only one of the allowed_instances or blocked_instances blocks can be set. The API has been upgraded from v2 to v3, so all clients need to be updated: lemmy-js-client: 0.9.9 -> 0.10.0 .

If you'd like to make a DB backup before upgrading, follow this guide.

To upgrade your instance to v0.10.0, simply follow the instructions in the documentation:

Upgrade with manual Docker installation Upgrade with Ansible installation Compilation time

|| v0.9.0 (Rust 1.47) | v0.10.0 (Rust 1.47) | v0.10.0 (Rust 1.51) | |-| -------- | -------- | -------- | |Clean | 140s | 146s | 119s | | Incremental | 28s | 22s | 19s |

Despite ongoing efforts to speed up compilation, it has actually gotten slower when comparing with the same Rust version. Only thanks to improvements in newer Rust versions has our build process gotten faster. This could be simply because we added more code, while Lemmy v0.9.0 had 22.4k lines of Rust, v0.10.0 has 23.8k (an increase of 6%).

v0.9.0 build graph:

v0.10.0 build graph:

We extracted the crates lemmy_api_crud and lemmy_apub_receive from lemmy_api and lemmy_apub, respectively, and renamed lemmy_structs to lemmy_api_common. In the second graph you can see how parts of the api and apub crates are now built nicely in parallel, speeding up builds on multi-core systems.

On the other hand, some crates have gotten much slower to compile, in particular lemmy_db_queries (6.5s slower), lemmy_apub (6.5s slower if we include lemmy_apub_receive). And lemmy_db_views is quite slow, just as before.