Email or username:

Password:

Forgot your password?
jonny (good kind)'s posts Post Back to profile
jonny (good kind)

alright, after like a year of halfheartedly trying on and off, #FetchAllReplies is pretty much finished - the problem of not being able to see all replies to a post is one of the largest complaints that people have with mastodon in particular but also the fedi in general. It is an especially potent problem for smaller servers, making them feel lonely, and making the whole fedi seem quiet. It is also a large contributor to the 'reply guy' problem where a moderately popular post will get the same replies over and over again and people won't even know they're doing it.

This patch recursively fetches replies using activitypub collections. it does it respectfully, only when someone is explicitly looking at a post (rather than fetching all replies for everything all the time) with some debounce, and spaces out the recursive calls to the other servers in deep threads.

the only thing left is to make the posts get inserted into the web client as they are received, currently you need to refresh to see them.

trying it locally now and it is a game changer.

i'm not "good at ruby" so if you ever wanna see this upstream, kindly spare a code review?

github.com/NeuromatchAcademy/m

#FediDev #MastoDev #UnFuckTheFedi #PubSubIsCoolButPresentsPrettySeriousUsabilityProblems #JustSmallInstanceThings

19 September at 4:24 | Open on neuromatch.social
20 comments
:PUA: Shlee fucked around and
Emelia πŸ‘ΈπŸ»

@jonny you'd probably want to push them across to the streaming server for that to happen

19 September at 4:29 | Open on hachyderm.io
jonny (good kind)

@thisismissem one of the major parts of the program i have yet to explore. i'll take a look, but from cursory glance it doesn't seem too hard :) taking a break for some food now and then will continue on that.

19 September at 4:30 | Open on neuromatch.social
Emelia πŸ‘ΈπŸ»

@jonny that or just return a header when the replies collection is out of date & refresh (poll) regularly.

19 September at 4:49 | Open on hachyderm.io
aburka 🫣

@thisismissem @jonny streaming seems better because then other frontends would get them too?

19 September at 4:56 | Open on hachyderm.io
jonny (good kind)

@thisismissem is there a method you would prefer? I am not seeing a streaming endpoint for any generic DetailedStatus feed, maybe the Direct feed is closest? if polling would be preferable to adding a new stream/adapting another feed then i could figure that out, i'm just never sure what the most efficient way to do things with redux is

19 September at 4:59 | Open on neuromatch.social
kouhai, resolver of merges

@jonny @thisismissem some of this feels like β€œit depends”. polling will be chattier, especially without connection reuse via H2/etc. websockets will make state management after disconnect a minor issue complexity wise

note that depending on implementation, polling may also incur the expense of a round trip vs half trip

19 September at 5:30 | Open on social.treehouse.systems
jonny (good kind)

@kouhai
@thisismissem
I am thinking it might be nice to just make a streaming endpoint for when one is in a thread both for this and in general. On a given instance, the thread ancestors are always known (I think?) From the threadresolveservice, or at least wouldnt mutate in the client (?). A new status can similarly always resolve its thread root on the server. So if there was a streaming feed for every root post, that might be a reasonably efficient way to do it, so eg. Multiple viewers with different points in a broad and deep thread could all sub to the same feed, and every new post has exactly one feed it would need to be added to.

Since the context endpoint is unpaginated and you always get the full parent tree when you mount a thread view from any point in a thread, I think you could mount and unmount the stream w/relatively low complexity in the client? Gonna give it a shot

@kouhai
@thisismissem
I am thinking it might be nice to just make a streaming endpoint for when one is in a thread both for this and in general. On a given instance, the thread ancestors are always known (I think?) From the threadresolveservice, or at least wouldnt mutate in the client (?). A new status can similarly always resolve its thread root on the server. So if there was a streaming feed for every root post, that might be a reasonably efficient way to do it, so eg. Multiple viewers with different...

19 September at 5:38 | Open on neuromatch.social
kouhai, resolver of merges

@jonny @thisismissem threads can’t always be resolved to a global concrete root. For example, you might have been tagged into a private thread that is otherwise entirely on another instance. This is also not necessarily immutable. For example, you might run into the above case with followers only visibility.

So, in this case, it might not actually matter, so long as you actually dedupe on the fly. That might not actually be necessary at all, but doing so could be an optimization.

19 September at 5:46 | Open on social.treehouse.systems
jonny (good kind)

@kouhai
@thisismissem
Right right, I meant global to an instance, like an instance will always have the same notion of root across accounts and client/server. Maybe there's a privacy argument to be made if eg. The OP has blocked someone downthread, but I dont see a huge harm scenario in someone merely knowing about a handle being the OP if they inspect their network traffic.

I guess since the current methods need to traverse and give you all the ancestors anyway, the redis publish call could check for the existence of any subscriptions to parents?

@kouhai
@thisismissem
Right right, I meant global to an instance, like an instance will always have the same notion of root across accounts and client/server. Maybe there's a privacy argument to be made if eg. The OP has blocked someone downthread, but I dont see a huge harm scenario in someone merely knowing about a handle being the OP if they inspect their network traffic.

19 September at 5:54 | Open on neuromatch.social
kouhai, resolver of merges

@jonny @thisismissem i think im too tired at past-midnight to do system design lollll

19 September at 6:18 | Open on social.treehouse.systems
Emelia πŸ‘ΈπŸ»

@jonny well yeah, there's no functionality in streaming for updates to replies on a single status so both approaches are about as "good" β€” iirc your intention isn't to upstream this?

19 September at 9:19 | Open on hachyderm.io
Jennifer Moore

@jonny

This is very cool! I send general good vibes to you and the project :-)

19 September at 6:03 | Open on scicomm.xyz
Baessando :Ryyca:β˜­πŸ‡§πŸ‡·πŸ‡ΊπŸ‡³πŸ‡΅πŸ‡Έ

@jonny amazing job! Full congrats mate :doguinho:

19 September at 6:41 | Open on ursal.zone
Dawn Ahukanna

@jonny thanks for sharing. ActivityPub is a protocol like TCP, where you have to check packet hits and misses rather than REST API over HTTPS 1.1.

19 September at 7:08 | Open on mastodon.social
Duncan

@jonny sounds like something pretty essential that should be rolled into base Masto if it's good enough with resources.

I'm aware people have very different ideas of what Masto should be (and changes/projects often end up in a half-way house that please neither).

This should please everyone - it solves a long-standing moderation problem *and* makes Masto look more lively. So well done.

19 September at 8:11 | Open on mas.to
Duncan

@jonny Also, I know it's the last thing on your mind right now - but after the current roadmap any chance of using it for like and retoot numbers too?

19 September at 8:12 | Open on mas.to
πŸ‡΅πŸ‡Έ single use plastique πŸ΄β€β˜ οΈ

@jonny@neuromatch.social thank you, this will surely other platforms not just mastodon

19 September at 8:22 | Open on possum.city
Go Up