Email or username:

Password:

Forgot your password?
marius's posts Post Back to profile
marius

Since I'm such a fan of handmade programming, I find myself this fine eve implementing document indexing from scratch(ish) for my #GoActivityPub library storage backends.

31 October at 16:29 | Open on metalhead.club
5 comments
marius

I'm sure I made plenty of mistakes, but I have to admit I find it surprisingly satisfying to be able to operate on a data type that I can overlay on top of the existing #FedBOX storage engines and get native and *fast* querying for them.

The indexes are quite chunky despite being built on top of roaring bitmaps because there's so many "indexable" elements in an #ActivityPub object. (Currently I'm indexing the type, the content, summary, name, preferredUsername, the recipients, the actor and the object)

As I explore some more, I hope I streamline some of these issues, and make the whole thing more robust.

I'm sure I made plenty of mistakes, but I have to admit I find it surprisingly satisfying to be able to operate on a data type that I can overlay on top of the existing #FedBOX storage engines and get native and *fast* querying for them.

The indexes are quite chunky despite being built on top of roaring bitmaps because there's so many "indexable" elements in an #ActivityPub object. (Currently I'm indexing the type, the content, summary, name, preferredUsername, the recipients, the actor and the object)

2 November at 10:45 | Open on metalhead.club
marius

By *native* I mean that I can have my own little API for searching:
2 November at 10:56 | Open on metalhead.club
marius

Frantic day today, around 10h of productive work on improving the Index and moving it as part of the go-ap/filters module.

+1510/-11 lines of which 987 belong to tests.

Coverage is not entirely sufficient yet, because it's missing the checks for the top level Index.Add() and Index.Search() methods.

Another thing left to do is the persistence to disk.

The **reason** why I wanted to move the work I've done yesterday to this module is that instead of the custom client.SearchByX() functions, I wanted to retrofit the existing functionality already present in the filters module. Ah, also moving the bitmaps themselves to a semblance of generic types....

Frantic day today, around 10h of productive work on improving the Index and moving it as part of the go-ap/filters module.

+1510/-11 lines of which 987 belong to tests.

Coverage is not entirely sufficient yet, because it's missing the checks for the top level Index.Add() and Index.Search() methods.

Another thing left to do is the persistence to disk.
3 November at 18:44 | Open on metalhead.club
Go Up