Email or username:

Password:

Forgot your password?
All posts Eugen's posts Post Back to profile
Eugen Rochko

Anyone who is good at #Golang can you tell me what memory management crimes I am committing here?

https://github.com/tootsuite/gamo/blob/master/gamo.go

8 Jan 2020 at 0:32 | Open on mastodon.social
No comments
Not ur toy
@h

@Gargron I would start by debugging all the lilliput library bits. (which is C based and probably not a straight Go problem)
Probably separate and hardcode all the lilliput operations until you make sure those work, and then integrate them into your main server programme when those bits are ready.

8 Jan 2020 at 0:37 | Open on mastodon.social
Chris Wiegman

@Gargron can't do the greatest review from my phone but nothing major sticks out to me.

8 Jan 2020 at 0:38 | Open on mastodon.chriswiegman.com
Eugen Rochko

What Gamo is useful/used for: Proxying the server thumbnails on joinmastodon.org in a way that optimizes them and resizes them to the dimensions they're actually displayed at, reducing the total page size from 16 MB to 4 MB (there's still work to be done to lower that number)

8 Jan 2020 at 0:44 | Open on mastodon.social
Luke GB

@Gargron Does that... work? You're calling Close() on the ImageOps as soon as you alloc it in the sync.Pool: https://github.com/tootsuite/gamo/blob/master/gamo.go#L50

If you _must_ have Close() called on that (if the sync.Pool decides to bin it rather than adding it back to the pool), you might need to use runtime.SetFinalizer.

If you need to do more debugging, I'd suggest the handlers from net/http/pprof. Be careful with it - it registers them under /debug/pprof/ by default and you don't want those exposed on the internet.

@Gargron Does that... work? You're calling Close() on the ImageOps as soon as you alloc it in the sync.Pool: https://github.com/tootsuite/gamo/blob/master/gamo.go#L50

If you _must_ have Close() called on that (if the sync.Pool decides to bin it rather than adding it back to the pool), you might need to use runtime.SetFinalizer.

8 Jan 2020 at 0:53 | Open on woof.group
Schoentoon 🇳🇱🇨🇭

@Gargron Not seeing anything directly obviously wrong with it honestly. Ideally this line https://github.com/tootsuite/gamo/blob/master/gamo.go#L159 would receive an io.Reader rather than the complete image as a []byte but this seems like a limitiation of lilliput (and down there probably an opencv limitiation). Perhaps add some tests and run some profilers if you want to be really sure

8 Jan 2020 at 1:28 | Open on mastodon.schoentoon.com
Dominik Nakamura

@Gargron are you getting tons of requests at once?

From what I see your server doesn't have any limits on concurrent http handlers and you load load a lot of data, instantiate the decoder and so on. All that before you even get to the processing part.

I recommend you use a semaphore at the beginning of your handler or a http framework that can limit concurrent connections.

Let's say the pool can process 5 req/s but you get 15 http reqs/s. Then your memory fills up.

8 Jan 2020 at 5:24 | Open on mastodon.technology
Go Up