Email or username:

Password:

Forgot your password?
scy

A few days ago, my groovebox, the #SynthstromDeluge, running the new and awesome community-built firmware, crashed, which was very cool.

Not that it crashed, but what happened next: It displayed a colorful pattern on its pads.

That's a stack trace. It's listing the last steps the code made before crashing. The devs have asked to be sent photos of these, to help them find bugs.

I've pasted it on Discord, and there's a _bot_ that recognizes these images and decodes them into addresses!

Me, in a Discord channel called "nightly-testing", writing "This is from 1.0.1 and happened a few days ago. I think I was adding synths to a kit when it crashed, but I don't remember with absolute certainty."

Below that, a picture of my Deluge. It's a black metal device with several silicone buttons with colorful LEDs below them. It also has seven knobs (some black, some gold) and an OLED screen. The lower two thirds of the device are taken up by a matrix of 8 rows and 16 columns of square pads with RGB LEDs below them. Then there's an additional 2 columns of 8 rows of the same pads to the right of that, used for muting, auditioning, and playing.

From left to right, the main matrix displays four binary patterns, each 4 columns and 8 rows in size, in different colors: purple, blue, green, and cyan. The two columns on the right also display a pattern, in red.

In each of the patterns, each LED is either illuminated in the color of that block, or not at all. Each block of 4×8 LEDs thus represents a 32 bit address. The two columns on the right represent the first four hex digits of the firmware's Git commit hash.

The small OLED screen says, in capital letters, white on black "Error: E339. Press select knob to attempt resume, then save to new file."

Below my post, there's a reply of the bot "Deluge Crash Reader", saying:

Thanks for the image, @scy!, it decodes as:
0x201157c0
0x2011585c
0x20110840
0x20048f4c
0xfe31
fe31a4f9b112890b16315796907788eb81314ff
50 comments
Denian

@scy Wow. Ok, that might just be the coolest use for a chatbot that I've ever seen.

scy

Each of the colored blocks of 4×8 LEDs represents a 32-bit address in the firmware, while the 2×8 section at the right is displaying the first 4 hex digits of the Git commit hash that's running. The colors represent different types of stack frames, if I understood it correctly.

Devs then translate these addresses into the actual functions and lines in the code and can start to reason about it.

Discord screenshot. A dev named sapphire_arches pastes the actual function calls these addresses represent:

0x201157c0:
GeneralMemoryAllocator::getRegion(void*)
(.constprop.0) at DelugeFirmware/src/deluge/memory/general_memory_allocator.cpp:187
0x2011585c:
GeneralMemoryAllocator::dealloc(void*)
(.constprop.0) at DelugeFirmware/src/deluge/memory/general_memory_allocator.cpp:230
0x20110840: DelayBuffer::discard(bool)
(.constprop.0) at DelugeFirmware/src/deluge/dsp/delay/delay_buffer.cpp:101
0x20048f4c:
Delay::discardBuffers() at DelugeFirmware/scr/deluge/dsp/delay/delay.cpp:234

The dev then says:

so that's actually the same crash we were just starting at, just on 1.0.1 instead of nightly
idk if that's good or bad
synth loading to kit rows has always been a bit sketchy, I wonder of that's what caused @moulfrit's crash as well
scy

Of course I had to ask whether that bot is open source. And it is! It's a few Python scripts to interact with Discord, fetch all the images posted to the channel, and use OpenCV to try and detect a crash pattern in it. If that succeeds, the result is posted back to Discord.

github.com/0beron/delugeqr/

(Don't get confused by the "QR" in the name, these are not QR codes at all.)

And the code that generates these patterns is here in the firmware:

github.com/SynthstromAudible/D

Of course I had to ask whether that bot is open source. And it is! It's a few Python scripts to interact with Discord, fetch all the images posted to the channel, and use OpenCV to try and detect a crash pattern in it. If that succeeds, the result is posted back to Discord.

github.com/0beron/delugeqr/

scy

The #Deluge had already been an outstanding piece of gear before, but when #SynthstromAudible open-sourced the firmware, a fan-driven development effort started that, as a Deluge owner, I can only describe as "mindblowing".

Bugs were fixed, refactorings took place, and an impressive amout of new features and quality of life improvements were added, and are still coming.

Check out youtu.be/J8hibZYJhoY (23 min) or github.com/SynthstromAudible/D for an overview of the community firmware improvements.

The #Deluge had already been an outstanding piece of gear before, but when #SynthstromAudible open-sourced the firmware, a fan-driven development effort started that, as a Deluge owner, I can only describe as "mindblowing".

Bugs were fixed, refactorings took place, and an impressive amout of new features and quality of life improvements were added, and are still coming.

/dev/urandom

@scy one might say it led to a

deluge of contributions

dezwo

@scy How has your experience with the community firmware been so far? I love the open source approach but at the same time are (without any actual indications) somewhat anxious about potential stability issues.

scy

@dezwo I had my Deluge crash three times. Once with the stock firmware, twice with the community. I don't think that means anything.

If I was performing live, I'd be hesitant. But I'm not, and the community firmware comes with _so_ many improvements big and small that I'd be missing out if I didn't use it. I don't regret upgrading.

That being said, I'd love an "auto-save" feature, but saving manually every few minutes doesn't really hurt either.

Let me know if you have any more questions.

scy

@dezwo Also, the community firmware definitely is the future. It has made lots of refactorings, which means that merging single features back to the Synthstrom firmware is hard or even impossible.

I expect (and the community devs do, too, afaict) that the Synthstrom firmware won't see much development in the future. Instead, the community one is where features & bugfixes are going to be.

It's already so much better than stock, and the stock firmware was already pretty good.

dezwo

@scy Thanks so much. Very helpful 👍

Enron Hubbard

@scy @dezwo seriously, what has gone on with the community firmware is miraculous. Deluge is a case study in how to create great hardware and let the community decide what magical things to do with it.

Tekknovator

@dezwo @scy Dude, open source firmware has become a major factor for hardware purchase decisions. My linnstrument is open source, my norms shield, my axoloti synths. The m8 was promised open source when I pulled the trigger, but the maker seems to be too lazy to release openly. Some of the major innovations in synth hardware were all open source (mutable instruments, for example). I am saving up for a deluge.

Tekknovator

@dezwo @scy what I wanted to say, stability is more likely not an issue, because people who care have access to the code.

dezwo

@tekknovator @scy I don't doubt that folks care. But forking and maintaining a proprietary code base is no small feat, so I was interested in @scy's experience (especially since he started the thread with an experience report or a crash and subsequent debugging). Anyhow, thanks for sharing your perspectives.

Tekknovator

@dezwo @scy true. But from my experience this is a matter of patience and stuff gets fixed and stable. Whereas something like, ie. a Roland Fantom, just gets discontinued and never fixed or improved.

scy

RIP my notifications I guess :)

If you're interested in the Deluge, I highly recommend it. The official website is synthstrom.com/product/deluge/ and youtu.be/aBeeDwukpTs has a detailed introduction.

#Synthstrom is a small company with a great attitude. When they introduced the OLED screen to the Deluge instead of the 7-segment display it had before, they offered retrofit upgrades to all existing customers instead of trying to sell you new devices like pretty much the rest of the industry would.

King Calyo Delphi

@scy That's actually really cool and clever use of the technology for debugging! 😮

scy

@dragonarchitect They also have debug logs via MIDI Out, but this requires you to actually start capturing them before the crash, which most people won't. Thus they needed to be able to extract the most important parts in another way.

King Calyo Delphi

@scy Also, that photo paired with the bot's transcription of the debug log in lights gives me a clue about how that keypad is wired up as well as the possible endianness of the bytes, since that keypad is a 8x32+8x2 matrix.

scy

@dragonarchitect I wouldn't count on it, there's quite a bit of bit shifting going on in the source code of the crash handler. But I didn't look into it too much.

King Calyo Delphi

@scy Fair! I'm just thinking out loud with a hypothesis anyways haha

I have very little experience with reverse engineering. I see something neat like this and I start thinking of ways I might make it work.

rhempel

@scy That is an awesome and unexpected way to help developers. :-)

I will make a bet that they had product management that fully understood what the developers wanted and supported the feature ...

OR

Some developer spent personal time adding the feature and didn't tell the PM because they wouldn't understand its value anyways.

scy

@rhempel This is an open source firmware built by volunteers. No management involved.

rhempel

@scy so I was half right :-) Not that I'm trying to win anything.

I think this is a beautiful example of the power of a small group of motivated developers doing amazing work - forgive me for not knowing more about this project, and now I have a rabbit hole to go down ;-)

thunfisch

@scy One day I'll probably spend the money on a deluge... have been hearing awesome things, but still can't get myself to commit so much on an instrument that I probably will not play that much after all :s

scy

@thunfisch Oh, I totally understand. I spent months comparing grooveboxes and then finally pulled the trigger a few years ago when Synthstrom announced (in advance!) they'd be increasing the price soon due to all the new features the Deluge got.

I then didn't really use it for two years or something, because I didn't have (or rather, take) time to actually make music.

Now, with #WeeklyBeats2024 going on, it's the first time this thing actually has to do something, and it's doing it quite well.

DELETED

@scy sweet googly moogly that’s cool.

Jamie Clark

@scy If my debugging was that pretty, I'd probably be better at it. Reminds me of reading the lights on a PDP-10. Pass the DECtape!
#cybersec #FredFlintstoneCISO

Saxnot

@scy displaying error information like that is so cool!
Adopt to the medium instead of succumbing to "open me and attach to debug port"

abortretryfail

¨@scy
Yay another #synthstromdeluge owner!

The community firmware has been very cool.

Maybe some day we'll get time signatures other than 4/4ths. 😁

scy

@abortretryfail Can't you already use the variable length of clips to fake pretty much any time signature?

abortretryfail

@scy
Not really. The metronome is still wrong, and then you're on the hook for keeping track of which beat in the measure the grid shows since what the display says will be lies. You also have to work totally zoomed in to the beat level in arranger mode or it will misrepresent the clip lengths.

The OLED version made that worse since it doesn't display the numbers during playback anymore. (Mine was 7seg originally)

Honestly, v1.x did it better since all it did was count beats...

@scy
Not really. The metronome is still wrong, and then you're on the hook for keeping track of which beat in the measure the grid shows since what the display says will be lies. You also have to work totally zoomed in to the beat level in arranger mode or it will misrepresent the clip lengths.

The OLED version made that worse since it doesn't display the numbers during playback anymore. (Mine was 7seg originally)

abortretryfail

@scy

I just gave it a try for giggles and found some other things that get in the way.

The lead-in when recording notes is still 4 beats, copy/paste end up totally misaligned, and the clip playback position in keyboard mode is confusing at best.

scy

@abortretryfail Okay I think I understand now.

To be fair, support for anything but 4/4 isn't that widespread among grooveboxes as a whole, but I can see where you're coming from.

The next version of the community firmware has a 7seg emulation mode btw, so you can convert your OLED screen back to the numeric style if you really want to :)

Generic Sadboy 1916

@scy glorious fucking nerds. :blobhaj_tinyheart:

Jigme Datse

@scy I was looking at that and thinking... "That's a really interesting beat" but only just saw that you said it's a stack trace. Which really is amazing way to handle it. Crashing isn't fun, but having a crash that *might* be easier to handle debugging of is really awesome.

Go Up