Question for MIDI builders: Arduino frame loss issue

[RAc]

Hi there to all of you who use an Arduino to drive your MIDI concertinas (everybody else please igonre):

 

I am facing the issue that has been raised here:

 

need delay to send messages reliably · Issue #44 · arduino-libraries/MIDIUSB · GitHub

 

(the last contrib in the thread is not by me but summarizes completly what I have tried so far).

 

in short, my Due based concertina works fine, but every once in a while a note off message is generated in the software but never makes it onto the wire, causing "sticky notes." I am absolutely sure that the midi send command is executed, so it must get lost somewhere in the Arduino libs. Very likely this also affects note on messages, but it is only the missing off notes that are audible.

 

Stepping through the code, I found that USB_send() has a hardcoded timeout of 250ms which will unconditionally drop the frame upon expiration. I may be running into that issue here. In this here thread:

 

Hardcoded timeout in USB_Send · Issue #478 · arduino/ArduinoCore-avr

 

an attempt has been made to make the timeout configurable, but the proposed change has never made it to the master branch.

 

Thus my questions:

 

- Have you run into the same issue during your development?

- If so, have you found a definite root cause, a workaround or even a solution?

- Has anyone experience in rebuilding Arduino libs (I would not mind experimenting with a custom usb driver)?

 

I would be grateful for any input, pm would be fine if needed.

 

Thanks so much! This is currently the only showstopper that prevents me from putting a lid on my midi crane project.

 

btw: Whoever now believes that this is witchcraft or rocket science - no. it isn't. The project pretty much looks like the out of the box sample presented here:

 

Create a MIDI Device | Arduino Documentation

 

only a wee little bit tailored to concertinas. Everybody can do that, truly...

Edited November 23 by RAc

[Don Taylor]

Just a thought...

 

If your own code is based upon that sample in: Create a MIDI Device and you are debouncing your switches using that delay(50) call then that might be the cause of your problem.

 

See: why is using delay bad?

 

So, what to do about it? You could write you own non-blocking timeouts in your code using the millis() function, or you could use a task scheduler library or a timer library to do that for you.  All of these techniques make your code grow a little 'hair' and become harder to read - which is why that example code uses delay() - to keep it simple.

 

If you are using 50ms debounce delays in your code then you could try reducing that delay time to see if it makes a difference.  I believe that you are using reed switches for your buttons, if so then these usually need much shorter debounce times than mechanical switches.

 

I don't think that adding delay()s into the midi library is a good solution.

 

 

 

[RAc]

Hi Don,

 

thanks for your thoughts, but I may not have made myself clear... debouncing is not an issue, this is way behind.

 

The problem is that I generate MIDI messages (whereever they come from, I might as well generate them programatically, forget about keyboards and switches for good for this discussion), but as soon as those messages arrive in a certain density (more than a single message in fast succession), they get lost somewhere in the call chain down to the wire. I (and the others who experience the problem) can guarantee 100% that the messages are generated on the application level, they just never reach the wire.

 

And yes, I am aware lf the article that some celebrate as if it was the ultimate computer wisdom. More like computing 101, but irrelevant to the problem at hand. Interestingly enough, adding a delay in this case relieves the problem significantly, and I believe I know why. It's more of a workaround than a solution, though, and I much prefer solutions over workarounds (one of the results of 30+ years in system software dev).

 

Thanks again for your input, I will get back to your bellows inquiry soon. And btw, I had to shelve the reed switches and now use plain old gaterons...

 

 

 

 

[Don Taylor]

3 hours ago, RAc said:

now use plain old gaterons...

I think that you are building a Crane midi concertina. 

 

Can you get the Gaterons close enough together to match a vintage Crane button layout?  I thought that they might be too large.

[RAc]

yes, you are absolutely right, the spacing is the big show stopper. On a Crane, the average vertical row spacing is eleven mms, but Gaterons will get you 18mms min which is a significant difference. Consequently, the end plates have ugly pan handles to make space for the hands, and I need a moveable hand rail to reach all buttons.

 

Fortunately, the end plates containing the button bearings are completly decoupled from everything else, so I cn redesign and replace them without changing anything else once I have a better solution. Pics and a video hopefully later today.

[SqueezyC]

11 hours ago, RAc said:

but as soon as those messages arrive in a certain density (more than a single message in fast succession), they get lost 

 

 

 

 

[SqueezyC]

 

Have you tried using a ring buffer ?  This could receive the live MIDI strings and then send them on to the output in a more controlled manner.

[RAc]

Thanks for the suggestion, SqueezyC - however I am not sure in how far a ring buffer on the app level would help as the timing problems are deeper down below. Do you mean not output the MIDI commands as they are detected but queue them and work off the queue asynchronously? I do not think that would be feasible with respect to latency.

 

What would help is collating several MIDI commands into a single message, but that is much harder and more error prone to code. I have already considered in case of a further code iteration.

 

Or were you getting at something else with respect to ring buffers?

[RAc]

So here is the making of video:

 

 

Thanks to everybody who contributed to the project with help and encouragement. The video is not very well made (I am not a good photographer nor camera operator, and I hate documentation). Looking forward to your questions and remarks.

[SqueezyC]

I used an Arduino Nano to interface a Hammond XK-3C with a Roland Integra 7 synth. There is an interrupt-driven routine which feeds any received MIDI data from the Hammond into a ring buffer. The main loop of the program reads byte by byte from the buffer and parses it to interpret the midi commands which are then processed and fed to the synth. This arrangement handles three midi channels (two manuals and pedals) including block chords and control signals from the swell pedal etc, and maps them out to multiple output channels. 
There is no perceptible latency.  
If your problems only occur during fast data bursts a buffer would probably help.

If you are combining several incoming midi sources you could would need to create a buffer for each input. 

[RAc]

are you talking about a MIDI controller or a device? I am designing a device, not a controller, so READING frames never occurs in my code, I only output frames.

 

[alex_holden]

16 hours ago, RAc said:

Thanks to everybody who contributed to the project with help and encouragement. The video is not very well made (I am not a good photographer nor camera operator, and I hate documentation). Looking forward to your questions and remarks.

 

Without gussets, what prevents the bellows opening too far?

[RAc]

6 minutes ago, alex_holden said:

 

Without gussets, what prevents the bellows opening too far?

Hi there Alex, please define "too far." From my understanding of bellows geometry, you can at maximum open them (theoretically) until the cardboard folds flatten to a perfect plane, then they will stop by themselves (of course, if you keep pulling hard after that, the cardboard will give in, but the prospective of rebuilding the bellows stops me from pulling that hard, and in my case, the length of the LH key board cable stretches to full extent before the bellows do). 

[alex_holden]

4 minutes ago, RAc said:

Hi there Alex, please define "too far." From my understanding of bellows geometry, you can at maximum open them (theoretically) until the cardboard folds flatten to a perfect plane, then they will stop by themselves (of course, if you keep pulling hard after that, the cardboard will give in, but the prospective of rebuilding the bellows stops me from pulling that hard, and in my case, the length of the LH key board cable stretches to full extent before the bellows do). 

 

OK. I find that when I have a skeletal bellows structure at the stage before fitting the gussets, it feels quite floppy and unstable, and there's nothing stopping you from opening it all the way so the hinges are stretched out at 180 degrees. I guess it doesn't matter as long as the valleys don't try to turn inside out.

[caj]

Hi,

 

250ms is a ridiculously long time --- a MIDI note-on or note-off message takes roughly 1ms to transmit, and your device is not doing much beyond reading sensors and sending MIDI messages.

 

If you're encountering a timeout, it's possible that you're sending way too many MIDI messages for it to keep up.  Are you sending volume messages from the bellows in your main loop?  How frequently?  

 

Another possibility is that the library has a problem specific to the Due.  It can happen that a library stops working with a faster model of microcontroller that exposes a bug, and I've encountered that phenomenon myself with an NFC reader library that I drove from a Teensy.  I had to debug the library and find places that had missing delays and put them in myself.

 

If that is the case, you might just try a different microcontroller.  I've been using a Teensy 4.0 without these issues.  However, I would first check if you are sending MIDI messages slowly enough, and if there isn't something else about your code that is the issue.   If you want to post your main source file, I'd be happy to take a peek at it and see if I notice anything.

[caj]

By the way, using corner molding for the handrests is really ingenious.  I bet that could be used to make the handrest position easily adjustable, by offsetting it with some washers or spacers.

[RAc]

21 minutes ago, caj said:

If you're encountering a timeout, it's possible that you're sending way too many MIDI messages for it to keep up.  Are you sending volume messages from the bellows in your main loop?  How frequently?

yes, actually, that appears to be a fairly hot track to follow up on as the problem is definitely related to a rapid succession of messages being sent out. Also, I added the code that adds the very short microdelay directly after transmitting each message, and I have not seen those packet losses since, so it may be related to back-to-back messages.

 

I have also realized that the problem aggrevates when I sample the key board less frequently. This also hints at the possibility that too narrow a spacing between outbound messages may be part of the problem, as then there are more messages to be pumped out in a single loop invocation. As you correctly observe, The Arduino does not have to do a whole lot; in the loop, I basically scan the sensor, then the keyboard, and as I have all of the information about velocity and keys to send, I pump out all resulting messages in a loop. It looks as if that is a contributing factor to the issue.

 

Thanks again for the input!  

[RAc]

Btw, here is a tidbit about the sensor I found out the hard way: The vl6180 has a resolution of 1mm, but not the accuracy. That means that if you statically and immovably put a constant distance (say, 25mm) between sensor and the next obstacle, subsequent readings (at least in single shot mode which I use) will yield oscillating results by a margin of around 3 mms. Others have observed this as well, even in total darkness. Those inaccuracies are apparently inevitable.

 

Now, 3mms are not a very bid deal when measuring bellows, but it makes the sampling algorithm much harder to realize as one has to distinguish between those oscillating samplings and true deltas in subsequent readings. In effect, one would have to constantly read the sensor, then create a median over time and observe the travel of the median - possibly the contiguous you can set the sensor to mode already does that - and from that make an educated guess about the velocity. There is no rocket science in any of that, but to choose a good compromise for the median period between accurate reading and introducing too much latency sounds like a lot of fine tuning.

 

That is why I wrote that there is a lot of room for imprvement on the software side. For the time being, I am not going to follow up on that as I need practice time more than nerding time, but if anyone feels like spending a lot of time in mapping very smooth and differentiated bellows movement to software, that sounds like a yielding project... 😉