Jump to content

midi sysex head scratching - arduino project


cedd

Recommended Posts

I'm hoping somebody considerably cleverer than me can help me wrap my head around the maths needed to calculate a couple of values to send via sysex to my Roland M300 mixer. 

I'm working on a remote control for my digital mixing desk (Roland M300). It consists of a rotary encoder and a small OLED display. I'm essentially building my own personal monitor mixer, controlling the sends to an aux. The display shows the user a channel number and a numeric level (plus a graphical representation of level using a bar graph). Rotate the encoder to select the desired channel, then click the encoder and then rotate it to change the send level. I've got a working user interface that selects the channel and increments or decrements an int to change the level (currently 0 to 127, but could be anything. Probably don't want to go much higher as it'll be a lot of rotations to turn something fully off) and am successfully sending a sysex with the correct model number, address, checksum etc.
My problem comes in calculating the 2 data bytes to be sent from my current int. The midi implementation can be found here; https://static.roland.com/assets/media/pdf/m300_midi_imple_e02.pdf - page 5, 7th field up from bottom right is the value I'm playing with - "channel 1 aux 1 send level". If my ASCII art is anything to go by then it looks something like this;

| 04 00 12 02   | 0aaaaaaa   | Channel 1 aux send pan |
| 04 00 12 03# | 0bbbbbbb |

It also comes with the note "less than -905, -905,,,100 = -Inf, -90.5,,,+10.0dB"
In other words it's using a range between <-905 and 100 to do a full travel of the control, from off to full.

It also gives the following notes on page 25;
"A 7-bit byte can express data in the range of 128 steps. For data where greater precision
is required, we must use two or more bytes. For example, two hexadecimal numbers aa
bbH expression two 7-bit bytes would indicate a value of aa x 128 + bb"

and

" In the case of values which have a +/- sign, 40H=-64, 00H=0, 3FH=+63, so that the
decimal expression would be 64 less than the value given in the above chart.
In the case of two types, 40 00H = -8192, 00 00H = 0, 3F 7FH = +8191"

I've really tried my best to understand this! I've done lots of searching and reading. I know 2's complement is involved, and I know I have to turn my integer in to a pair of 7 bit bytes, but I'm waving my white flag and admitting I'm really struggling to work this one out.

In a bid of desperation I monitored the midi coming out of the desk, while changing these controls. The levels set are very approximate - it's the position of a fader, without any numeric way of checking what value it's at. The first column is the rough fader position, the rest is the sysex as seen on the connected computer. I'll only show the full sysex once, as it's all the same apart from these few key bytes (the first 2 bytes being the value, the 3rd being the checksum, which I'm able to calculate and have working fine);

-inf   F0 41 0F 00 00 24 12 04 00 12 02 40 00 28 F7
-20                                     7E 60 0A 
-5                                      7F 5E 0B 
0                                       00 11 57 
+5                                      00 3C 2C
+10                                     00 64 04

Can someone cleverer than me help me come up with a way of converting my integer (0-127 or whatever) to give me a full travel of this control, from -inf up to +10, sending the 2 appropriate bytes? I'm well and truly stuck! I'm sending the sysex as an array, so would just be placing the 2 values in to the relevant spots in the array before sending.

Thank you so much in advance!

Link to comment
Share on other sites

You seem to be most of the way there.

Quote

In other words it's using a range between <-905 and 100 to do a full travel of the control, from off to full.

and you want to map this to range 0 to 127. This bit is just the maths for y = mx +c, a straight line scale-and-offset.
Calculate the output range 100 - -905 = 1005
Input range is 127 (your encoder).
Scale factor is 1005 / 127 = 7.98 ish.

Then you need an offset so that controller 1 gives output -905, after the scale factor has been applied. 1 x 7.98 is 7.98, so the offset is -905 - 7.98 = -913 ish.

So step one is to scale your 8-bit controller integer (0 to 127) into a signed integer of at 16 bits width, by applying the formula. You don;t need to work in decimals, doing it in integers is close enough. Take the encoder position, multiply by 8 and subtract 913. This gives a little less than -906 at zero, which is OK, because it says you can go below -906, they are just all treated as -Inf.

I have attached a spreadsheet which does the scaling factors calculations to this post if you want to see the numbers and play with it.

Now the "putting it into the midi message" bit. What you have in your integer, which I said make 16 bits wide, is a signed number.

All integer variables in C-like languages are signed, unless the keyword `unsigned` is used to make them unsigned. That's good, because we want to use signed numbers. By convention (if not absolute statute) all computers deal with the question of signed numbers in binary by using "2's complement" encoding. One way of thinking of this is that the most significant bit, instead of having a positive value like all the others, has it's negative instead. You don't actually need to get too deep into number formats for this however.

What we do need is a variable which will be exactly 16 bits wide - not more or less - and signed. In the Arduino Reference this seems to be `short` - in normal C you would write `int16_t`. OK, so make the variable holding the result of the above calculation of this type.

Now we need to convert that value into the two bytes (each of 8 bits, and by convention an unsigned value) to go in the midi message. The exercise here is telling the compiler to ignore what it thinks it knows about the meaning of bits, and just copy them around as 1s and zeros. What the Roland is going to do on the other end is reverse this back out to get the value, all we have to do is make sure the two ends do it the same. We do this in two parts

In the Arduino Reference it looks like highByte() and lowByte() functions do what you want, so call each of those, and (judging by the example from the manual), put the high byte first, and the low byte second in the midi message. These functions will completely ignore what the bits are supposed to mean, and just copy either the lowest 8 bits, or the 8 above that, to their output.

The C code for this is matter of masking out the 8 bits you want (using a bitwise AND), shifting (if necessary) and a final (static) cast to push the 16 bit value into an 8-bit value by throwing away what by then are a bunch of zeros. The implementation is this: https://android.googlesource.com/platform/external/arduino/+/d06daf9bbc46838400461eb8e15842974e38d82a/hardware/arduino/cores/arduino/wiring.h#93 (not the canonical source, but the one Google found). This is the sort of stuff which embedded developers write without thinking, and other programmers blink at confusedly.

FaderScaler.ods

  • Upvote 1
Link to comment
Share on other sites

Highbyte and lowbyte won't work for this as they are making 8 bit numbers and yours are 7 bit

 

So if v is your 16 bit signed value as described above

a=(v>>7) & 0x7F; //get the upper 7 bits

b=v & 0x7F; //get the lower 7 bits

 

(edit: that is the character "x" not a multiply, i.e. a hex value. Seems the forum software is being clever and changing my x to a multiplication symbol)

  • Upvote 1
Link to comment
Share on other sites

You, gentlemen, are wonderful! 

Tested tonight and it's working a treat. I just need to map it logarithmically as the bottom end of the fader is painfully slow, but that is pretty straightforward. Have also worked out how to decide a received sysex to update my remote unit values with what's actually in the desk (which I'll do on Arduino startup, and then just track changes). I can even get the channel names via midi, which will be a nice added bonus. 

 

Currently using a Teensy, which is a little overkill for all of this. Planning to use an Arduino nano or similar in the final units for a bit if cost saving. 

Will keep you updated with progress! 

Link to comment
Share on other sites

 Final question (I hope!);

I thought I'd nailed the code for receiving the sysex, but the reverse of the above procedure isn't working. Again I think I'm 90% there, but the integer it's spitting out just isn't correct. 

Would you mind casting an eye over my code and telling me what stupid mistake I've made? 

I'll add that this code has been messed about with in order to try and get this working. int's have been changed to int16_t's, I've tried adding masks to both hex values (although I don't think I need it on the lower byte). There are lots of unnecessary Serial Prints in here for debugging purposes. 

The incoming sysex goes in to an array named "received", and I then use a nest of If statements to decide what to do with it (there will eventually be incoming name data as well, so I need to correctly discern what's coming in). I'm sending a hex string in to the teensy (one that I've sent previously, and have checked it's fine). The correct bytes print out when I print the high and low bytes. It's just the maths to switch it back to a signed integer. I'm then mapping it back to 0-127, though the problem is before this step. The +1 on the chan value is because the sysex uses channels starting at 0, whereas I and the user am using chanels starting at 1. I stick the final calculated value in an array called channelvals. 

 

Any help hugely appreciated!

 

void receivemidi(const byte *a, uint16_t sizeofsysex, bool last){
  const byte *received = MIDI.getSysExArray();
  if(received[6] == 0x12){                      //it's a Data set
  Serial.println("data set");
    if(received[7] == 0x04 && received[9] == 0x12){   //it's aux data
      Serial.println("aux data");
        if (received[10] == auxnumber){               //it's for my aux
        int chan = received[8]+1;
        Serial.println(chan);
        byte rxhigh_part = received[11];
        byte rxlow_part = received[12];
        Serial.print(rxhigh_part,HEX);
        Serial.print(" : ");
        Serial.println(rxlow_part,HEX);
        int val = (rxhigh_part*127)&0x7f + rxlow_part&0x7f;
        Serial.println(val);
        val = map(val,-905,100,0,127); 
        channelvals[received[8]]=val;
        }
     }
  }


}

 

Link to comment
Share on other sites

Just tried that and whilst it did change the values, it didn't make them right. Now using Alister's method above, I send it 0x78 and 0x77 (-905 - fader should be to the bottom) and it returns 15487. That's with the Serial Print before the mapping (so it's the raw 16 bit number). If I send it 0x0 and 0x64 (100 - fader should be at the top) it returns 100, so the top end is working fine, but it looks like once it goes negative, it all goes wrong. 

Link to comment
Share on other sites

Yes, the quick'n'dirty code I gave you assumes it's an unsigned 16 bit number composed of two 7 bit parts. I think you just need to sign extend the unsigned value you get ... 

So if you take the unsigned value you've got, call it u, then do 

    if (u & 0x2000) u |= 0xC000;
 

this sign extends your 14 bit number (because it was composed of two 7 bit numbers. This correctly returns 100 and -905 for your inputs. This assumes u is a 16 bit integer. If it's a 32 bit integer then you need to or with 0x8888C000.

 

 

 

 

Edited by alistermorton
Adding new text
  • Upvote 1
Link to comment
Share on other sites

That looked so promising, but alas it's still just returning 15487 when I try and send 0x78 and 0x77, which should be -905. 

The code now looks like this with the additions from above. I've done some Serial Prints in binary as well, just to see if seeing the bits themselves creates any clues. I'll just post the returned results next to the serial prints for ease. 

Edit to add - I tried the 32 bit version too

void receivemidi(const byte *a, uint16_t sizeofsysex, bool last){
  const byte *received = MIDI.getSysExArray();
  if(received[6] == 0x12){                      //it's a Data set
  Serial.println("data set");
    if(received[7] == 0x04 && received[9] == 0x12){   //it's aux data
      Serial.println("aux data");
        if (received[10] == auxnumber){               //it's for my aux
        int chan = received[8]+1;
        Serial.println(chan);
        byte rxhigh_part = received[11];
        byte rxlow_part = received[12];
        Serial.print(rxhigh_part,HEX);                                             0x78
        Serial.print(" : ");
        Serial.println(rxlow_part,HEX);                                           0x7F
                Serial.print(rxhigh_part,BIN);                                     1111000
        Serial.print(" : ");
        Serial.println(rxlow_part,BIN);                                            1111111
        int val = ((rxhigh_part&0x7F)<<7)|(rxlow_part&0x7F);
        if(val&0*2000)val|=0xC000;
        Serial.println(val,DEC);                                                        15487
       Serial.println(val,HEX);                                                         0x3C7F
        Serial.println(val,BIN);                                                         11110001111111
        val = map(val,-905,100,0,127); 
        channelvals[received[8]]=val;
        
        }
     }
  }

 

Link to comment
Share on other sites

That looked so promising, but alas it's still just returning 15487 when I try and send 0x78 and 0x77, which should be -905. 

The code now looks like this with the additions from above. I've done some Serial Prints in binary as well, just to see if seeing the bits themselves creates any clues. I'll just post the returned results next to the serial prints for ease. 

void receivemidi(const byte *a, uint16_t sizeofsysex, bool last){
  const byte *received = MIDI.getSysExArray();
  if(received[6] == 0x12){                      //it's a Data set
  Serial.println("data set");
    if(received[7] == 0x04 && received[9] == 0x12){   //it's aux data
      Serial.println("aux data");
        if (received[10] == auxnumber){               //it's for my aux
        int chan = received[8]+1;
        Serial.println(chan);
        byte rxhigh_part = received[11];
        byte rxlow_part = received[12];
        Serial.print(rxhigh_part,HEX);                                             0x78
        Serial.print(" : ");
        Serial.println(rxlow_part,HEX);                                           0x7F
                Serial.print(rxhigh_part,BIN);                                     1111000
        Serial.print(" : ");
        Serial.println(rxlow_part,BIN);                                            1111111
        int val = ((rxhigh_part&0x7F)<<7)|(rxlow_part&0x7F);
        if(val&0*2000)val|=0xC000;
        Serial.println(val,DEC);                                                        15487
       Serial.println(val,HEX);                                                         0x3C7F
        Serial.println(val,BIN);                                                         11110001111111
        val = map(val,-905,100,0,127); 
        channelvals[received[8]]=val;
        
        }
     }
  }

 

Link to comment
Share on other sites

The forum has turned 0 x 2000 into 0 * 2000 - your code has the same mistake. That will stop it doing the sign extension. 

  if(val&0*2000)val|=0xC000;

replace the * with an x

Also be aware of the length of the int - if it's a 32 bit int, you need the longer value to or with (0 x 8888C000)

Also - strictly speaking, the test ought to be 

if ((val & 0X2000) != 0) val |= 0XC000;

Edited by alistermorton
Link to comment
Share on other sites

Understood and changed. It's now returning 64639 (or 1111110001111111) as 16 bit or -2004288385 in 32 bit. 

Why did Roland decide you'd need this much resolution on an aux send, when the channel faders are done with a simple 0-127!?

Link to comment
Share on other sites

FYI:  If you put your code stuff in code tags by clicking this button in the editor and then typing/pasting your code:

1850676224_ScreenShot2021-12-02at12_38_33PM.png.faa705a92f8033ded701bc2c44bdd847.png

then the forum won't attempt any fancy auto-formatting wizardry.

 

Code

If 3x7 keeps changing to 3*7 then this will help.

 

  • Upvote 1
Link to comment
Share on other sites

20 hours ago, cedd said:

Currently using a Teensy, which is a little overkill for all of this. Planning to use an Arduino nano or similar in the final units for a bit if cost saving. 

Have you used ESP32? Boards are about £3 off AliExpress and can be programmed in Arduino IDE, often using the same libraries.

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
×
  • Create New...

Important Information

We have placed cookies on your device to help make this website better. You can adjust your cookie settings, otherwise we'll assume you're okay to continue.