Jump to content

Automatic fingering generation for Anglo


Recommended Posts

I started working on a tool that can automatically generate fingerings on 30b Anglo. While it's still extremely rough, I just got it to the point where it can generate the fingering for a complete melody.  Example using Oh Susanna below.

 

The approach uses an optimization algorithm called the Partioned Boolean Quadratic Problem (PBQP). I happen to have shared an office at one point with someone who wrote his PhD thesis on PBQP solvers, and based on what I had learned from him I was able to spot it potential application to generating Anglo fingerings.

 

The PBQP solver is able to solve for "costs" (which might be infinite, in the case of things that are impossible) on individual notes and between pairs of notes. I use these costs to build in a number of ergonomic constraints above and beyond the physical constraints, such as the requirement that simultaneous notes must be on the same bellows direction. Feedback appreciated on these ergonomic heuristics. Do they make sense, are there ones I'm missing?

 

Single Note Constraints: Pinky buttons are more costly than other buttons.

Pairwise Note Constraints for Sequential Notes: Cost to changing hands, cost to changing bellows direction (except when pressing the same button), cost to moving directly between the upper and lower rows, cost jumping within a column, cost to consecutive pinky buttons.

Pairwise Note Constraints for Simultaneous Notes: cost to multiple buttons in the same column.

 

Here are some samples:

 

Oh Susanna!

Sequential notes:
  Reed assigned: R01Push
  Reed assigned: R02Pull
  Reed assigned: R02Push
  Reed assigned: R03Push
  Reed assigned: R03Push
  Reed assigned: R02aPush
  Reed assigned: R03Push
  Reed assigned: R02Push
  Reed assigned: R01Push
  Reed assigned: L10Push
  Reed assigned: L10Pull
  Reed assigned: L10Pull
  Reed assigned: L10Push
  Reed assigned: R01Push
  Reed assigned: L10Push
  Reed assigned: R01Push
  Reed assigned: R02Pull
  Reed assigned: R02Push
  Reed assigned: R03Push
  Reed assigned: R03Push
  Reed assigned: R02aPush
  Reed assigned: R03Push
  Reed assigned: R02Push
  Reed assigned: R01Push
  Reed assigned: L10Push
  Reed assigned: L10Pull
  Reed assigned: L10Pull
  Reed assigned: L10Push
  Reed assigned: L10Push
  Reed assigned: R01Push
  Reed assigned: R02Pull
  Reed assigned: R02Push
  Reed assigned: R03Pull
  Reed assigned: R03Pull
  Reed assigned: R02aPush
  Reed assigned: R02aPush
  Reed assigned: R02aPush
  Reed assigned: R03Push
  Reed assigned: R03Push
  Reed assigned: R02Push
  Reed assigned: R01Push
  Reed assigned: L10Push
  Reed assigned: R01Push
  Reed assigned: R02Pull
  Reed assigned: R02Push
  Reed assigned: R03Push
  Reed assigned: R03Push
  Reed assigned: R02aPush
  Reed assigned: R03Push
  Reed assigned: R02Push
  Reed assigned: R01Push
  Reed assigned: L10Push
  Reed assigned: L10Pull
  Reed assigned: L10Pull
  Reed assigned: L10Push
  Reed assigned: L10Push
  Reed assigned: R01Push
Gmajor7 Chord

Simultaneous notes:
  Reed assigned: L01Pull
  Reed assigned: L02Pull
  Reed assigned: L03Pull
  Reed assigned: L04Pull
Bar 7 of Rights of Man from Easy Anglo 1-2-3

Mixed notes:
  Reed assigned: R02Push
  Reed assigned: L09Push
  Reed assigned: L04Push

 

Code available here, if you can read C++: https://github.com/resistor/concertina-pbqp

  • Like 2
Link to comment
Share on other sites

On 9/10/2022 at 5:26 AM, Owen Anderson said:

[1] I started working on a tool that can automatically generate fingerings on 30b Anglo. While it's still extremely rough, I just got it to the point where it can generate the fingering for a complete melody.  Example using Oh Susanna below.

 

The approach uses an optimization algorithm called the Partioned Boolean Quadratic Problem (PBQP). I happen to have shared an office at one point with someone who wrote his PhD thesis on PBQP solvers, and based on what I had learned from him I was able to spot it potential application to generating Anglo fingerings.

 

The PBQP solver is able to solve for "costs" (which might be infinite, in the case of things that are impossible) on individual notes and between pairs of notes. I use these costs to build in a number of ergonomic constraints above and beyond the physical constraints, such as the requirement that simultaneous notes must be on the same bellows direction. Feedback appreciated on these ergonomic heuristics. Do they make sense, are there ones I'm missing?

 

Single Note Constraints: Pinky buttons are more costly than other buttons.

Pairwise Note Constraints for Sequential Notes: Cost to changing hands, cost to changing bellows direction (except when pressing the same button), cost to moving directly between the upper and lower rows, cost jumping within a column, cost to consecutive pinky buttons.

Pairwise Note Constraints for Simultaneous Notes: cost to multiple buttons in the same column.

 

Here are some samples:

 

[2] Code available here, if you can read C++: https://github.com/resistor/concertina-pbqp

[1] I've been working since late 2017 on program(s) to do the same, but using fixed note/button mappings. I started serious work in late 2019/early 2020, and arrived at a 'final' version in about March 2021. The program(s) operate by adding tablature (fingering) to an existing ABC script.

 

[2] I use the Icon/Unicon programming language (as far as I know, only one other reader of this forum has heard of it)

 

As an illustration, I've included a 'tabbed' score for 'Swaggering Boney' (as it's recently been discussed in another context). ABC is also included - you will note that I've included the use of barlines to act as 'spacers' in the w: lines as discussed in our exchanges in July😊 Such a simple idea - which never occurred to me till I saw your hand-crafted code!

 

The tabs in the ABC  are for a G/D concertina uusing a fixed 'along-the-row' note/button mapping based on the Australian Bush Traditions button numbering. (I can do different concertinas, different numbering systems, and different mappings...).

 

%%gdalongtherow

I:abc-charset utf-8
%PAGE LAYOUT
%%pageheight 29.7cm
%%pagewidth 21.0cm
%%leftmargin 1.0cm
%%rightmargin 1.0cm
%%topmargin 1.0cm
%%bottommargin 1.0cm

%%annotationfont * 10
%%vocalfont * 12
%%gchordfont * 12
%%footerfont * 8
%%measurenb 0
%%writefields Q 0

%%partsbox
%%scale 0.7
%%pagescale 0.850

%%MIDI ratio 3 1 %ensures that A>B is played the same as A3/2B/2, What happens if R:is set to Hornpipe?

X:12655
T:Swaggering Boney (Longborough)
%A lightly edited tune from the Rude Mechanicals tune library: www.rudemex.co.uk
M:6/8
L:1/8
Q:3/8=90
P:(A2B3)4(A2C3)4AA
K:Gmaj %1-sharp (nominal key of G)
P:A
|: d | "G" B>AG "C" Bcd | "C" e>fe "G" d2 d | "G" g>fg "C" e>ag | "C" f>e "G" f g2 :|
w: R3 | R2 R2^ R1 R2 R3^ R3 | R4^ R5^ R4^ R3 R3 | R4 R5^ R4 R4^ dR3 R4 | R5^ R4^ R5^ R4 | ABTGDA
P:B
|: g | "D" afd "D" afd | "D" afd "G" d2 B | "C" c>de "C" e>fe | "C" c>AB "C" c3 |
w: R4 | dR3 R5^ R3 dR3 R5^ R3 | dR3 R5^ R3 R3 R2 | R3^ R3 R4^ R4^ R5^ R4^ | R3^ R2^ R2 R3^ | ABTGDA
"C" B>cd "G" G3 | "C" c>de "Am" A3 | "C" B>cd "C" e>ag | "C" f>e "G" f g2 :|
w: R2 R3^ R3 R1 | R3^ R3 R4^ R2^ | R2 R3^ R3 R4^ dR3 R4 | R5^ R4^ R5^ R4 | ABTGDA
P:C
|: g | "D" afd "D" afd | "D" afd "G" d2 B | "C" c>de "C" e>fe | "C" c>AB "C" c3 |
w: R4 | dR3 R5^ R3 dR3 R5^ R3 | dR3 R5^ R3 R3 R2 | R3^ R3 R4^ R4^ R5^ R4^ | R3^ R2^ R2 R3^ | ABTGDA
M:4/4
"^Slows" "C" B2 c2 "G" d4 | "G" G4 "G" G4 | "C" c2 d2 "C" e4 | "Am" A4 "Am" A4 | \
w: R2 R3^ R3 | R1 R1 | R3^ R3 R4^ | R2^ R2^ | ABTGDA
M:6/8
"^ROT" "C" B>cd "C" e>ag | "C" f>e "G" f g2 :|
w: R2 R3^ R3 R4^ dR3 R4 | R5^ R4^ R5^ R4 | ABTGDA

 

Damn! The score has gone in as an attachment - I never can get that right!!!

 

Swaggering Boney (Longborough).pdf

Edited by lachenal74693
  • Like 1
Link to comment
Share on other sites

4 minutes ago, lachenal74693 said:

[1] I've been working since late 2017 on program(s) to do the same, but using fixednote/button mappings. I started

serious work in late 2019/early 2020, and arrived at a 'final' version in about March 2021. The program(s) operate

by adding tablature (fingering) to an existing ABC script.

 

[2] I use the Icon/Unicon programming language (as far as I know, only one other reader of this forum has heard of it)

 

As an illustration, I've included a 'tabbed' score for 'Swaggering Boney' (as it's recently been discussed in another context). ABC is also included - you will note that I've included the use of barlines to act as 'spacers' in the w: lines

as discussed in our exchanges in July😊

 

%%gdalongtherow

I:abc-charset utf-8
%PAGE LAYOUT
%%pageheight 29.7cm
%%pagewidth 21.0cm
%%leftmargin 1.0cm
%%rightmargin 1.0cm
%%topmargin 1.0cm
%%bottommargin 1.0cm

%%annotationfont * 10
%%vocalfont * 12
%%gchordfont * 12
%%footerfont * 8
%%measurenb 0
%%writefields Q 0

%%partsbox
%%scale 0.7
%%pagescale 0.850

%%MIDI ratio 3 1 %ensures that A>B is played the same as A3/2B/2, What happens if R:is set to Hornpipe?

X:12655
T:Swaggering Boney (Longborough)
%A lightly edited tune from the Rude Mechanicals tune library: www.rudemex.co.uk
M:6/8
L:1/8
Q:3/8=90
P:(A2B3)4(A2C3)4AA
K:Gmaj %1-sharp (nominal key of G)
P:A
|: d | "G" B>AG "C" Bcd | "C" e>fe "G" d2 d | "G" g>fg "C" e>ag | "C" f>e "G" f g2 :|
w: R3 | R2 R2^ R1 R2 R3^ R3 | R4^ R5^ R4^ R3 R3 | R4 R5^ R4 R4^ dR3 R4 | R5^ R4^ R5^ R4 | ABTGDA
P:B
|: g | "D" afd "D" afd | "D" afd "G" d2 B | "C" c>de "C" e>fe | "C" c>AB "C" c3 |
w: R4 | dR3 R5^ R3 dR3 R5^ R3 | dR3 R5^ R3 R3 R2 | R3^ R3 R4^ R4^ R5^ R4^ | R3^ R2^ R2 R3^ | ABTGDA
"C" B>cd "G" G3 | "C" c>de "Am" A3 | "C" B>cd "C" e>ag | "C" f>e "G" f g2 :|
w: R2 R3^ R3 R1 | R3^ R3 R4^ R2^ | R2 R3^ R3 R4^ dR3 R4 | R5^ R4^ R5^ R4 | ABTGDA
P:C
|: g | "D" afd "D" afd | "D" afd "G" d2 B | "C" c>de "C" e>fe | "C" c>AB "C" c3 |
w: R4 | dR3 R5^ R3 dR3 R5^ R3 | dR3 R5^ R3 R3 R2 | R3^ R3 R4^ R4^ R5^ R4^ | R3^ R2^ R2 R3^ | ABTGDA
M:4/4
"^Slows" "C" B2 c2 "G" d4 | "G" G4 "G" G4 | "C" c2 d2 "C" e4 | "Am" A4 "Am" A4 | \
w: R2 R3^ R3 | R1 R1 | R3^ R3 R4^ | R2^ R2^ | ABTGDA
M:6/8
"^ROT" "C" B>cd "C" e>ag | "C" f>e "G" f g2 :|
w: R2 R3^ R3 R4^ dR3 R4 | R5^ R4^ R5^ R4 | ABTGDA

 

 

Swaggering Boney (Longborough).pdf 16.39 kB · 0 downloads

 

Here's the first line of your tune with fingering generated by my optimizer:

 

Sequential notes:
  Reed assigned: R02Pull
  Reed assigned: R01Pull
  Reed assigned: L08Pull
  Reed assigned: L08Push
  Reed assigned: L09Push
  Reed assigned: L09Pull
  Reed assigned: R02Pull
  Reed assigned: R02Push
  Reed assigned: R03Pull
  Reed assigned: L10Pull
  Reed assigned: L10Push
  Reed assigned: L10Push
  Reed assigned: R03Push
  Reed assigned: R03Pull
  Reed assigned: R03Push
  Reed assigned: R02Push
  Reed assigned: R07Pull
  Reed assigned: R03Push
  Reed assigned: R03Pull
  Reed assigned: R02Push
  Reed assigned: R03Pull
  Reed assigned: R03Push

 

I would love to be able to read-in and read-out ABC notation in the future, but one step at a time. Since I am able to model sequential AND simultaneous notes, I hope to be able to generate fingering for melodies WITH harmonic accompaniments in the near future. The PBQP model is also quite compatible with supporting "fixed" notes: I should be able to manually specify the fingering for some notes, and have the PBQP solver fill in the remainder.

Link to comment
Share on other sites

8 minutes ago, Owen Anderson said:

[1] Here's the first line of your tune with fingering generated by my optimizer:

 

[2] I would love to be able to read-in and read-out ABC notation in the future, but one step at a time. Since I am able to model sequential AND simultaneous notes, I hope to be able to generate fingering for melodies WITH harmonic accompaniments in the near future. The PBQP model is also quite compatible with supporting "fixed" notes: I should be able to manually specify the fingering for some notes, and have the PBQP solver fill in the remainder.

Thanks for that speedy response!

 

[1] Visual inspection makes me think that your fingering is for a C/G concertina? I need to go and get brekker,

right now, but I'll look at this later, and generate the ABC for a C/G so I can compare my fixed fingering

with your optimised fingering.

 

[2] I'd love to incorporate the sort of heuristic optimisation you are using into my own programs. Maybe

we need to talk (offline) a little further down the road, after I've looked into the techniques you mention...

Edited by lachenal74693
Link to comment
Share on other sites

  • 2 weeks later...

There is a stylistic risk to over-avoidance of bellows changes, depending on the genre of music, because bellows changes can be desirable even if there are options without them being required.  

 

It'd be interesting to see what they full contraint set could look like (making sure to balance the bellows to minimize the need for air buttons, but also to try to be consistent on phrases, and to include bellows changes for emphasis and how much genre awareness the constraints need for that)...

 

Link to comment
Share on other sites

On 9/10/2022 at 5:26 AM, Owen Anderson said:

The approach uses an optimization algorithm called the Partioned Boolean Quadratic Problem (PBQP). 

 

Looks an interesting approach. In my work life, I implemented optimisation software to solve problems of map generalisation (abstracting too much detailed information to create a simpler, smaller-scale map). To do it deterministically would require comparing everything with everything, taking millennia to complete.

 

We used Simulated Annealing, which uses a conceptual reducing 'temperature' while evaluating the 'happiness' of the system, and a set of possible actions to make things happier. While the temperature is high, then pretty drastic changes can be tried out. As the temperature reduces less and less drastic changes can be done, so that the system converges to a good (though not necessarily best) state. I'd wondered about applying it to music optimisation.

Link to comment
Share on other sites

14 hours ago, Paul_Hardy said:

Looks an interesting approach etc...

I modified one of my own programs to make different choices of which button to select

(when there are alternatives). It was a 'proof-of-concept' exercise, and I chose the button

using what I can only call a 'weighted-random' approach which is (literally!) worse than

doing nothing! The next step is to replace the 'random' code with something which makes

'sensible' choices. I don't anticipate anything happening any time soon - that sort of clever

programming is a bit above my pay grade at the moment - interesting though, if you are

that way inclined...

Edited by lachenal74693
Link to comment
Share on other sites

Here is a more complex example, a version of In The Hall of the Mountain King in A minor, but with lots of accidentals. I manually annotated the score with the output of the PBQP optimization. One interesting tidbit is that I was able to test out the model's support for optimizing around repetitions. It successfully chooses to use L10 for the D at the beginning of bar 5, avoiding a same-finger jump when repeating from the B at the end of bar 8.

 

X:1
T:In the Hall of the Mountain King
M:4/4
L:1/8
K:C
Q:100
|: ABcd ec .e2 | ^dB .d2 =d_B .d2 | 
ABcd ecea | gede g4 :|
|: e^f^ga bg .b2 | c'^g .c'2 bg .b2 |
e^f^ga bg .b2 | c'^g .c'2 b4 :|
|: ABcd ec .e2 | ^dB .d2 =d_B .d2 | 
ABcd ecea | gede g4 :|
ABcd eBea | ^gegb a4 |]

 

InTheHallOfTheMountainKing.pdf

Link to comment
Share on other sites

On 9/19/2022 at 11:19 AM, Paul_Hardy said:

 

Looks an interesting approach. In my work life, I implemented optimisation software to solve problems of map generalisation (abstracting too much detailed information to create a simpler, smaller-scale map). To do it deterministically would require comparing everything with everything, taking millennia to complete.

 

We used Simulated Annealing, which uses a conceptual reducing 'temperature' while evaluating the 'happiness' of the system, and a set of possible actions to make things happier. While the temperature is high, then pretty drastic changes can be tried out. As the temperature reduces less and less drastic changes can be done, so that the system converges to a good (though not necessarily best) state. I'd wondered about applying it to music optimisation.

 

IMO the hard part is coming up with an optimization-friendly model that can represent the cost functions of interest. PBQP happens to match to it fairly well - a lot of the costs you want to optimize are easily expressible in the PBQP model.

 

I'm using a heuristic PBQP solver under the hood, which has a reputation for working well in most instances, especially when the matrices are sparse as they are in the Anglo optimization problem. You could probably use simulated annealing as an alternative solver backend for PBQP instances with some effort, but I'd be surprised if the solutions were meaningfully better.

Link to comment
Share on other sites

On 9/19/2022 at 10:02 AM, Dave Weinstein said:

There is a stylistic risk to over-avoidance of bellows changes, depending on the genre of music, because bellows changes can be desirable even if there are options without them being required.  

 

It'd be interesting to see what they full contraint set could look like (making sure to balance the bellows to minimize the need for air buttons, but also to try to be consistent on phrases, and to include bellows changes for emphasis and how much genre awareness the constraints need for that)...

 

 

One nice thing about the PBQP model is that it express a lot of "partial" constraints. I can fairly easily tell the solver that I want a particular note to be played a particular way and let it fill in the rest, or that I want a particular pair of notes to be played in opposite bellows directions, etc.

 

In my ideal world, I think an iterative, interactive workflow would be very cool. Input an arrangement, get a first pass automated fingering, make manual tweaks, generate an updated fingering, rinse and repeat.

  • Like 1
Link to comment
Share on other sites

  • 1 month later...

After some more tinkering with the various weights of the model, here's another example: the refrain from Cielito Lindo, the well-known mariachi aye-aye-aye-aye song. This tune demonstrates the system's ability to generate a good fingering in the face of many simultaneous notes. I'm particularly pleased with the playability of the solution. It does a good job of minimizing adjacent note jumping, often reusing one finger from a pair for the next note in the sequence.

 

 

cielitolindo2.pdf

  • Like 2
Link to comment
Share on other sites

On 11/25/2022 at 4:08 AM, Owen Anderson said:

After some more tinkering with the various weights of the model, etc...

May I ask whether those tabs are for a C/G or a G/D concertina?

 

I'd like to try that out on the same tune with my own software (which uses a slightly different approach), and I need to know if it's for C/G or G/D.

 

Thanks.

Edited by lachenal74693
Link to comment
Share on other sites

22 minutes ago, lachenal74693 said:

May I ask whether those tabs are for a C/G or a G/D concertina?

 

I'd like to try that out on the same tune with my own software (which uses a slightly different approach), and I need to know if it's for C/G or G/D.

 

Thanks.

 

It's for C/G. Here's the output from running the model for a G/D Wheatstone layout. All using gcoover tab numbering.

 

  Reed assigned: ( R09Pull, pinky , R02aPull, middle )
  Reed assigned: ( R08Push, ring , R06Push, index )
  Reed assigned: ( R04Push, pinky , R04Push, pinky )
  Reed assigned: ( R04Pull, pinky , R03Pull, ring )
  Reed assigned: ( R08Push, middle , R03Pull, middle )
  Reed assigned: ( R08Push, middle , R03Pull, middle )
  Reed assigned: ( R04Push, pinky , R04Push, pinky )
  Reed assigned: ( R04Push, pinky , R04Push, pinky )
  Reed assigned: ( R06Push, index , R02Push, middle )
  Reed assigned: ( R04Push, pinky , R02Push, middle )
  Reed assigned: ( R06Push, index , R06Push, index )
  Reed assigned: ( R07Push, ring , R02Push, middle )
  Reed assigned: ( R06Push, index , R06Push, index )
  Reed assigned: ( R04Pull, pinky , L10Pull, index )
  Reed assigned: ( R07Push, middle , R07Push, middle )
  Reed assigned: ( R06Push, index , R06Push, index )
  Reed assigned: ( R06Push, index , R06Push, index )
  Reed assigned: ( R05aPull, pinky , R08Push, middle )
  Reed assigned: ( R05aPull, pinky , R08Push, middle )
  Reed assigned: ( R08Push, middle , R07Pull, index )
  Reed assigned: ( R07Pull, index , R07Pull, index )
  Reed assigned: ( R03Push, ring , R03Push, ring )
  Reed assigned: ( R04Pull, pinky , R03Pull, ring )
  Reed assigned: ( R04Pull, pinky , R03Pull, ring )
  Reed assigned: ( R03Push, ring , R02Push, middle )
  Reed assigned: ( R03Pull, ring , R02Pull, middle )
  Reed assigned: ( R06Push, index , R02Push, middle )
  Reed assigned: ( R02Push, middle , R02Push, middle )
  Reed assigned: ( R02Pull, middle , R02Pull, middle )
  Reed assigned: ( R01Push, index , R01Push, index )
  Reed assigned: ( R01Push, index , L03Push, ring )

 

Link to comment
Share on other sites

1 hour ago, yrneH said:

Cool! What does it do with the verse?

 

I haven't entered the verse yet. I expect the melody proper not to be super interesting, and inputting the melody + chords into the program is currently rather a pain. I have a plan in mind for making that easier, but I haven't implemented it yet.

Link to comment
Share on other sites

43 minutes ago, Owen Anderson said:

It's for C/G. Here's the output from running the model for a G/D Wheatstone layout. All using gcoover tab numbering.

Thanks for that extremely fast response!

 

I used the time between my question and your reply to quickly enter the tune in ABC, and run it past my software, and I now see your original was for C/G.

 

Interesting! Given the different approaches we are using, there is adequate agreement between the two. One difference in this particular case, is that my stuff generates left-hand button 8 in the penultimate bar, whereas yours generates left-hand button 5. I think I know why this happens. (Another difference is that my stuff does not generate tabs for chords (multi-headed notes, if you prefer). At some point, I need to look at how to do these - they are outwith the original design parameters for my programs, and I haven't bothered so far...)

 

I've put this one on the back-burner recently while I do other stuff. I need to pick it up again to try and implement 'intelligent' choice of buttons where there is an alternative. It will be a much less sophisticated approach than the one you are using, but still 'viable', I hope. My winter project, I think...

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...