While that sort of comparison seems necessary at a minimum, as described it's not sufficient. For the comparison to be valid one would have to be able to insure that 1) the two reed shoes are precisely identical in all dimensions, and that 2) the fit of the reed within the frame is identical -- to within 1/1000 cm, unless and until lesser precision can be demonstrated to be adequate -- at all points, since these factors also potentially affect the sound quality. The difficulty of precisely controlling details like these has probably contributed to the fact that no definitive studies have yet been done.
Before concluding that aluminum frames give a harsher sound, would it not be a good idea to transfer the same reeds from aluminum to brass frames and see if there are differences in sound chararcteristics between the two types of frames with the same reeds?
If it was a factor of weight, then larger and therefore heavier reeds [reed shoes!] would sound different to smaller brass shoes.
Nope. I don't have to believe that. Because there's more involved than just resonance. A more fundamental process is the mechanical analog to electrical impedance.
If it was a matter of the substance of the material, then you have to believe a piece of metal rigidly trapped in a piece of wood can somehow resonate more or less sympathetically depending on its makeup.
Hardness is only one property of these materials. Density and elasticity are two others which also affect mechanical impedance. And while aluminum and brass may be closer to each other in these properties (with duralumin somewhere in between?) than either is to sycamore, they are still quite different from each other.
Think about it. Relatively, aluminium and brass are together somewhere way up the hardness scale from Sycamore.
I suspect the reason that that is effective is not simply that the wood is "soft". You're also introducing a boundary between two materials with dissimilar characteristics, and unless their impedances are precisely "matched" in a particular way, the boundary between the different sets of properties of the two materials will interfere with any resonances. It should also be possible to "kill" resonances by using a denser material (iron? lead?) in place of the wood. Even the same material will work, as long as there are at least two parts, with a boundary between them, since the discontinuity itself will interfere with resonances.
And on top of that, what do you do if you want to cancel resonance in a piece of metal. You fix it to something soft like a piece of wood.
(The earliest Ford Falcons were advertised as having "Unibody" construction. I.e., the entire body was cast in one piece. Ever wonder why that advertising campaign disappeared? The joints between the different segments of an ordinary car's body damp out vibrations and prevent resonances from building up. I rode across the US in one of the "Unibody" Falcons, and it was quite easy to keep it below the 70 mph speed limit then current on Interstate highways, because at exactly 70 mph the car developed a resonance that felt like it was shaking itself to pieces!)
As in electrical circuits, certain combinations of impedance and frequency can result in resonances, but there are other significant effects, such as differential absorption and dispersion. In electrical circuits, e.g., control of impedances can be used to create bandpass filters, while improperly controlled impedance can result in runaway feedback. There's even an equivalent to optical indices of refraction, differences in which are used to produce lenses and color-dispersing prisms.
Why not all their qualities that have demonstrable effects? The question which started this thread is whether there are tonal effects, and I don't believe that has yet been established either way.
I can understand the emotional reaction against aluminium, I have it myself. Brass gives a feel of Victorian quality; aluminium, well, is sort of cheap. But they need to be judged for the other qualities they bring rather than their tonal qualities.
I understand that it's not pure aluminum, but duralumin -- an aluminum alloy, -- which is used for concertina reed shoes. Aluminum itself has not been successful. And this raises in my mind the question of whether there are various kinds of duralumin with differing properties, just as there are many different kinds of "brass" and many different iron alloys generally known as "steel". Perhaps there are aluminum alloys which would be better suited to making concertina reed frames than what have been used to date? I don't know, but in a Google search I did find a reference to "super-duralumin", whatever that might be.
And when I think about it I can't think of a single technical reason to use brass over aluminium. Al is easier to work, about half the weight, easier to get, cheaper.
Two technical differences in which brass might be considered superior to duralumin -- resistance to compressive force, and resistance to some forms of corrosion -- have already been referred to in this thread.
Do they? I thought most new concertinas used "accordion" reeds, mounted on aluminum plates. But even of those contemporary makers producing new "concertina" reeds, I believe that Steve Dickinson (Wheatstone), Colin Dipper, and Wim Wakker all offer both brass and "aluminum" reed shoes as options, though Jürgen Suttner's web site only mentions brass. (I don't have information on Carroll and others handy.)
But I also would guess the major reasons why almost all new concertinas have brass shoes...
Empirical testing of aspects of concertina construction is not easy as there are many factors, making tests complicated and expensive. ...
I think I have shown above that in your "thinking about it", you have overlooked or neglected some potentially important factors. I wouldn't be surprised to discover that I have, also. It's precisely the possibility that we will not succeed in thinking of all the significant factors that makes experimentation necessary.
But I suspect anyone with the technical wherewithall to set up such a test would not do so because after thinking about it you would realise it was not necessary.