You could do stencil stuff but I think it would be easier to just grab the
bracket tips from a font that was already designed by somebody in the first
place. Like from gonville for example (Gonville has the smallest bracket tips
out of the fonts that I checked):
\layout {
\override Score.SystemStartBracket.fonts.music = "gonville"
}
-----
Or swapping out the notation font entirely is another option:
\paper {
property-defaults.fonts.music = "Gonville"
}
Though this might not be an option for you since i noticed the example you
linked had shape notes which gonville might not have
-----
If you wanted to do stencil stuff, you could redraw them yourself (which might
be a bit of work), or rescale or rotate the existing emmentaler bracket tips
(might involve some trial and error to get right)
