I would say that recompiling code for each new type (each new combination of types that a function is called with) is the main performance cost, but that could be quite substantial if you use many different parameter values or functions with many arguments whose types can vary independently.
The other pitfall that I can see is type instability: it seems that there might be a risk that a lot of code will have to deal with objects whose type parameter can not be inferred since the object is not an argument to the function where the code resides, eg each time an instance is created based on a float value for the type parameter.
