From my understanding, `ast.arguments` and `inspect.Signature` are the
two builtin ways of defining the function name and arguments in a
structured way.
What I am creating is a way of configuring… well let's be specific to
my use-case. I am building a way to configure TensorFlow.
One which is very type-driven, and will error as early-as-possible
when incorrect types or lack of required parameters are provided.
I can dynamically infer things like so:
```
import inspect
import tensorflow as tf
sig = inspect.signature(tf.keras.optimizers.Adam)
tuple({
"default": sig.parameters[param].default,
"name": param,
"typ": type(sig.parameters[param].default).__name__
if sig.parameters[param].default is not inspect._empty
and sig.parameters[param].annotation is inspect._empty
else sig.parameters[param].annotation,
}
for param in sig.parameters if param != 'name'
)
```
I can also parse the docstring, as I do in my doctrans library and tool.
Which will give me the options I can provide the class. So there's an
obvious solution, to generate these classes:
```
class TensorFlowConfig(object):
optimizer: Optional[Optimizer] = None
# Or maybe a `Protocol`?
class Optimizer(object): pass
class AdamConfig(Optimizer):
learning_rate: float = 0.001
beta_1: float = 0.9
beta_2: float = 0.999
epsilon: float = 1e-07
amsgrad: bool = False
kwargs: dict = {}
TensorFlowConfig().optimizer = AdamConfig()
```
But, keeping in mind the goal to expose everything in all interfaces,
the question then becomes how to generate an argparse parser from
this. And how to generate a function from this. And how to ensure that
whatever code-completion interface one uses in Python, that it can
figure out what the correct parameters are.
So I should get an error about incorrect type when I:
```
# CLI
--optimizer 'Adam' 'learning_rate = True'
# class*
TensorFlowConfig().optimizer = tf.keras.optimizers.Adam(learning_rate=True)
TensorFlowConfig().optimizer = AdamConfig(learning_rate=True)
# Function*
MyTensorFlowClass.train(optimizer=tf.keras.optimizers.Adam(learning_rate=True))
MyTensorFlowClass.train(optimizer=AdamConfig(learning_rate=True))
* both of these next two lines—after the heading—should be equivalent
```
Syntax like this should also be valid
```
TensorFlowConfig().optimizer = 'tf.keras.optimizers.Adam'
TensorFlowConfig().optimizer = 'Adam'
MyTensorFlowClass.train(optimizer='tf.keras.optimizers.Adam')
MyTensorFlowClass.train(optimizer='Adam')
```
Do I have huge annotations like this*, with an `Any` lopped on for
further analysis down the line? - So it can easily generate into
`choices` for argparse? [which will need to be subclassed in order to
enable that "--optimizer 'Adam' 'learning_rate = True'" syntax]
*
https://github.com/SamuelMarks/ml-params-tensorflow/blob/1d48502/ml_params_tensorflow/ml_params/trainer.py#L213-L215
What makes sense?
Thanks for your insights,
Samuel Marks
Charity <https://sydneyscientific.org> | consultancy
<https://offscale.io> | open-source <https://github.com/offscale> |
LinkedIn <https://linkedin.com/in/samuelmarks>
PS: This is the `doctrans` project I referenced
https://github.com/SamuelMarks/doctrans
--
https://mail.python.org/mailman/listinfo/python-list
