Get classes from "self.MyClass" to improve subclassability
Here is a snippet from the argparse module:
{{{
def parse_known_args(self, args=None, namespace=None):
...
# default Namespace built from parser defaults
if namespace is None:
namespace = Namespace() # < === my issue
}}}
I subclass from the class of the above snippet.
I would like to use a different Namespace class.
if the above could would use
namespace = self.Namespace()
it would be very easy for me to inject a different Namespace class.
Yes, I have seen the "namespace" kwarg.
This is not the first time I would like the upstream code to make subclassing
more fun.
Some months ago I asked myself how to call the "self.Namespace()" pattern:
http://stackoverflow.com/questions/27571848/name-of-design-pattern-get-class-from-class-level
The answer with the most upvotes states it is the Factory method pattern.
I prefer "self.Namespace()" to namespace kwargs.
What do you think?
--
https://mail.python.org/mailman/listinfo/python-list
Re: Get classes from "self.MyClass" to improve subclassability
Hi Steven,
I understand your solution. It is correct and works.
But the missing five characters "self." in the upstream code
produces a lot of more lines in the final result.
Regards,
Thomas Güttler
Am Freitag, 12. Juni 2015 14:24:06 UTC+2 schrieb Steven D'Aprano:
> On Fri, 12 Jun 2015 04:12:52 -0700, Thomas Güttler wrote:
>
> > Here is a snippet from the argparse module:
> >
> > {{{
> > def parse_known_args(self, args=None, namespace=None):
> > ...
> > # default Namespace built from parser defaults if namespace is
> > None:
> > namespace = Namespace() # < === my issue
> > }}}
> >
> > I subclass from the class of the above snippet.
> >
> > I would like to use a different Namespace class.
> >
> > if the above could would use
> >
> > namespace = self.Namespace()
> >
> > it would be very easy for me to inject a different Namespace class.
>
> Yes it would.
>
> And here is how you do it, even when the parent class doesn't:
>
> class MySubclass(ParentClass):
> Namespace = Namespace
> def parse_known_args(self, args=None, namespace=None):
> if namespace is None:
> namespace = self.Namespace()
> # any other method overriding needed
> return super().parse_known_args(args, namespace)
>
> In Python 2, you cannot use super(), you have to explicitly provide the
> arguments:
>
> return super(MySubclass,self).parse_known_args(args,namespace)
>
>
> --
> Steve
--
https://mail.python.org/mailman/listinfo/python-list
Re: Get classes from "self.MyClass" to improve subclassability
Hi,
crazy. I develop python since several years. I was not aware, that you can
change the defaults of kwargs. I am amazed, but won't use it :-)
Am Samstag, 13. Juni 2015 01:09:47 UTC+2 schrieb Terry Reedy:
> On 6/12/2015 7:12 AM, Thomas Güttler wrote:
> > Here is a snippet from the argparse module:
> >
> > {{{
> > def parse_known_args(self, args=None, namespace=None):
> > ...
> > # default Namespace built from parser defaults
> > if namespace is None:
> > namespace = Namespace() # < === my issue
> > }}}
> >
> > I subclass from the class of the above snippet.
> >
> > I would like to use a different Namespace class.
> >
> > if the above could would use
> >
> > namespace = self.Namespace()
> >
> > it would be very easy for me to inject a different Namespace class.
>
> The default arg (None) for the namespace parameter of the
> parse_known_args is an attribute of the function, not of the class or
> instance thereof. Unless the default Namespace is used elsewhere, this
> seems sensible.
>
> In CPython, at least, and probably in pypy, you can change this
> attribute. (But AFAIK, this is not guaranteed in all implementations.)
>
> >>> def f(n = 1): pass
>
> >>> f.__defaults__
> (1,)
> >>> f.__defaults__ = (2,)
> >>> f.__defaults__
> (2,)
>
> So the following works
>
> >>> class C():
> def f(n=1): print(n)
>
> >>> class D(C):
> C.f.__defaults__ = (2,)
>
> >>> D.f()
> 2
>
> Of course, this may or may not do more than you want.
>
> >>> C.f()
> 2
>
> --
> Terry Jan Reedy
--
https://mail.python.org/mailman/listinfo/python-list
From logging to files to a better solution: syslog, Sentry, Logstash, ....
Up to now we use simple logging to files. We don't use syslog or an other server based solution. I am unsure which architecture works for our environment. Our environment: - django based applications - a lot of batch/cron jobs (non web gui) processing - One linux server runs several systems. Each system has its own virtualenv. - servers are running in the intranet of our customers. - We have access via VPN, but it is slow. And sometimes down for some minutes. - 40MByte logs per day per server. Why we are unhappy with logging to files: - filtering: We don't want to get INFO messages over the VPN. Background: Sometimes a there is too much logging and we don't want to pull gigabytes over the VPN. - Rotating: Rotating files is possible, but somehow cumbersome. - Support structured logging of values (json) in the future. I have not locked at Sentry or Logstash in detail. Which solution could fit for our environment? Please ask if you have question! Regards, Thomas Güttler -- https://mail.python.org/mailman/listinfo/python-list
Re: From logging to files to a better solution: syslog, Sentry, Logstash, ....
Am Donnerstag, 10. September 2015 08:42:47 UTC+2 schrieb dieter: > Thomas Güttler writes: > > ... > > Why we are unhappy with logging to files: > > > > - filtering: We don't want to get INFO messages over the VPN. > > You can quite easily control at what level messages are logged with > the standard Python logging framework. Each handler has a level > and will ignore messages at a lower level. I want INFO to be logged and stored on the remote host. Therefore I must not filter INFO messages. I don't want to pull INFO messages over the VPN. Ergo, the filtering at Python level does not help in my use case. Or I am missing something. And now I have an ugly soup. The ugly soup is a text file with not well defined syntax. It looks line based. But sometimes there are log messages which span multiple lines Remember: "Life is too short to (re)write parsers" Yes, there are tools to parse that soup. But why create this soup in the first place? That's why I want to move from file based logging to a different solution. Unfortunately there too many choices (graylog, logstash, sentry, ...) :-( > > - Rotating: Rotating files is possible, but somehow cumbersome. > > There are standard tools to rotate logfiles. > > > - Support structured logging of values (json) in the future. > > Again, the Python logging framework is quite flexible with > respect to the format of logged messages. > > > ... > > Which solution could fit for our environment? > > I work for a customer with a similar environment (he uses "Zope" instead > of "Django") - and he uses logfiles. The logfiles are automatically > rotated and there are in the order of half a dozen to a dozen logfiles > per day. > > When I have to analyse a problem with the help of the logfiles, > I do not copy them via VPN but do the filtering remotely and only > copy the filtered portion, if necessary. Good to know that I am not the only one running servers in remote intranets. Regards, Thomas Güttler -- https://mail.python.org/mailman/listinfo/python-list
Re: From logging to files to a better solution: syslog, Sentry, Logstash, ....
Am Freitag, 11. September 2015 11:03:52 UTC+2 schrieb jmp:
> On 09/11/2015 09:22 AM, Thomas Güttler wrote:
> >
> > I want INFO to be logged and stored on the remote host.
> > Therefore I must not filter INFO messages.
> >
> > I don't want to pull INFO messages over the VPN.
> >
> > Ergo, the filtering at Python level does not help in my use case.
> > Or I am missing something.
>
> Probably,
>
> Your logger should have
>
>* a remote host handler
>* and a VPN handler.
>
> You can set filters and log levels separately for each handler.
> More info here
> https://docs.python.org/2/library/logging.html#handler-objects
> https://docs.python.org/2/howto/logging-cookbook.html#logging-to-multiple-destinations
>
> Something like (python 2.7)
>
> import logging
>
> logCfg = {
> 'remote':(
> logging.StreamHandler(),
> logging.Formatter('Remote - %(levelname)s - %(message)s'),
> logging.INFO,
> ),
> 'vpn':(
> logging.StreamHandler(),
> logging.Formatter('VPN - %(levelname)s - %(message)s'),
> logging.ERROR,
> ),
> }
Yes, I could do it this way.
But somehow I am not happy with this solution.
I think the filtering should be outside of python.
I like DevOp, but sometimes clear responsibilities help to
cut big problems into smaller ones.
I would like to handle the log message filtering outside of python.
Can you understand my concerns?
Thomas Güttler
--
https://mail.python.org/mailman/listinfo/python-list
Re: From logging to files to a better solution: syslog, Sentry, Logstash, ....
Am Freitag, 11. September 2015 10:18:11 UTC+2 schrieb marco@colosso.nl: > On Friday, September 11, 2015 at 9:22:42 AM UTC+2, Thomas Güttler wrote: > > Am Donnerstag, 10. September 2015 08:42:47 UTC+2 schrieb dieter: > > > Thomas Güttler writes: > > > > ... > > > > Why we are unhappy with logging to files: > > > > > > > > - filtering: We don't want to get INFO messages over the VPN. > > > > > > You can quite easily control at what level messages are logged with > > > the standard Python logging framework. Each handler has a level > > > and will ignore messages at a lower level. > > > > > > I want INFO to be logged and stored on the remote host. > > Therefore I must not filter INFO messages. > > > > I don't want to pull INFO messages over the VPN. > > > > Ergo, the filtering at Python level does not help in my use case. > > Or I am missing something. > > > > And now I have an ugly soup. > > > > The ugly soup is a text file with not well defined syntax. It looks line > > based. But sometimes there are log messages which span multiple lines > > > > Remember: "Life is too short to (re)write parsers" > > > > Yes, there are tools to parse that soup. But why create this soup in > > the first place? > > > > That's why I want to move from file based logging to a different solution. > > > > Unfortunately there too many choices (graylog, logstash, sentry, ...) :-( > > > > > > > > > > - Rotating: Rotating files is possible, but somehow cumbersome. > > > > > > There are standard tools to rotate logfiles. > > > > > > > - Support structured logging of values (json) in the future. > > > > > > Again, the Python logging framework is quite flexible with > > > respect to the format of logged messages. > > > > > > > ... > > > > Which solution could fit for our environment? > > > > > > I work for a customer with a similar environment (he uses "Zope" instead > > > of "Django") - and he uses logfiles. The logfiles are automatically > > > rotated and there are in the order of half a dozen to a dozen logfiles > > > per day. > > > > > > When I have to analyse a problem with the help of the logfiles, > > > I do not copy them via VPN but do the filtering remotely and only > > > copy the filtered portion, if necessary. > > > > Good to know that I am not the only one running servers in remote intranets. > > > > Regards, > > Thomas Güttler > > So, if logging to json is on the horizon anyway, why don't you create > something like a MongoDb handler in the standard Python logging framework > and run a MongoDb server in the client intranet? You could then connect > over VPN to MongoDb, filter the warning/error messages there on the server > side and fetch them to your local systems for analysis. Yes I could set up a MongoDB server. But I would like to reuse and not to reinvent. I would like to go a proven solid road. -- https://mail.python.org/mailman/listinfo/python-list
Re: From logging to files to a better solution: syslog, Sentry, Logstash, ....
Am Mittwoch, 16. September 2015 07:58:11 UTC+2 schrieb dieter:
> Thomas Güttler writes:
> > Am Freitag, 11. September 2015 11:03:52 UTC+2 schrieb jmp:
> > ...
> >> Something like (python 2.7)
> >>
> >> import logging
> >>
> >> logCfg = {
> >> 'remote':(
> >> logging.StreamHandler(),
> >> logging.Formatter('Remote - %(levelname)s - %(message)s'),
> >> logging.INFO,
> >> ),
> >> 'vpn':(
> >> logging.StreamHandler(),
> >> logging.Formatter('VPN - %(levelname)s - %(message)s'),
> >> logging.ERROR,
> >> ),
> >> }
> >
> >
> > Yes, I could do it this way.
> >
> > But somehow I am not happy with this solution.
> >
> > I think the filtering should be outside of python.
>
> Do you think, it will be easier there?
>
> You could also use the "syslog" handler and use "syslog"
> configuration features to separate the various message levels.
> >From my point of view, this will not be easier - but outside of Python :-)
>
> And you can develop your own Python logging handler delegating logging to
> your favorite external logging subsystem and then configure that.
> Likely the hardest approach...
Yes, this is a python list. I like python programming.
But I don't want to solve everything with one tool.
I wanted to know how python folks handle their logs.
I think filtering mails should be done outside the python interpreter.
Strange that no one seems to use one of the mentioned tools for log handling.
Regards,
Thomas
--
https://mail.python.org/mailman/listinfo/python-list
