How to accept argparse.log_level parameter and propagate its value to all other modules

[Zoran]

In the same folder I have three files: main.py, app_logger.py and mymodule.py 
with their contents:

# main.py
import argparse
import app_logger
import mymodule
import logging

logger = app_logger.get_logger(__name__)

def log_some_messages():
logger.debug(f'{__name__} - debug message')
logger.info(f'{__name__} - info message')
logger.warning(f'{__name__} - warning message')

if __name__ == '__main__':
parser = argparse.ArgumentParser(description="Script for executing data 
quality checks.")
parser.add_argument("--log-level", default='info', choices=['notset', 
'debug', 'info', 'warning', 'error', 'critical'], help="set log level")
args = parser.parse_args()

logger.setLevel(eval(f"logging.{args.log_level.upper()}"))

log_some_messages()
mymodule.log_some_messages()


#mymodule.py
import logging

from script_logging import app_logger

logger = app_logger.get_logger(__name__)


def log_some_messages():
logger.debug(f'{__name__} - debug message')
logger.info(f'{__name__} - info message')
logger.warning(f'{__name__} - warning message')


#app_logger.py
import logging

_log_format = f"%(asctime)s - [%(levelname)s] - %(name)s - 
(%(filename)s).%(funcName)s(%(lineno)d) - %(message)s"
_log_level = logging.DEBUG

def get_stream_handler():
stream_handler = logging.StreamHandler()
stream_handler.setLevel(_log_level)
stream_handler.setFormatter(logging.Formatter(_log_format))
return stream_handler

def get_logger(name):
logger = logging.getLogger(name)
logger.setLevel(_log_level)
logger.addHandler(get_stream_handler())
return logger

When I run it with main.py --log-level debug I get the following output:

2021-01-25 13:21:01,151 - [DEBUG] - __main__ - (main.py).log_some_messages(10) 
- __main__ - debug message
2021-01-25 13:21:01,151 - [INFO] - __main__ - (main.py).log_some_messages(11) - 
__main__ - info message
2021-01-25 13:21:01,152 - [WARNING] - __main__ - 
(main.py).log_some_messages(12) - __main__ - warning message
2021-01-25 13:21:01,152 - [DEBUG] - mymodule - 
(mymodule.py).log_some_messages(10) - mymodule - debug message
2021-01-25 13:21:01,152 - [INFO] - mymodule - 
(mymodule.py).log_some_messages(11) - mymodule - info message
2021-01-25 13:21:01,152 - [WARNING] - mymodule - 
(mymodule.py).log_some_messages(12) - mymodule - warning message

which is OK. If I change --log-level parameter to 'warning', than output is the 
following:

2021-01-25 13:22:12,760 - [WARNING] - __main__ - 
(main.py).log_some_messages(12) - __main__ - warning message
2021-01-25 13:22:12,760 - [DEBUG] - mymodule - 
(mymodule.py).log_some_messages(10) - mymodule - debug message
2021-01-25 13:22:12,761 - [INFO] - mymodule - 
(mymodule.py).log_some_messages(11) - mymodule - info message
2021-01-25 13:22:12,761 - [WARNING] - mymodule - 
(mymodule.py).log_some_messages(12) - mymodule - warning message

As you can see, in main.py log level is set correctly, but in mymodule.py it is 
not.

How to accept argparse.log_level parameter in if __name__ == '__main__': and 
propagate its value to all other modules?
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to accept argparse.log_level parameter and propagate its value to all other modules

[Zoran]

Yes you are right. I changed the files:

# main.py
import argparse
import mymodule
import logging

logger = logging.getLogger(__name__)

def log_some_messages():
logger.debug(f'{__name__} - debug message')
logger.info(f'{__name__} - info message')
logger.warning(f'{__name__} - warning message')

if __name__ == '__main__':
parser = argparse.ArgumentParser(description="Script for executing data 
quality checks.")
parser.add_argument("--log-level", default='info', choices=['notset', 
'debug', 'info', 'warning', 'error', 'critical'], help="set log level")
args = parser.parse_args()

log_level = logging.getLevelName(args.log_level.upper())
logging.basicConfig(level=log_level, format=f"%(asctime)s - [%(levelname)s] 
- %(name)s - (%(filename)s).%(funcName)s(%(lineno)d) - %(message)s")
logger.setLevel(log_level)

log_some_messages()
mymodule.log_some_messages()

#mymodule.py
import logging

logger = logging.getLogger(__name__)


def log_some_messages():
logger.debug(f'{__name__} - debug message')
logger.info(f'{__name__} - info message')
logger.warning(f'{__name__} - warning message')

and everything works as it should.
I was milsleaded by this article 
https://towardsdatascience.com/8-advanced-python-logging-features-that-you-shouldnt-miss-a68a5ef1b62d

Thanks.

-- 
https://mail.python.org/mailman/listinfo/python-list

Async telnet, which package and how?

[Zoran]

Hi, 

I have to asynchronously connect to many home routers with telnet protocol in 
order to get some values.

Upon getting input data (ip, username, password), I should connect to the 
router, execute commands, get their results, postprocess results and finally 
return certain values.
Everything should work in asyncio loop.


I searched a bit and I found that people usually use telnetlib3 for such task, 
but I can't find out examples that shows how to use telnetlib3.TelnetClient for 
my use case.

If someone here has some experiences or has some demo code, please share.

Regards.
-- 
https://mail.python.org/mailman/listinfo/python-list

async watch directory for new files

[Zoran]

Hi,

I need to watch for new files in directory, and when it shows up, I should 
create async task with file's full path for it, and wait for new file.

If anyone here used a library for such task, please share which one.

Regards.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: async watch directory for new files

[Zoran]

On Saturday, 24 April 2021 at 18:52:24 UTC+2, Dieter Maurer wrote:
> Zoran wrote at 2021-4-23 14:31 -0700: 
> >I need to watch for new files in directory, and when it shows up, I should 
> >create async task with file's full path for it, and wait for new file. 
> > 
> >If anyone here used a library for such task, please share which one.
> The solution likely depends on the OS you are using. 
> 
> For Linux, `inotify` informs applications about file system changes. 
> PyPI contains a binding for it -- with identical name.

As I can see https://pypi.org/project/inotify/ is not asyncio frendly.
Whatever I use, it should be asynchronous. 
Final OS is linux (Centos 7), but I am doing development on Windows 10 machine, 
so it would be nice if I have something that works on both.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: async watch directory for new files

[Zoran]

> Bear in mind that asyncio is NOT the only way to be asynchronous. So 
> what you're asking for is, very specifically, an asyncio-compatible 
> inotify. Turns out, there's an ainotify on PyPI that might be what you 
> want. 

Yes there is:
https://github.com/rbarrois/aionotify
https://asyncinotify.readthedocs.io/en/latest/
https://pypi.org/project/aionotify/
https://pypi.org/project/butter/
https://github.com/giannitedesco/minotaur
https://pypi.org/project/watchgod/
...

There are many of them.
I hoped that someone with experience here can suggest a library that he has 
been used on some project.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: async watch directory for new files

[Zoran]

>  
> 
> Implementations are usually just callback-based. (Apologies for the 
> generic link, I haven't needed this in Python yet: anyway, those are the 
> keywords.) 

:) before asking a question here I googled the subject a lot.
Anyway, the simplest solution is this one:

import asyncio
import pathlib
from watchgod import awatch, Change

watch_dir = pathlib.Path(r'C:\Users\User\watch_folder')


async def main():
async for changes in awatch(str(watch_dir)):
while changes:
change_type, path = changes.pop()
if change_type == Change.added:
print('processing:', path)


loop = asyncio.get_event_loop()
loop.run_until_complete(main())

watchgod library is quite young. I hope that there will be no suprises.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: async watch directory for new files

[Zoran]

> inotifywait contains several variations of a notifier, I use it rather 
> heavly here to automate such goings on as my incoming e-mail. 

Can you be more specific about inotifywait?
Do you have an link or something?
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: async watch directory for new files

[Zoran]

Anyway, thanks for participating.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Why dropbox.client.DropboxClient.put_file needs file_obj as a parameter?

[Zoran Ljubisic (mob)]

Looks like you are right.
Thanks for clarification.

Regards.
On May 25, 2015 10:02 PM, "Skip Montanaro"  wrote:

> Looks like the file_obj is what you are reading locally, and
> magnum_opus.txt is the destination name on Dropbox.
>
> Skip
>
-- 
https://mail.python.org/mailman/listinfo/python-list