Logging with 2 process in application
Hello, Greetings!! Have a good day! This is regarding logging issue i am facing in my application. My requirement is to create log on daily basis while my application is running, So to do this i am creating new log file at midnight by comparing the day it starts and enter to new day with the while loop continuously checking on date. In my application i have 2 processes, So what my observation is on the day of start of application both process are able to log into the file, but as new day starts new file getting created and only main process log are getting written into new file, and other process is still writing to old day log. So could you please help me to understand this issue and it will be great to know solution if you can help me with this. Thanks in advance! -- https://mail.python.org/mailman/listinfo/python-list
Re: Logging with 2 process in application
On Monday, February 22, 2021 at 10:47:57 AM UTC+5:30, Dan Stromberg wrote: > On Sun, Feb 21, 2021 at 9:10 PM gayatri funde > wrote: > > Hello, > > > > Greetings!! Have a good day! > > > > This is regarding logging issue i am facing in my application. My > > requirement is to create log on daily basis while my application is > > running, So to do this i am creating new log file at midnight by comparing > > the day it starts and enter to new day with the while loop continuously > > checking on date. > > > > In my application i have 2 processes, So what my observation is on the day > > of start of application both process are able to log into the file, but as > > new day starts new file getting created and only main process log are > > getting written into new file, and other process is still writing to old > > day log. > > > > So could you please help me to understand this issue and it will be great > > to know solution if you can help me with this. > > > Is it possible each process has open a different "file" on the same > pathname, with only one actually being visible in the filesystem? > > Linuxes and Unixes allow such things. I am unaware of Windows being > powerful enough to do so, and besides there you'd probably use the Event > Log anyway. > > If that's the case, I do not know of a Python logging-module way of > correcting the problem, but you could introduce a third process that is > used just for logging, and have the original two send their log messages to > the new 3rd logging process. Just to clarify, file names are different as i am appending date to each file name. On new day start main process is writing to new day file and another process is still writing to old date file. This i am trying on Linux system. Thanks for suggestion of creating one dedicated process for logging. But if you can help to find any is other reason why second process not able to find the new day file handler object. Thank you for your reply. -- https://mail.python.org/mailman/listinfo/python-list
Re: Logging with 2 process in application
On Monday, February 22, 2021 at 10:49:36 AM UTC+5:30, Chris Angelico wrote: > On Mon, Feb 22, 2021 at 4:11 PM gayatri funde wrote: > > > > Hello, > > > > Greetings!! Have a good day! > > > > This is regarding logging issue i am facing in my application. My > > requirement is to create log on daily basis while my application is > > running, So to do this i am creating new log file at midnight by comparing > > the day it starts and enter to new day with the while loop continuously > > checking on date. > > > > In my application i have 2 processes, So what my observation is on the day > > of start of application both process are able to log into the file, but as > > new day starts new file getting created and only main process log are > > getting written into new file, and other process is still writing to old > > day log. > > > > So could you please help me to understand this issue and it will be great > > to know solution if you can help me with this. > > > What makes the decision to create a new file? How is that done? If > you're closing the old one and opening a new one, both processes will > have to do it; if you're renaming it away and then closing and opening > with the same name, one process has to do the rename, and then the > other has to just close/open. > > It may be easier and cleaner to make use of a separate logging > facility. Many operating systems come with them. > > ChrisA By checking the new day start continuously, I am able to create new file before that i am clearing old day file handler. My doubt is Main process able to find the new day file handler but why not the other process. And before creating new file handler, i am clearing old file handler. So how other process is still have access to old day File handler. I am having little confusion here. Thank you for your reply! -- https://mail.python.org/mailman/listinfo/python-list
Re: Logging with 2 process in application
On Monday, February 22, 2021 at 11:15:27 AM UTC+5:30, Chris Angelico wrote:
> On Mon, Feb 22, 2021 at 4:41 PM gayatri funde wrote:
> >
> > On Monday, February 22, 2021 at 10:47:57 AM UTC+5:30, Dan Stromberg wrote:
> > > On Sun, Feb 21, 2021 at 9:10 PM gayatri funde
> > > wrote:
> > > > Hello,
> > > >
> > > > Greetings!! Have a good day!
> > > >
> > > > This is regarding logging issue i am facing in my application. My
> > > > requirement is to create log on daily basis while my application is
> > > > running, So to do this i am creating new log file at midnight by
> > > > comparing
> > > > the day it starts and enter to new day with the while loop continuously
> > > > checking on date.
> > > >
> > > > In my application i have 2 processes, So what my observation is on the
> > > > day
> > > > of start of application both process are able to log into the file, but
> > > > as
> > > > new day starts new file getting created and only main process log are
> > > > getting written into new file, and other process is still writing to
> > > > old
> > > > day log.
> > > >
> > > > So could you please help me to understand this issue and it will be
> > > > great
> > > > to know solution if you can help me with this.
> > > >
> > > Is it possible each process has open a different "file" on the same
> > > pathname, with only one actually being visible in the filesystem?
> > >
> > > Linuxes and Unixes allow such things. I am unaware of Windows being
> > > powerful enough to do so, and besides there you'd probably use the Event
> > > Log anyway.
> > >
> > > If that's the case, I do not know of a Python logging-module way of
> > > correcting the problem, but you could introduce a third process that is
> > > used just for logging, and have the original two send their log messages
> > > to
> > > the new 3rd logging process.
> >
> >
> > Just to clarify, file names are different as i am appending date to each
> > file name. On new day start main process is writing to new day file and
> > another process is still writing to old date file. This i am trying on
> > Linux system.
> >
> > Thanks for suggestion of creating one dedicated process for logging. But if
> > you can help to find any is other reason why second process not able to
> > find the new day file handler object.
> > Thank you for your reply.
> >
> You'll need to show your code. Most likely, though, BOTH processes
> need to independently update to the new file.
>
> ChrisA
I have 2 files mainModule.py and loggingModule.py files as below:
In loggingModule.py file i am managing new log to create , which i am importing
to mainModule.py file.
loggingModule.py
import logging
import os, sys
import os.path
import datetime
from threading import Thread
import time
firstTime = "false"
def initialize_logger(fileName):
global firstTime
try:
logger = logging.getLogger()
logger.setLevel(logging.DEBUG)
output_dir = os.getcwd()
if firstTime == "true":
for handler in logger.handlers[:]: # get rid of existing old
handlers
logger.removeHandler(handler)
# create debug file handler and set level to debug
try:
handler = logging.FileHandler(os.path.join(output_dir, fileName),
"w")
except:
print("problem to create log")
handler.setLevel(logging.DEBUG)
formatter = logging.Formatter("[%(levelname)s] (%(threadName)-30s)
%(asctime)s %(message)s ")
handler.setFormatter(formatter)
logger.addHandler(handler)
firstTime = "true"
except Exception as ex:
exc_type, exc_obj, tb = sys.exc_info()
template = "An exception of type {0} occurred at {1}. Arguments:\n{2!r}"
message = template.format(type(ex).__name__, tb.tb_lineno, ex.args)
logging.error(message)
def daily_log():
global firstTime
try:
now = datetime.datetime.now()
log_day = now.day
initialize_logger("Log_start.log")
while True:
currentDate = datetime.datetime.now().day
time.sleep(60)
if currentDate != log_day: # New day started
initialize_logger("Log_continue.log")
except Exception as ex:
exc_type, exc_obj, tb = sy
Re: Logging with 2 process in application
On Monday, February 22, 2021 at 11:33:57 AM UTC+5:30, Chris Angelico wrote:
> On Mon, Feb 22, 2021 at 5:01 PM gayatri funde wrote:
> > I have 2 files mainModule.py and loggingModule.py files as below:
> > In loggingModule.py file i am managing new log to create , which i am
> > importing to mainModule.py file.
> >
> > def child_Process(var1):
> > while True:
> > time.sleep(10)
> > logging.info('Log from Child Process')
> >
> > if __name__ == "__main__":
> > var1 = "LotsOfSunshine"
> > time.sleep(1)
> > logging.info("Log from Main process")
> >
> > child_Multiprocess = multiprocessing.Process(target=child_Process,
> > args=(var1,))
> > child_Multiprocess.start()
> >
> When you spawn a subprocess, its variables are completely separate
> from the parent. I suspect that you'll do far FAR better with
> threading here, rather than trying to share things across processes.
>
> ChrisA
Thank you Chris!
But i have need of creating a another process instead of thread for application
performance stand point.
Just i was trying to log both process events in a single file and just wanted
to create each day new log for easy debug.
--
https://mail.python.org/mailman/listinfo/python-list
Re: Logging with 2 process in application
On Monday, February 22, 2021 at 4:14:53 PM UTC+5:30, Peter Otten wrote:
> On 22/02/2021 06:57, gayatri funde wrote:
> > On Monday, February 22, 2021 at 11:15:27 AM UTC+5:30, Chris Angelico wrote:
> >> On Mon, Feb 22, 2021 at 4:41 PM gayatri funde wrote:
> >>>
> >>> On Monday, February 22, 2021 at 10:47:57 AM UTC+5:30, Dan Stromberg
> >>> wrote:
> >>>> On Sun, Feb 21, 2021 at 9:10 PM gayatri funde
> >>>> wrote:
> >>>>> Hello,
> >>>>>
> >>>>> Greetings!! Have a good day!
> >>>>>
> >>>>> This is regarding logging issue i am facing in my application. My
> >>>>> requirement is to create log on daily basis while my application is
> >>>>> running, So to do this i am creating new log file at midnight by
> >>>>> comparing
> >>>>> the day it starts and enter to new day with the while loop continuously
> >>>>> checking on date.
> >>>>>
> >>>>> In my application i have 2 processes, So what my observation is on the
> >>>>> day
> >>>>> of start of application both process are able to log into the file, but
> >>>>> as
> >>>>> new day starts new file getting created and only main process log are
> >>>>> getting written into new file, and other process is still writing to
> >>>>> old
> >>>>> day log.
> >>>>>
> >>>>> So could you please help me to understand this issue and it will be
> >>>>> great
> >>>>> to know solution if you can help me with this.
> >>>>>
> >>>> Is it possible each process has open a different "file" on the same
> >>>> pathname, with only one actually being visible in the filesystem?
> >>>>
> >>>> Linuxes and Unixes allow such things. I am unaware of Windows being
> >>>> powerful enough to do so, and besides there you'd probably use the Event
> >>>> Log anyway.
> >>>>
> >>>> If that's the case, I do not know of a Python logging-module way of
> >>>> correcting the problem, but you could introduce a third process that is
> >>>> used just for logging, and have the original two send their log messages
> >>>> to
> >>>> the new 3rd logging process.
> >>>
> >>>
> >>> Just to clarify, file names are different as i am appending date to each
> >>> file name. On new day start main process is writing to new day file and
> >>> another process is still writing to old date file. This i am trying on
> >>> Linux system.
> >>>
> >>> Thanks for suggestion of creating one dedicated process for logging. But
> >>> if you can help to find any is other reason why second process not able
> >>> to find the new day file handler object.
> >>> Thank you for your reply.
> >>>
> >> You'll need to show your code. Most likely, though, BOTH processes
> >> need to independently update to the new file.
> >>
> >> ChrisA
> >
> >
> > I have 2 files mainModule.py and loggingModule.py files as below:
> > In loggingModule.py file i am managing new log to create , which i am
> > importing to mainModule.py file.
> >
> > loggingModule.py
> [...]
> > def daily_log():
> > global firstTime
> >
> > try:
> > now = datetime.datetime.now()
> > log_day = now.day
> > initialize_logger("Log_start.log")
> > while True:
> Unless there is an exception you are forever stuck in this loop...
> > currentDate = datetime.datetime.now().day
> > time.sleep(60)
> >
> > if currentDate != log_day: # New day started
> > initialize_logger("Log_continue.log")
> >
> > except Exception as ex:
> > exc_type, exc_obj, tb = sys.exc_info()
> > template = "An exception of type {0} occurred at {1}. Arguments:\n{2!r}"
> > message = template.format(type(ex).__name__, tb.tb_lineno, ex.args)
> > logging.error(message)
> >
> >
> > daily_log()
> >
> >
> > mainModule.py
> >
> > import Logic_Check
> > import logging
> > import multiprocessing
> > from loggingModule import *
> ... and as the import implicitly invokes daily_log() you won't get here,
> i. e. the code below is not executed.
> > def child_Process(var1):
> > while True:
> > time.sleep(10)
> > logging.info('Log from Child Process')
> >
> > if __name__ == "__main__":
> > var1 = "LotsOfSunshine"
> > time.sleep(1)
> > logging.info("Log from Main process")
> >
> > child_Multiprocess = multiprocessing.Process(target=child_Process,
> > args=(var1,))
> > child_Multiprocess.start()
> >
daily_log() call is necessary in loggingModule.py because if we import
loggingModule.py module into mainModule.py, Import won't execute daily_log()
function by itself, either I make explicit call of daily_log() in mainModule.py
or loggingModule.py
--
https://mail.python.org/mailman/listinfo/python-list
