subprocess equivalent for "os.execvp()"?
Hello, is there an equivalent in the subprocess module for "os.execvp()" to replace the current process with the new called one? Kind Christian -- https://mail.python.org/mailman/listinfo/python-list
Re: subprocess equivalent for "os.execvp()"?
On Sun, 8 Jan 2023 at 21:51, wrote: > > Hello, > > is there an equivalent in the subprocess module for "os.execvp()" to > replace the current process with the new called one? It won't make a subprocess, so no. It's in the os module - under the name execvp. You found it already :) ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Recommendations in terms of threading, multi-threading and/or asynchronous processes/programming? - Sent Mail - Mozilla Thunderbird
Ok, the specific usage case right now is that I need to set up a process pulling contents of e-mail messages from an IMAP protocol mail server, which I then populate into a postgresql database, and, since this is the inbox of a relatively large-scale CRM/support system, there are currently over 2.5 million e-mails in the inbox, but, it can grow by over 5 per day. I already have the basic process operating, using imap_tools, but, wanted to enable you to query the process during run-time, without needing to either check logs, or query the database itself while it is on-the-go - even if this is just for initial population time-period, since later on I will just set up code to run under a form of cron job, or handling time-based repeats itself on a separate machine. Also wanted to offer the ability to either pause, or terminate processes while it's busy batch processing large chunks of e-mail messages - either send a message to the thread, or set a global variable to tell it to end the run after the current process item has finished off, just in case. So, I think that for now, threading is probably the simplest to look into. Later on, was also considering forms of low-level monitoring for UI elements, but, this is not really related to initial task, but, could almost relate to forms of non-visual gaming interfaces, for blind/VI individuals - I am myself 100% blind, but, that's not really relevant in this context. Stay well Jacob Kruger +2782 413 4791 "Resistance is futile...but, acceptance is versatile..." On 2023/01/06 21:19, Chris Angelico wrote: On Sat, 7 Jan 2023 at 04:54, jacob kruger wrote: I am just trying to make up my mind with regards to what I should look into working with/making use of in terms of what have put in subject line? As in, if want to be able to trigger multiple/various threads/processes to run in the background, possibly monitoring their states, either via interface, or via global variables, but, possibly while processing other forms of user interaction via the normal/main process, what would be recommended? Any. All. Whatever suits your purpose. They all have different goals, different tradeoffs. Threads are great for I/O bound operations; they're easy to work with (especially in Python), behave pretty much like just having multiple things running concurrently, and generally are the easiest to use. But you'll run into limits as your thread count climbs (with a simple test, I started seeing delays at about 10,000 threads, with more serious problems at 100,000), so it's not well-suited for huge scaling. Also, only one thread at a time can run Python code, which limits them to I/O-bound tasks like networking. Multiple processes take a lot more management. You have to carefully define your communication channels (for instance, a multiprocessing.Queue() to collect results), but they can do CPU-bound tasks in parallel. So multiprocessing is a good way to saturate all of your CPU cores. Big downsides include it being much harder to share information between the processes, and much MUCH higher resource usage than threads (with the same test as the above, I ran into limitations at just over 500 processes - way fewer than the 10,000 threads!). Asynchronous I/O runs a single thread in a single process. So like multithreading, it's only good for I/O bound tasks like networking. It's harder to work with, though, since you have to be very careful to include proper await points, and you can stall out the entire event loop with one mistake (common culprits being synchronous disk I/O, and gethostbyname). But the upside is that you get near-infinite tasks, basically just limited by available memory (or other resources). Use whichever one is right for your needs. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: subprocess equivalent for "os.execvp()"?
On 1/8/23, c.bu...@posteo.jp wrote: > > is there an equivalent in the subprocess module for "os.execvp()" to > replace the current process with the new called one? A note for Windows users Avoid using any of the `os.exec*` functions on Windows. There's no support for replacing a Windows process image, so the `exec*()` functions simply spawn a child process and terminate the current one. This is a mess in general because a waiting parent isn't aware of the spawned descendant. It's particularly bad for a console process that inherits the console session, since two processes will try to use console I/O simultaneously. -- https://mail.python.org/mailman/listinfo/python-list
Re: Recommendations in terms of threading, multi-threading and/or asynchronous processes/programming? - Sent Mail - Mozilla Thunderbird
On 2023-01-08 13:49:38 +0200, jacob kruger wrote: > Ok, the specific usage case right now is that I need to set up a process > pulling contents of e-mail messages from an IMAP protocol mail server, which > I then populate into a postgresql database, and, since this is the inbox of > a relatively large-scale CRM/support system, there are currently over 2.5 > million e-mails in the inbox, but, it can grow by over 5 per day. This is probably I/O-bound. You will likely spend much more time waiting for the IMAP server or the database than parsing the messages. So you probably don't need multi-processing just to utilize all your cores. On the other hand you have some nicely separated task which can be parallelized, so multi-threading should help (async probably would work just as well or as badly as multi-threading but I find that harder to understand so I would discard it at this point). I might be mistaken, though: Depending on how much processing you need to do on these messages it might be worth it split the work across multiple processes. Check the CPU-usage of your process: If it's close to 100% you will probably gain significantly from multi-processing. > I already have the basic process operating, using imap_tools, but, wanted to > enable you to query the process during run-time, without needing to either > check logs, or query the database itself while it is on-the-go [...] > Also wanted to offer the ability to either pause, or terminate processes > while it's busy batch processing large chunks of e-mail messages So that would be an http (or other socket-based) interface? Should also be possible to add as an additional thread (or process). > So, I think that for now, threading is probably the simplest to look into. I agree with that assessment. hp -- _ | Peter J. Holzer| Story must make more sense than reality. |_|_) || | | | h...@hjp.at |-- Charles Stross, "Creative writing __/ | http://www.hjp.at/ | challenge!" signature.asc Description: PGP signature -- https://mail.python.org/mailman/listinfo/python-list
Suggestion: Regex string specifier like r and f
Maybe something like re"" It should behave exactly like a raw string but would be useful for syntax highlighting and debugging. Perhaps also for type hinting expected regex input (don't know if this is feasible). -- https://mail.python.org/mailman/listinfo/python-list
Question.
Hello, i installed phyton in Windows 10 22H2 and i can’t find the program. I used the repair that doesnt work. Enviado desde [1]Correo para Windows [2][IMG] Libre de virus.[3]www.avast.com References Visible links 1. https://go.microsoft.com/fwlink/?LinkId=550986 2. https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=emailclient 3. https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=emailclient -- https://mail.python.org/mailman/listinfo/python-list
Re: Question.
> On 8 Jan 2023, at 21:20, Angitolol36 wrote: > > Hello, i installed phyton in Windows 10 22H2 and i can’t find the program. > I used the repair that doesnt work. Does this help? https://docs.python.org/3/using/windows.html Barry > > > > > > Enviado desde [1]Correo para Windows > > > > [2][IMG] Libre de virus.[3]www.avast.com > > References > > Visible links > 1. https://go.microsoft.com/fwlink/?LinkId=550986 > 2. > https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=emailclient > 3. > https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=emailclient > -- > https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: Suggestion: Regex string specifier like r and f
On 08Jan2023 12:44, Raphael Santiago wrote: Maybe something like re"" It should behave exactly like a raw string but would be useful for syntax highlighting and debugging. Perhaps also for type hinting expected regex input (don't know if this is feasible). A nice idea. (Though I'm personally reluctant to make regexps any "easier" to use than they are already, a discussion for another thread.) But the IDE/syntax help is cool. There was discussion of essentially this idea just recently here: https://discuss.python.org/t/allow-for-arbitrary-string-prefix-of-strings/19740 which you will find interesting. I think it fizzled for 2 main reasons: - string prefixes current have semantic meaning for the Python parser/interpreter, while an artbitrary prefix ... doesn't - "too hard"? to get IDEs to use it? Of these, I only find the former compelling, and one could argue that type annotations already form a "has no effect" syntax extension to Python, so we're already in that playground :-) Cheers, Cameron Simpson -- https://mail.python.org/mailman/listinfo/python-list
Re: Suggestion: Regex string specifier like r and f
> On 8 Jan 2023, at 21:16, Raphael Santiago > wrote: > > Maybe something like re"" > It should behave exactly like a raw string but would be useful for syntax > highlighting and debugging. Perhaps also for type hinting expected regex > input (don't know if this is feasible). This is unlikely to be implemented. See https://discuss.python.org/t/allow-for-arbitrary-string-prefix-of-strings/19740/12 for related idea discussion. Barry > -- > https://mail.python.org/mailman/listinfo/python-list > -- https://mail.python.org/mailman/listinfo/python-list
Re: Question.
On 1/8/2023 7:54 AM, Angitolol36 wrote: Hello, i installed phyton in Windows 10 22H2 and i can’t find the program. I used the repair that doesnt work. This is as if you had said "I bought a car and it doesn't work". Please tell us what you did and noticed that caused you to say "i can’t find the program". Enviado desde [1]Correo para Windows [2][IMG] Libre de virus.[3]www.avast.com References Visible links 1. https://go.microsoft.com/fwlink/?LinkId=550986 2. https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=emailclient 3. https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=emailclient -- https://mail.python.org/mailman/listinfo/python-list
