Re: Eventfd with epoll BlockingIOError

[Barry]

> On 24 Nov 2021, at 23:09, Jen via Python-list  wrote:
> 
> I have a C program that uses fork-execv to run Python 3.10 in a child 
> process, and I am using eventfd with epoll for IPC between them.  The eventfd 
> file descriptor is created in C and passed to Python through execv.  Once the 
> Python child process starts I print the file descriptor to verify that it is 
> correct (it is).  
> 
> In this scenario C will write to the eventfd at intervals and Python will 
> read the eventfd and take action based on the value in the eventfd.  But in 
> the Python while True loop I get "BlockingIOError: [Errno 11] Resource 
> temporarily unavailable" then with each new read it prints "Failed epoll_wait 
> Bad file descriptor." 
> 
> This is the Python code:
> 
> #!/usr/bin/python3
> import sys
> import os
> import select
> 
> print("Inside Python")
> 
> event_fd = int(sys.argv[3])
> 
> print("Eventfd received by Python")
> print(event_fd)
> 
> ep = select.epoll(-1)
> ep.register(event_fd, select.EPOLLIN | select.EPOLLOUT)
> 
> event_write_value = 100
> 
> while True:
> 
> print("Waiting in Python for event")
> ep.poll(timeout=None, maxevents=- 1)
> v = os.eventfd_read(event_fd)
> 
> if v != 99:
> print("found")
> print(v)
> os.eventfd_write(event_fd, event_write_value)
> 
> if v == 99:
> os.close(event_fd)
> 
> This is the C code that writes to Python, then waits for Python to write back:
> 
> ssize_t epoll_write(int event_fd, int epoll_fd, struct epoll_event * 
> event_struc, int action_code)
> {
>int64_t ewbuf[1];
>ewbuf[0] = (int64_t)action_code;
>int maxevents = 1;
>int timeout = -1;
> 
>fprintf(stdout, " Writing to Python \n%d", event_fd);
> 
>write(event_fd, &ewbuf, 8);
> 
> if (epoll_wait(epoll_fd, event_struc, maxevents, timeout) == -1)
> {
> fprintf(stderr, "Failed epoll_wait %s\n", strerror(errno));
> }
> 
> ssize_t rdval = read(event_fd, &ewbuf, 8);   
> 
> fprintf(stdout, " Received from Python \n%ld", rdval);
> 
> return 0;
> }
> 
> This is the screen output when I run with gdb:
> 
>   Inside Python
> Eventfd received by Python
> 5
> Waiting in Python for event
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.10/runpy.py", line 196, in
> _run_module_as_main
> return _run_code(code, main_globals, None,
>   File "/usr/local/lib/python3.10/runpy.py", line 86, in _run_code
> exec(code, run_globals)
>   File "/opt/P01_SH/NPC_CPython.py", line 36, in 
> v = os.eventfd_read(event_fd)
> BlockingIOError: [Errno 11] Resource temporarily unavailable
> Writing to Python
> 5 Received from Python
> 8 Writing to Python
> Failed epoll_wait Bad file descriptor
> 5 Received from Python
> 8 Writing to Python
> Failed epoll_wait Bad file descriptor
> 5 Received from Python
> -1time taken 0.000548
> Failed to close epoll file descriptor
> Unlink_shm status: Bad file descriptor
> fn() took 0.000648 seconds to execute
> [Inferior 1 (process 12618) exited normally]
> (gdb)
> 
> So my question is why do I get "BlockingIOError: [Errno 11] Resource 
> temporarily unavailable" and "Failed epoll_wait Bad file descriptor" from 
> Python? 

Is the fd set to non blocking? Try use fcntl to set the fd non blocking.
I am not sure if the non block state is inherited on a fork.

Do you have a C version of this code that works as a test?
Try running the python version under strace to see exactly what system calls 
are used.

Barry

> 
> -- 
> Sent with Tutanota, the secure & ad-free mailbox. 
> -- 
> https://mail.python.org/mailman/listinfo/python-list

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: pyinstaller wrong classified as Windows virus

[Ulli Horlacher]

Dan Purgert  wrote:

> > When I compile my programs with pyinstaller, Windows classifies them as
> > virus and even deletes them!
> > [...]
> 
> Have you tried compiling from a different machine? Maybe there's
> something broken on the one that's flagging them. 

I have only this Windows installation.

But meanwhile I can compile my program and Windows does not complain any
more about Viruses, though I have not changed anything in my source code!


> Alternatively, maybe run updates?

I have done this before without any success (in respect to the wrong virus
report).

-- 
Ullrich Horlacher  Server und Virtualisierung
Rechenzentrum TIK 
Universitaet Stuttgart E-Mail: horlac...@tik.uni-stuttgart.de
Allmandring 30aTel:++49-711-68565868
70569 Stuttgart (Germany)  WWW:http://www.tik.uni-stuttgart.de/
-- 
https://mail.python.org/mailman/listinfo/python-list

pyinstaller wrong classified as Windows virus

[Ulli Horlacher]

When I compile my programs with pyinstaller, Windows classifies them as
virus and even deletes them!

pyinstaller.exe --onefile --noconsole -i fex.ico fextasy.py
187 INFO: PyInstaller: 4.7
187 INFO: Python: 3.10.0
218 INFO: Platform: Windows-10-10.0.19041-SP0
218 INFO: wrote P:\W10\fextasy.spec
(...)
14392 INFO: Copying 0 resources to EXE
14392 INFO: Emedding manifest in EXE
14392 INFO: Updating manifest in P:\W10\dist\fextasy.exe
14533 INFO: Updating resource type 24 name 1 language 0
14579 INFO: Appending PKG archive to EXE
18836 INFO: Building EXE from EXE-00.toc completed successfully.

https://fex.flupp.org/fop/ylds7Y9d/X-20211125101112.png

What can I do?

Rebooting does not help :-}

-- 
Ullrich Horlacher  Server und Virtualisierung
Rechenzentrum TIK 
Universitaet Stuttgart E-Mail: horlac...@tik.uni-stuttgart.de
Allmandring 30aTel:++49-711-68565868
70569 Stuttgart (Germany)  WWW:http://www.tik.uni-stuttgart.de/
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: pyinstaller wrong classified as Windows virus

[Dan Purgert]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA512

Ulli Horlacher wrote:
> When I compile my programs with pyinstaller, Windows classifies them as
> virus and even deletes them!
> [...]
> What can I do?

Stop writing viruses ;)

Have you tried compiling from a different machine? Maybe there's
something broken on the one that's flagging them. 

Alternatively, maybe run updates?

> Rebooting does not help :-}
If testing from a different machine works ... format and reinstall might
be the fastest fix for the affected machine.

-BEGIN PGP SIGNATURE-

iQIzBAEBCgAdFiEE3asj+xn6fYUcweBnbWVw5UznKGAFAmGfbS8ACgkQbWVw5Uzn
KGBvQA//YuSJChhmj5fXzkOHD5W+horNsyS3psmFjQzp/opeeF+tqfkeDNXyUwkb
+YxDgq7KDvpaUbaMiYtmzs0fZ6fLaWPHA6pJFS3+DS8Wu/EoPmHyfe6bw0OOFiBI
AB8Zr8Y7e1GntyE/HQ53+outTxDwpTStU+MTyqTjfMDRYl3NyYhnb0rbt0aTQWP0
LzYC6HM5g2EFTrfiAImuoJisu5aofw90QjJEqlRn+MQNcFm8bvAdXpWQ6fp0iNRx
IbXhb+HoL/WNy9sxeWoBRMrNY9wu1lgBDI8IPIu8m+rz13WgIX/l2CMbgxiXW3bN
WbKa4UZ1z7NB2sYhhJ9D0V4pUukFae2A9RaDV6aDC5Vm7ZNQ7eLuEsbjgXABP/qc
0OZODTUN/KYSlMa6E5azdxR5um4jQ85hcsKjtOr7UqwMurrzRL9KxSgsZVlj5FzY
vCxCw1aoqD8WBxunOLnwKd8Fonch0/Ov8a6IA1ZKxx3VbDlY3ZgVMpqYtjzw+wgH
hMD2Int0fJtiC+TBl18P7s47q5RlXbA7mCrjEXuJGdIbZ6K9SxMjWnWycCcVwNHC
LrAKWiA2xOSNcJzpXYN8U2TMsYZCLdLeuNyR5T8QLTvHokQTYuTba2M2EYraQt73
oWQsu47ws/IIAQ/EXOBGMcenhE3wTPHT/36a6udOAQktHzbqKGg=
=nm2S
-END PGP SIGNATURE-

-- 
|_|O|_| Github: https://github.com/dpurgert
|_|_|O| PGP: DDAB 23FB 19FA 7D85 1CC1  E067 6D65 70E5 4CE7 2860
|O|O|O| Former PGP: 05CA 9A50 3F2E 1335 4DC5  4AEE 8E11 DDF3 1279 A281
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Eventfd with epoll BlockingIOError

[Barry Scott]

> On 24 Nov 2021, at 22:42, Jen via Python-list  wrote:
> 
> I have a C program that uses fork-execv to run Python 3.10 in a child 
> process, and I am using eventfd with epoll for IPC between them.  The eventfd 
> file descriptor is created in C and passed to Python through execv.  Once the 
> Python child process starts I print the file descriptor to verify that it is 
> correct (it is).  
> 
> In this scenario C will write to the eventfd at intervals and Python will 
> read the eventfd and take action based on the value in the eventfd.  But in 
> the Python while True loop I get "BlockingIOError: [Errno 11] Resource 
> temporarily unavailable" then with each new read it prints "Failed epoll_wait 
> Bad file descriptor." 
> 
> This is the Python code:
> 
> #!/usr/bin/python3
> import sys
> import os
> import select
> 
> print("Inside Python")
> 
> event_fd = int(sys.argv[3])
> 
> print("Eventfd received by Python")
> print(event_fd)
> 
> ep = select.epoll(-1)
> ep.register(event_fd, select.EPOLLIN | select.EPOLLOUT)

This says tell me if I can read or write to the event_fd.
write will be allowed until the kernel buffers are full.

Usually you only add EPOLLOUT if you have data to write.
In this case do not set EPOLLOUT.

And if you know that you will never fill the kernel buffers then you
do not need to bother polling for write.

> 
> event_write_value = 100
> 
> while True:
> 
> print("Waiting in Python for event")
> ep.poll(timeout=None, maxevents=- 1)

You have to get the result of the poll() and process the list of entries that 
are returned.

You must check that POLLIN is set before attempting the read.


> v = os.eventfd_read(event_fd)

Will raise EWOULDBLOCK because there is no data available to read.

Here is the docs from python:

poll.poll([timeout]) 

Polls the set of registered file descriptors, and returns a possibly-empty list 
containing (fd, event) 2-tuples for the descriptors that have events or errors 
to report. fd is the file descriptor, and event is a bitmask with bits set for 
the reported events for that descriptor — POLLIN for waiting input, POLLOUT to 
indicate that the descriptor can be written to, and so forth. An empty list 
indicates that the call timed out and no file descriptors had any events to 
report. If timeout is given, it specifies the length of time in milliseconds 
which the system will wait for events before returning. If timeout is omitted, 
negative, or None 
,
 the call will block until there is an event for this poll object.

You end up with code like this:

for fd_event in ep.poll():
fd, event == fd_event
if (event&select.POLLIN) != 0 and fd == event_fd:
v = os.eventfd_read(event_fd)

> 
> if v != 99:
> print("found")
> print(v)
> os.eventfd_write(event_fd, event_write_value)
> 
> if v == 99:
> os.close(event_fd)
> 
> This is the C code that writes to Python, then waits for Python to write back:
> 
> ssize_t epoll_write(int event_fd, int epoll_fd, struct epoll_event * 
> event_struc, int action_code)
> {
>int64_t ewbuf[1];
>ewbuf[0] = (int64_t)action_code;
>int maxevents = 1;
>int timeout = -1;
> 
>fprintf(stdout, " Writing to Python \n%d", event_fd);
> 
>write(event_fd, &ewbuf, 8);
> 
> if (epoll_wait(epoll_fd, event_struc, maxevents, timeout) == -1)
> {
> fprintf(stderr, "Failed epoll_wait %s\n", strerror(errno));
> }
> 
> ssize_t rdval = read(event_fd, &ewbuf, 8);   
> 
> fprintf(stdout, " Received from Python \n%ld", rdval);
> 
> return 0;
> }
> 
> This is the screen output when I run with gdb:
> 
>   Inside Python
> Eventfd received by Python
> 5
> Waiting in Python for event
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.10/runpy.py", line 196, in 
> _run_module_as_main
> return _run_code(code, main_globals, None,
>   File "/usr/local/lib/python3.10/runpy.py", line 86, in _run_code
> exec(code, run_globals)
>   File "/opt/P01_SH/NPC_CPython.py", line 36, in 
> v = os.eventfd_read(event_fd)
> BlockingIOError: [Errno 11] Resource temporarily unavailable

Expected as there you have not checked that there is data to read.
Check for POLLIN being set.

> Writing to Python
> 5 Received from Python
> 8 Writing to Python
> Failed epoll_wait Bad file descriptor
> 5 Received from Python
> 8 Writing to Python
> Failed epoll_wait Bad file descriptor
> 5 Received from Python
> -1time taken 0.000548
> Failed to close epoll file descriptor
> Unlink_shm status: Bad file descriptor
> fn() took 0.000648 seconds to execute
> [Inferior 1 (process 12618) exited normally]
> (gdb)
> 
> So my question is why do I get "BlockingIOError: [Errno 11] Resource 
> temporarily unavailable" and "Failed epoll_wait Bad file descriptor" from 
> Python? 

If your protocol is not trivia you should implement a state machine to know 
what to do at each event.

Barry

> 
> -- 
> Sent with Tutanota, the secure

Re: pyinstaller wrong classified as Windows virus

[Barry Scott]

> On 25 Nov 2021, at 09:20, Ulli Horlacher  
> wrote:
> 
> When I compile my programs with pyinstaller, Windows classifies them as
> virus and even deletes them!

Microsoft will fix the malware detection if you provide the info they need.

Submit false positive info to: 
https://www.microsoft.com/security/portal/submission/submit.aspx 


I have done this for a false positive in the past and they do resolve the false 
positive.

Barry


> 
> pyinstaller.exe --onefile --noconsole -i fex.ico fextasy.py
> 187 INFO: PyInstaller: 4.7
> 187 INFO: Python: 3.10.0
> 218 INFO: Platform: Windows-10-10.0.19041-SP0
> 218 INFO: wrote P:\W10\fextasy.spec
> (...)
> 14392 INFO: Copying 0 resources to EXE
> 14392 INFO: Emedding manifest in EXE
> 14392 INFO: Updating manifest in P:\W10\dist\fextasy.exe
> 14533 INFO: Updating resource type 24 name 1 language 0
> 14579 INFO: Appending PKG archive to EXE
> 18836 INFO: Building EXE from EXE-00.toc completed successfully.
> 
> https://fex.flupp.org/fop/ylds7Y9d/X-20211125101112.png
> 
> What can I do?
> 
> Rebooting does not help :-}
> 
> -- 
> Ullrich Horlacher  Server und Virtualisierung
> Rechenzentrum TIK 
> Universitaet Stuttgart E-Mail: horlac...@tik.uni-stuttgart.de
> Allmandring 30aTel:++49-711-68565868
> 70569 Stuttgart (Germany)  WWW:http://www.tik.uni-stuttgart.de/
> -- 
> https://mail.python.org/mailman/listinfo/python-list
> 

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: pyinstaller wrong classified as Windows virus

[Michael Torrie]

On 11/25/21 2:20 AM, Ulli Horlacher wrote:
> When I compile my programs with pyinstaller, Windows classifies them as
> virus and even deletes them!
> 
> pyinstaller.exe --onefile --noconsole -i fex.ico fextasy.py
> 187 INFO: PyInstaller: 4.7
> 187 INFO: Python: 3.10.0
> 218 INFO: Platform: Windows-10-10.0.19041-SP0
> 218 INFO: wrote P:\W10\fextasy.spec
> (...)
> 14392 INFO: Copying 0 resources to EXE
> 14392 INFO: Emedding manifest in EXE
> 14392 INFO: Updating manifest in P:\W10\dist\fextasy.exe
> 14533 INFO: Updating resource type 24 name 1 language 0
> 14579 INFO: Appending PKG archive to EXE
> 18836 INFO: Building EXE from EXE-00.toc completed successfully.
> 
> https://fex.flupp.org/fop/ylds7Y9d/X-20211125101112.png
> 
> What can I do?

False positive virus detection is pretty common with pyinstaller from
what I can see on the Googles.  It's actually very common problem with
less-popular compilers and languages too. Not sure what it is that trips
them all up.

Submit your exe to virustotal.com and then the only real solution is to
submit it to each major antivirus vendor as a false positive and hope
things get changed.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: pyinstaller wrong classified as Windows virus

[Ulli Horlacher]

Ulli Horlacher  wrote:
> Dan Purgert  wrote:
> 
> > > When I compile my programs with pyinstaller, Windows classifies them as
> > > virus and even deletes them!
> > > [...]
> > 
> > Have you tried compiling from a different machine? Maybe there's
> > something broken on the one that's flagging them. 
> 
> I have only this Windows installation.
> 
> But meanwhile I can compile my program and Windows does not complain any
> more about Viruses, though I have not changed anything in my source code!

And now the bad virus reports are back. With the SAME sourcecode!
It is totally erratic.

-- 
Ullrich Horlacher  Server und Virtualisierung
Rechenzentrum TIK 
Universitaet Stuttgart E-Mail: horlac...@tik.uni-stuttgart.de
Allmandring 30aTel:++49-711-68565868
70569 Stuttgart (Germany)  WWW:http://www.tik.uni-stuttgart.de/
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: pyinstaller wrong classified as Windows virus

[Ulli Horlacher]

Barry Scott  wrote:
> 
> 
> > On 25 Nov 2021, at 09:20, Ulli Horlacher  
> > wrote:
> > 
> > When I compile my programs with pyinstaller, Windows classifies them as
> > virus and even deletes them!
> 
> Microsoft will fix the malware detection if you provide the info they need.
> 
> Submit false positive info to: 
> https://www.microsoft.com/security/portal/submission/submit.aspx 
> 

I cannot submit my executables, because the Windows Virus scannners
deletes them as soon as I compile my program!

And I need a Microsoft login to submit a file!
I do not have such a login.


-- 
Ullrich Horlacher  Server und Virtualisierung
Rechenzentrum TIK 
Universitaet Stuttgart E-Mail: horlac...@tik.uni-stuttgart.de
Allmandring 30aTel:++49-711-68565868
70569 Stuttgart (Germany)  WWW:http://www.tik.uni-stuttgart.de/
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: pyinstaller wrong classified as Windows virus

[Chris Angelico]

On Fri, Nov 26, 2021 at 3:49 AM Ulli Horlacher
 wrote:
>
> Ulli Horlacher  wrote:
> > Dan Purgert  wrote:
> >
> > > > When I compile my programs with pyinstaller, Windows classifies them as
> > > > virus and even deletes them!
> > > > [...]
> > >
> > > Have you tried compiling from a different machine? Maybe there's
> > > something broken on the one that's flagging them.
> >
> > I have only this Windows installation.
> >
> > But meanwhile I can compile my program and Windows does not complain any
> > more about Viruses, though I have not changed anything in my source code!
>
> And now the bad virus reports are back. With the SAME sourcecode!
> It is totally erratic.
>

Your Python source code is only a very small part of the code.
Unfortunately, if you're not going to go to the effort of getting your
executables signed and added to the full ecosystem, this sort of
hassle is going to be eternal. It's yet another way that turning
Python scripts into executables is a ridiculous amount of hassle. Yet
another reason to just distribute .py files.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: pyinstaller wrong classified as Windows virus

[Ulli Horlacher]

Chris Angelico  wrote:

> Unfortunately, if you're not going to go to the effort of getting your
> executables signed

I cannot sign my executables (how can I do it anyway?), because Windows
deletes my executable as soon as I have compiled them! They exist only
for a few seconds and then they are gone.


> another reason to just distribute .py files.

I cannot do that because my users do not have Python installed and they
are not allowed to do it.

-- 
Ullrich Horlacher  Server und Virtualisierung
Rechenzentrum TIK 
Universitaet Stuttgart E-Mail: horlac...@tik.uni-stuttgart.de
Allmandring 30aTel:++49-711-68565868
70569 Stuttgart (Germany)  WWW:http://www.tik.uni-stuttgart.de/
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: pyinstaller wrong classified as Windows virus

[Chris Angelico]

On Fri, Nov 26, 2021 at 4:18 AM Ulli Horlacher
 wrote:
>
> Chris Angelico  wrote:
>
> > Unfortunately, if you're not going to go to the effort of getting your
> > executables signed
>
> I cannot sign my executables (how can I do it anyway?), because Windows
> deletes my executable as soon as I have compiled them! They exist only
> for a few seconds and then they are gone.
>
>
> > another reason to just distribute .py files.
>
> I cannot do that because my users do not have Python installed and they
> are not allowed to do it.
>

Are they really allowed to install your unsigned executables but are
not allowed to install Python from a known and trusted source?

If there's some bizarre loophole that allows them to run completely
untrusted binary code, but not to run legitimate code that can be
fetched from a variety of trusted places (including python.org, the
Windows store, etc), then I'm afraid you're on your own, and will
probably need to play around with the exact loophole to figure out
what is going to be permitted.

Alternatively, just go find the person who decides what gets
installed, and request a Python interpreter to be added to the
permitted list. That's probably easier, and it's certainly going to be
better long-term.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: pyinstaller wrong classified as Windows virus

[Barry]

> On 25 Nov 2021, at 16:51, Ulli Horlacher  
> wrote:
> 
> Barry Scott  wrote:
>> 
>> 
 On 25 Nov 2021, at 09:20, Ulli Horlacher  
 wrote:
>>> 
>>> When I compile my programs with pyinstaller, Windows classifies them as
>>> virus and even deletes them!
>> 
>> Microsoft will fix the malware detection if you provide the info they need.
>> 
>> Submit false positive info to: 
>> https://www.microsoft.com/security/portal/submission/submit.aspx 
>> 
> 
> I cannot submit my executables, because the Windows Virus scannners
> deletes them as soon as I compile my program!

You should be able to tell the software to quarantine instead of delete.
Or even disable while you build one instance.
I am assuming you have admin control.

> And I need a Microsoft login to submit a file!
> I do not have such a login.
It’s free to get an account. How badly do you want to fix this?

> 
> 
> -- 
> Ullrich Horlacher  Server und Virtualisierung
> Rechenzentrum TIK 
> Universitaet Stuttgart E-Mail: horlac...@tik.uni-stuttgart.de
> Allmandring 30aTel:++49-711-68565868
> 70569 Stuttgart (Germany)  WWW:http://www.tik.uni-stuttgart.de/
> -- 
> https://mail.python.org/mailman/listinfo/python-list
> 

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: pyinstaller wrong classified as Windows virus

[Richard Damon]

On 11/25/21 12:21 PM, Chris Angelico wrote:

On Fri, Nov 26, 2021 at 4:18 AM Ulli Horlacher
 wrote:

Chris Angelico  wrote:


Unfortunately, if you're not going to go to the effort of getting your
executables signed

I cannot sign my executables (how can I do it anyway?), because Windows
deletes my executable as soon as I have compiled them! They exist only
for a few seconds and then they are gone.



another reason to just distribute .py files.

I cannot do that because my users do not have Python installed and they
are not allowed to do it.


Are they really allowed to install your unsigned executables but are
not allowed to install Python from a known and trusted source?

If there's some bizarre loophole that allows them to run completely
untrusted binary code, but not to run legitimate code that can be
fetched from a variety of trusted places (including python.org, the
Windows store, etc), then I'm afraid you're on your own, and will
probably need to play around with the exact loophole to figure out
what is going to be permitted.

Alternatively, just go find the person who decides what gets
installed, and request a Python interpreter to be added to the
permitted list. That's probably easier, and it's certainly going to be
better long-term.

ChrisA


My first guess is it isn't so much what is 'allowed' but what can be 
easily done.


On a somewhat locked down computer, the user does not have admin rights, 
so needs to get 'IT' to run any installers that need admin permissions 
to run.


And EXE that just needs to be copied to the computer and rhen just RUN, 
doesn't need IT to 'install' it (they just can't put it into Program 
Files, but that isn't really that important for programs that don't need 
an installer.


Likely, just copying an EXE file from an outside source may still be 
against the rules (and needs approval), but some think if they can do it 
and no one complains, it must be ok. On the other hand, they may have 
given approval, knowing the source.


--
Richard Damon

--
https://mail.python.org/mailman/listinfo/python-list

Re: pyinstaller wrong classified as Windows virus

[Chris Angelico]

On Fri, Nov 26, 2021 at 4:50 AM Richard Damon  wrote:
>
> On 11/25/21 12:21 PM, Chris Angelico wrote:
> > On Fri, Nov 26, 2021 at 4:18 AM Ulli Horlacher
> >  wrote:
> >> Chris Angelico  wrote:
> >>
> >>> Unfortunately, if you're not going to go to the effort of getting your
> >>> executables signed
> >> I cannot sign my executables (how can I do it anyway?), because Windows
> >> deletes my executable as soon as I have compiled them! They exist only
> >> for a few seconds and then they are gone.
> >>
> >>
> >>> another reason to just distribute .py files.
> >> I cannot do that because my users do not have Python installed and they
> >> are not allowed to do it.
> >>
> > Are they really allowed to install your unsigned executables but are
> > not allowed to install Python from a known and trusted source?
> >
> > If there's some bizarre loophole that allows them to run completely
> > untrusted binary code, but not to run legitimate code that can be
> > fetched from a variety of trusted places (including python.org, the
> > Windows store, etc), then I'm afraid you're on your own, and will
> > probably need to play around with the exact loophole to figure out
> > what is going to be permitted.
> >
> > Alternatively, just go find the person who decides what gets
> > installed, and request a Python interpreter to be added to the
> > permitted list. That's probably easier, and it's certainly going to be
> > better long-term.
> >
> > ChrisA
>
> My first guess is it isn't so much what is 'allowed' but what can be
> easily done.
>
> On a somewhat locked down computer, the user does not have admin rights,
> so needs to get 'IT' to run any installers that need admin permissions
> to run.

Can someone confirm that it's still possible to run the Python
installer without admin rights, for a per-user installation? It always
used to be possible, but I haven't checked.

> Likely, just copying an EXE file from an outside source may still be
> against the rules (and needs approval), but some think if they can do it
> and no one complains, it must be ok. On the other hand, they may have
> given approval, knowing the source.

Maybe. I would still consider it unlikely that you can run an EXE from
an arbitrary source, but can't run a trusted installer from a known
source. You're right that admin perms would be harder, but that
shouldn't stop you from installing Python.

Also, obligatory XKCD: https://xkcd.com/1200/

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: pyinstaller wrong classified as Windows virus

[Michael Torrie]

On 11/25/21 9:08 AM, Ulli Horlacher wrote:
> I cannot submit my executables, because the Windows Virus scannners
> deletes them as soon as I compile my program!

Add an exclusion rule to your machine. While this is not an option for
your end users, this will certainly allow you to work on the problem,
submitting the exe to the various virus vendors.

> And I need a Microsoft login to submit a file!
> I do not have such a login.

I sympathize.  But if you want to develop for Windows, you might just
have to get one.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: pyinstaller wrong classified as Windows virus

[Michael Torrie]

On 11/25/21 9:08 AM, Ulli Horlacher wrote:

> I cannot submit my executables, because the Windows Virus scannners
> deletes them as soon as I compile my program!

I forgot to post this link:
https://support.microsoft.com/en-us/windows/add-an-exclusion-to-windows-security-811816c0-4dfd-af4a-47e4-c301afe13b26
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: pyinstaller wrong classified as Windows virus

[Mats Wichmann]

On 11/25/21 11:00, Chris Angelico wrote:


Can someone confirm that it's still possible to run the Python
installer without admin rights, for a per-user installation? It always
used to be possible, but I haven't checked.


You only need admin rights for some special cases.  While Win7 was still 
supported, you needed to run the (included) installer for vcredist of a 
different version than the one that comes with Win7, and that needed 
admin rights (unfortunately, the Python installer didn't prompt for that 
and so the vcredist install failed silently, leaving a broken install if 
you didn't already have it from other means - but that's all in the past 
now).  It's possible you also need it for the Python Launcher, since 
that goes into a "system location" (not sure about that one).  And if 
you asked for an install for "all users" that requires admin rights.

--
https://mail.python.org/mailman/listinfo/python-list

Re: pyinstaller wrong classified as Windows virus

[Chris Angelico]

On Fri, Nov 26, 2021 at 7:53 AM Mats Wichmann  wrote:
>
> On 11/25/21 11:00, Chris Angelico wrote:
>
> > Can someone confirm that it's still possible to run the Python
> > installer without admin rights, for a per-user installation? It always
> > used to be possible, but I haven't checked.
>
> You only need admin rights for some special cases.  While Win7 was still
> supported, you needed to run the (included) installer for vcredist of a
> different version than the one that comes with Win7, and that needed
> admin rights (unfortunately, the Python installer didn't prompt for that
> and so the vcredist install failed silently, leaving a broken install if
> you didn't already have it from other means - but that's all in the past
> now).  It's possible you also need it for the Python Launcher, since
> that goes into a "system location" (not sure about that one).  And if
> you asked for an install for "all users" that requires admin rights.

That's what I thought, yeah. So even on a system you don't own, where
you're not allowed to get admin privileges, it should still be
possible to install Python just fine.

Would be worth testing (if someone has a fresh Windows system around)
to see if it's possible to set up the "double click on .py file to run
it" association without admin privileges. That's the only part that
might be a limitation.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Eventfd with epoll BlockingIOError

[Jen via Python-list]

Thanks very much for your reply.  

I am now getting a single event returned in Python, but it's not the right 
event, as I'll explain below. 

I rearranged the Python code based on your comments:

#!/usr/bin/python3
import sys
import os
import select

print("Inside Python")

event_fd = int(sys.argv[3])

print("Eventfd received by Python")
print(event_fd)

event_write_value = 100

ep = select.epoll(-1)
ep.register(event_fd, select.EPOLLIN | select.EPOLLOUT )

os.set_blocking(event_fd, False)

#__

print("Starting poll loop")

for fd_event in ep.poll():
    print("Python fd_event")
    print(fd_event)
    fd_received = fd_event[0]
    event_received = fd_event[1]

You advised to leave off select.EPOLLOUT from the line ep.register(event_fd, 
select.EPOLLIN | select.EPOLLOUT ) -- which makes sense because I'm not waiting 
for that event -- but without it both processes freeze in the for loop (below 
print("Starting poll loop")) so we never receive an EPOLLIN event.  So I 
included it, and here is the screen output from gdb:

Inside Python
Eventfd received by Python
5
Everything OK in Python
Starting poll loop
Python fd_event
(5, 4)
Writing to Python
5 Received from Python
8 Writing to Python
Failed epoll_wait Bad file descriptor
5 Received from Python
8 Writing to Python
Failed epoll_wait Bad file descriptor
5 Received from Python
-1time taken 0.000629
Failed to close epoll file descriptor
Unlink_shm status: Bad file descriptor
fn() took 0.000717 seconds to execute
[Inferior 1 (process 26718) exited normally]
(gdb) q

The Python fd_event tuple is 5, 4 -- 5 is the correct file descriptor and 4 is 
an EPOLLOUT event, which is not what I want. 

The eventfd is created in C as nonblocking:

int eventfd_initialize() {
  int efd = eventfd(0, EFD_NONBLOCK);
  return efd; }

When C writes it calls epoll_wait:

ssize_t epoll_write(int event_fd, int epoll_fd, struct epoll_event * 
event_struc, int action_code)
{
   int64_t ewbuf[1];
   ewbuf[0] = (int64_t)action_code;
   int maxevents = 1;
   int timeout = -1;

   fprintf(stdout, " Writing to Python \n%d", event_fd);

    write(event_fd, &ewbuf, 8);

    if (epoll_wait(epoll_fd, event_struc, maxevents, timeout) == -1)
    {
    fprintf(stderr, "Failed epoll_wait %s\n", strerror(errno));
    }

    ssize_t rdval = read(event_fd, &ewbuf, 8);   

    fprintf(stdout, " Received from Python \n%ld", rdval);

    return 0;
}

The C side initializes its epoll this way:

int epoll_initialize(int efd, int64_t * output_array)
{
  struct epoll_event ev = {};
  int epoll_fd = epoll_create1(0);

  struct epoll_event * ptr_ev = &ev;
  
  if(epoll_fd == -1)
  {
    fprintf(stderr, "Failed to create epoll file descriptor\n");
    return 1;
  }

  ev.events = EPOLLIN | EPOLLOUT;
  ev.data.fd = efd; //was 0

  if(epoll_ctl(epoll_fd, EPOLL_CTL_ADD, efd, &ev) == -1)
  {
  fprintf(stderr, "Failed to add file descriptor to epoll\n");
  close(epoll_fd);
  return 1;
  }

  output_array[0] = epoll_fd;
  output_array[1] = (int64_t)ptr_ev; //&ev;

  return 0;
}

Technically C is not waiting for an EPOLLIN event, but again without it both 
processes freeze unless either C or Python includes both events.  So that 
appears to be where the problem is. 

The Linux epoll man page says, "epoll_wait waits for I/O events, blocking the 
calling thread if no events are currently available."   
https://man7.org/linux/man-pages/man7/epoll.7.html.  That may be the clue to 
why both processes freeze when I poll on only one event in each one. 

Thanks for any ideas based on this update, and thanks again for your earlier 
reply. 

Jen


-- 
 Sent with Tutanota, the secure & ad-free mailbox. 



Nov 25, 2021, 06:34 by ba...@barrys-emacs.org:

>
>
>
>> On 24 Nov 2021, at 22:42, Jen via Python-list <>> python-list@python.org>> > 
>> wrote:
>>
>> I have a C program that uses fork-execv to run Python 3.10 in a child 
>> process, and I am using eventfd with epoll for IPC between them.  The 
>> eventfd file descriptor is created in C and passed to Python through execv.  
>> Once the Python child process starts I print the file descriptor to verify 
>> that it is correct (it is).  
>>
>> In this scenario C will write to the eventfd at intervals and Python will 
>> read the eventfd and take action based on the value in the eventfd.  But in 
>> the Python while True loop I get "BlockingIOError: [Errno 11] Resource 
>> temporarily unavailable" then with each new read it prints "Failed 
>> epoll_wait Bad file descriptor." 
>>
>> This is the Python code:
>>
>> #!/usr/bin/python3
>> import sys
>> import os
>> import select
>>
>> print("Inside Python")
>>
>> event_fd = int(sys.argv[3])
>>
>>
>> print("Eventfd received by Python")
>> print(event_fd)
>>
>> ep = select.epoll(-1)
>> ep.register(event_fd, select.EPOLLIN | select.EPOLLOUT)
>>
>
> This says tell me if I can read or write to the event_fd.
> write will be allowed until the kernel buffers are full.
>
> Usually you only add EPOLLOUT if you have dat

RE: pyinstaller wrong classified as Windows virus

[Avi Gross via Python-list]

I am not sure what your real problem is, Ulli, but many antivirus programs
can be TEMPORARILY shut off. Not highly recommended, of course, but if you
properly disable it on a newly rebooted system running little, and it still
happens, then something else may be going on.

If one recognizes your code a potentially having a virus, it may be for an
assortment of reasons such as a table it contains to look at position N in
the executable for an exact match with some bit-string. If so, one potential
fix is a slight change in the code that compiles a bit differently like
x=sin(30) or other filler. 

But consider another possibility that your compiler software is compromised
and actually placing something into everything it compiles. Is this
happening to only one set of code?

You can think of other similar experiments based on your setup that may help
you debug and perhaps your problem is something else entirely.


-Original Message-
From: Python-list  On
Behalf Of Ulli Horlacher
Sent: Thursday, November 25, 2021 12:10 PM
To: python-list@python.org
Subject: Re: pyinstaller wrong classified as Windows virus

Chris Angelico  wrote:

> Unfortunately, if you're not going to go to the effort of getting your 
> executables signed

I cannot sign my executables (how can I do it anyway?), because Windows
deletes my executable as soon as I have compiled them! They exist only for a
few seconds and then they are gone.


> another reason to just distribute .py files.

I cannot do that because my users do not have Python installed and they are
not allowed to do it.

-- 
Ullrich Horlacher  Server und Virtualisierung
Rechenzentrum TIK 
Universitaet Stuttgart E-Mail: horlac...@tik.uni-stuttgart.de
Allmandring 30aTel:++49-711-68565868
70569 Stuttgart (Germany)  WWW:http://www.tik.uni-stuttgart.de/
--
https://mail.python.org/mailman/listinfo/python-list

-- 
https://mail.python.org/mailman/listinfo/python-list