Hi Martin. I was thinking of doing something with the handle function, but just this little tweak:
https://github.com/morphex/stp/commit/9910ca8c80e9d150222b680a4967e53f0457b465 made a huge difference in CPU usage. Hundreds of waiting sockets are now using 20-30% of CPU instead of 10x that. So for example making the handle function exit / stop and wait isn't necessary at this point. It also opens up the possibility of sending a noop that is appropriate for the given protocol. I've not done a lot of thread programming before, but yes, locks can be used and will be used if necessary. I wasn't sure what data types were thread safe in Python, and it might be that some variables could be off by 1 or more, if using <= >= checks is an option and that there is no risk of the variable containing "garbage". I think with a simple focus, that the project is aimed at one task, will make it easier to manage even complex matters such as concurrency and threads. -Morten On Wed, Jul 27, 2022 at 11:00 PM Martin Di Paola <martinp.dipa...@gmail.com> wrote: > > On Wed, Jul 27, 2022 at 08:32:31PM +0200, Morten W. Petersen wrote: > >You're thinking of the backlog argument of listen? > > From my understanding, yes, when you set up the "accepter" socket (the > one that you use to listen and accept new connections), you can define > the length of the queue for incoming connections that are not accepted > yet. > > This will be the equivalent of your SimpleQueue which basically puts a > limits on how many incoming connections are "accepted" to do a real job. > > Using skt.listen(N) the incoming connections are put on hold by the OS > while in your implementation are formally accepted but they are not > allowed to do any meaningful work: they are put on the SimpleQueue and > only when they are popped then they will work (send/recv data). > > The difference then between the OS and your impl is minimal. The only > case that I can think is that on the clients' side it may exist a > timeout for the acceptance of the connection so your proxy server will > eagerly accept these connections so no timeout is possible(*) > > On a side note, you implementation is too thread-naive: it uses plain > Python lists, integers and boolean variables which are not thread safe. > It is a matter of time until your server will start behave weird. > > One option is that you use thread-safe objects. I'll encourage to read > about thread-safety in general and then which sync mechanisms Python > offers. > > Another option is to remove the SimpleQueue and the background function > that allows a connection to be "active". > > If you think, the handlers are 99% independent except that you want to > allow only N of them to progress (stablish and forward the connection) > and when a handler finishes, another handler "waiting" is activated, "in > a queue fashion" as you said. > > If you allow me to not have a strict queue discipline here, you can achieve > the same results coordinating the handlers using semaphores. Once again, > take this email as starting point for your own research. > > On a second side note, the use of handlers and threads is inefficient > because while you have N active handlers sending/receiving data, because > you are eagerly accepting new connections you will have much more > handlers created and (if I'm not wrong), each will be a thread. > > A more efficient solution could be > > 1) accept as many connections as you can, saving the socket (not the > handler) in the thread-safe queue. > 2) have N threads in the background popping from the queue a socket and > then doing the send/recv stuff. When the thread is done, the thread > closes the socket and pops another from the queue. > > So the queue length will be the count of accepted connections but in any > moment your proxy will not activate (forward) more than N connections. > > This idea is thread-safe, simpler, efficient and has the queue > discipline (I leave aside the usefulness). > > I encourage you to take time to read about the different things > mentioned as concurrency and thread-related stuff is not easy to > master. > > Thanks, > Martin. > > (*) make your proxy server slow enough and yes, you will get timeouts > anyways. > > > > >Well, STP will accept all connections, but can limit how many of the > >accepted connections that are active at any given time. > > > >So when I bombed it with hundreds of almost simultaneous connections, all > >of them were accepted, but only 25 were actively sending and receiving > data > >at any given time. First come, first served. > > > >Regards, > > > >Morten > > > >On Wed, Jul 27, 2022 at 8:00 PM Chris Angelico <ros...@gmail.com> wrote: > > > >> On Thu, 28 Jul 2022 at 02:15, Morten W. Petersen <morp...@gmail.com> > >> wrote: > >> > > >> > Hi. > >> > > >> > I'd like to share with you a recent project, which is a simple TCP > proxy > >> > that can stand in front of a TCP server of some sort, queueing > requests > >> and > >> > then allowing n number of connections to pass through at a time: > >> > >> How's this different from what the networking subsystem already does? > >> When you listen, you can set a queue length. Can you elaborate? > >> > >> ChrisA > >> -- > >> https://mail.python.org/mailman/listinfo/python-list > >> > > > > > >-- > >I am https://leavingnorway.info > >Videos at https://www.youtube.com/user/TheBlogologue > >Twittering at http://twitter.com/blogologue > >Blogging at http://blogologue.com > >Playing music at https://soundcloud.com/morten-w-petersen > >Also playing music and podcasting here: > >http://www.mixcloud.com/morten-w-petersen/ > >On Google+ here https://plus.google.com/107781930037068750156 > >On Instagram at https://instagram.com/morphexx/ > >-- > > > >I am https://leavingnorway.info > > > >Videos at https://www.youtube.com/user/TheBlogologue > >Twittering at http://twitter.com/blogologue > > > >Blogging at http://blogologue.com > >Playing music at https://soundcloud.com/morten-w-petersen > > > >Also playing music and podcasting here: > >http://www.mixcloud.com/morten-w-petersen/ > > > >On Instagram at https://instagram.com/morphexx/ > >-- > >https://mail.python.org/mailman/listinfo/python-list > -- > https://mail.python.org/mailman/listinfo/python-list > -- I am https://leavingnorway.info Videos at https://www.youtube.com/user/TheBlogologue Twittering at http://twitter.com/blogologue Blogging at http://blogologue.com Playing music at https://soundcloud.com/morten-w-petersen Also playing music and podcasting here: http://www.mixcloud.com/morten-w-petersen/ On Google+ here https://plus.google.com/107781930037068750156 On Instagram at https://instagram.com/morphexx/ -- https://mail.python.org/mailman/listinfo/python-list
