Off Topic- Architectural Question..
Hey List. This is off topic, but I noticed the list has been quiet!! Trying to get different thoughts on how to approach an issue. I've got a situation, where I'm going to have a shared nfs, with a bunch of files.. I'm going to have multiple client boxes/(vms) with each client, running apps that access the files on the main nfs. The idea, is that a clientApp, on a given clientVM will access a file on the nfs, and then the nfs file is no long used. The goal is to rip through/process the nfs files as fast as possible, in a simple manner. clientVM1 clientApp1 clientVM2 clientApp2 clientVM3 clientApp3 . . clientVMN clientAppN nfsShare dataFiles... The process can't have the same data file operated on by multiple clientApps. I've thought of having a "pid" file on the nfs, where each clientApp would reset/lock/use the pidFile to then get/use/delete the file as required, but this seems tb slow.. but doable. Surprisingly, in looking around, haven't come up with anything that appears to be really better. (I've also thought of setting up a "webservice" kind of situation, where clients would get the "file/data" from a webservice, and the webservice, would interface with the nfs dir.. but this would be overkill!) Any thoughts/comments? Thanks ___ users mailing list -- users@lists.fedoraproject.org To unsubscribe send an email to users-le...@lists.fedoraproject.org
Re: Off Topic- Architectural Question..
On 10/22/2016 06:13 AM, bruce wrote: The idea, is that a clientApp, on a given clientVM will access a file on the nfs, and then the nfs file is no long used. It's difficult to offer performance tuning advice without more information about what the app is doing, exactly. Is it reading it into memory and then doing some sort of processing on its in-memory copy? The process can't have the same data file operated on by multiple clientApps. Now, that statement makes me think that the application is reading the data, processing it, and then modifying it. If that's the case, then having multiple clients may not be of any benefit. If the applications have to operate in serial, they'll end up doing the work in the same amount of time (or worse) as one system doing all of the work. I've thought of having a "pid" file on the nfs, where each clientApp would reset/lock/use the pidFile to then get/use/delete the file as required, but this seems tb slow.. but doable. Probably. You'd have a bunch of clients periodically checking for the lock file before they do work. You'll lose some time to idling when one application finishes, before another application gets the lock and starts working. You may be better off having one process scheduling work, starting up or signaling an application when the data is available. You'll avoid the complexity of the locking, and possibly reduce idle time. Surprisingly, in looking around, haven't come up with anything that appears to be really better. Your problem is too specific for a general answer. ___ users mailing list -- users@lists.fedoraproject.org To unsubscribe send an email to users-le...@lists.fedoraproject.org
Re: Off Topic- Architectural Question..
On 10/22/2016 12:14 PM, Gordon Messmer wrote: On 10/22/2016 06:13 AM, bruce wrote: I've thought of having a "pid" file on the nfs, where each clientApp would reset/lock/use the pidFile to then get/use/delete the file as required, but this seems tb slow.. but doable. Probably. You'd have a bunch of clients periodically checking for the lock file before they do work. You'll lose some time to idling when one application finishes, before another application gets the lock and starts working. You may be better off having one process scheduling work, starting up or signaling an application when the data is available. You'll avoid the complexity of the locking, and possibly reduce idle time. So, you are suggesting: One process on the nfs server that knows about all the files in question and waits for clients to ask for a file (any file), returns back to that nfs client an nfs handle for some specific file chosen by the server, creates some flag somewhere (on the server) that the file has been served to username xyz, uid cde, on client a.b.c. When is finished with that file, does something like ask the server to delete that file, or says to the server I am done with that file; at which point server might delete the file per client request, or simply remove the flag that the file is in use by username xyz, uid cde, on client a.b.c, ... and so on. Well, now you have another wait problem for each request for some unserved file. In such a serial service of requests - clients might time out, and can err out or simply retry. What if the list of files is in the thousands? Serially serving said files would be very time consuming. Parallel server processes: The server process might be coded so that for each request, it forks a child process to serve the request. Now, child processes have to compete for a lock on the list of files to be served, one of the children will succeed, and set the flag (mark username xyz, uid cde, on client a.b.c has file /filename/), serve the file, unlock the list and exit. Same client that requested the file must then request the server to delete the file or tell the server it is done with the file - again requiring the child server process (not same child process that served the file, but a new child spawned to handle the request) to lock the list + delete the file or simply remove the flag of the client. On the surface of it, it sounds like this is a slow process. But it is not. A sever process that spawns children that compete for the list is a good way to serve said files. ___ users mailing list -- users@lists.fedoraproject.org To unsubscribe send an email to users-le...@lists.fedoraproject.org
Re: Off Topic- Architectural Question..
On Sat, Oct 22, 2016 at 4:00 PM, jd1008 wrote: > > > On 10/22/2016 12:14 PM, Gordon Messmer wrote: >> >> On 10/22/2016 06:13 AM, bruce wrote: >> >>> I've thought of having a "pid" file on the nfs, where each clientApp >>> would reset/lock/use the pidFile to then get/use/delete the file as >>> required, but this seems tb slow.. but doable. >> >> >> Probably. You'd have a bunch of clients periodically checking for the >> lock file before they do work. You'll lose some time to idling when one >> application finishes, before another application gets the lock and starts >> working. You may be better off having one process scheduling work, starting >> up or signaling an application when the data is available. You'll avoid the >> complexity of the locking, and possibly reduce idle time. > > So, you are suggesting: One process on the nfs server that knows about all > the files in question and waits for clients to ask for a file (any > file), returns back to that nfs client an nfs handle for some specific > file chosen by the server, creates some flag somewhere (on the server) that > the file has been served to username xyz, uid cde, on client a.b.c. When > is finished with that file, does something like ask the server to delete > that file, or says to the server I am done with that file; at which point > server might delete the file per client request, or simply remove the flag > that the file is in use by username xyz, uid cde, on client a.b.c, ... and > so on. > Well, now you have another wait problem for each request for some unserved > file. In such a serial service of requests - clients might time out, and can > err out or simply retry. > What if the list of files is in the thousands? Serially serving said files > would be very time consuming. > > Parallel server processes: > The server process might be coded so that for each request, it forks a child > process to serve the request. > Now, child processes have to compete for a lock on the list of files to be > served, one of the children will succeed, and > set the flag (mark username xyz, uid cde, on client a.b.c has file > /filename/), serve the file, unlock the list and exit. Same client that > requested the file must then request the server to delete the file or tell > the server it is done with the file - again requiring the child server > process (not same child process that served the file, but a new child > spawned to handle the request) to lock the list + delete the file or simply > remove the flag of the client. > > On the surface of it, it sounds like this is a slow process. But it is not. > > A sever process that spawns children that compete for the list is a good way > to serve said files. - Hey JD! Thanks for the reply. I thought about having some sort of "nfs/server" side process that iterates through the list of files, and then determines when a "client" is ready, and pushes the file to the client. In having the clients, pull the file (set the pidFILE), there's the possibility that the pidFile, could be messed up, along with a few other minor issues, but it would be simple to implement. Thanks -b > > ___ > users mailing list -- users@lists.fedoraproject.org > To unsubscribe send an email to users-le...@lists.fedoraproject.org ___ users mailing list -- users@lists.fedoraproject.org To unsubscribe send an email to users-le...@lists.fedoraproject.org
Re: Claws-Mail
On Mon, 10 Oct 2016 10:35:01 -0700, stan wrote: > > After last update of Claws-Mail reading new mails, something is > > changed. I read first mail, after reading I move it into trash, > > cursor jumps to last unread mail. Before update worked, after moved > > trash, cursor went next unread mail. > > > > Anyone else noticed? > > Yes, on one system (rawhide, future f26). But it isn't happening on > this system. I think this is happening because a fix was issued for a > bug that was in 3.13. It was related to the next_on_delete setting in > clawsrc. If set to sort_descending, it would move up instead of down > in the inbox when a message was deleted. It's mentioned in the release notes, also in the Fedora Update system ticket. Plus, there is the "clawsker" package that includes a tool you can use to toggle various hidden options available in Claws Mail. ___ users mailing list -- users@lists.fedoraproject.org To unsubscribe send an email to users-le...@lists.fedoraproject.org
Re: What Does Gnome-looks.org Look for to Find Gnome?
On 22/10/16 14:04, Samuel Sieb wrote: On 10/21/2016 06:16 PM, Stephen Morris wrote: Thanks Sam, I think I have that package installed so I'll have to find where it installed the plugin to configure Firefox to use it as I'm using the nightly version of upstream Firefox which at the moment is 52.0a1. /usr/lib64/mozilla/plugins/libgnome-shell-browser-plugin.so Thanks Sam, I'll put a link to that in /usr/lib/mozilla/plugins. From what I've been able to determine upstream Firefox uses /usr/lib for its directories for both the 32bit and 64bit versions. regards, Steve ___ users mailing list -- users@lists.fedoraproject.org To unsubscribe send an email to users-le...@lists.fedoraproject.org ___ users mailing list -- users@lists.fedoraproject.org To unsubscribe send an email to users-le...@lists.fedoraproject.org
Re: What Does Gnome-looks.org Look for to Find Gnome?
On 22/10/16 14:04, Samuel Sieb wrote: On 10/21/2016 06:16 PM, Stephen Morris wrote: Thanks Sam, I think I have that package installed so I'll have to find where it installed the plugin to configure Firefox to use it as I'm using the nightly version of upstream Firefox which at the moment is 52.0a1. /usr/lib64/mozilla/plugins/libgnome-shell-browser-plugin.so I've checked the both the /usr/lib64 and /usr/lib paths and firefox is only finding one of the plugins again, so I'll need to work out again why its only picking up the flashplayer plugin via the link. regards, Steve ___ users mailing list -- users@lists.fedoraproject.org To unsubscribe send an email to users-le...@lists.fedoraproject.org ___ users mailing list -- users@lists.fedoraproject.org To unsubscribe send an email to users-le...@lists.fedoraproject.org
Re: Off Topic- Architectural Question..
On 10/22/2016 02:39 PM, bruce wrote: On Sat, Oct 22, 2016 at 4:00 PM, jd1008 wrote: On 10/22/2016 12:14 PM, Gordon Messmer wrote: On 10/22/2016 06:13 AM, bruce wrote: I've thought of having a "pid" file on the nfs, where each clientApp would reset/lock/use the pidFile to then get/use/delete the file as required, but this seems tb slow.. but doable. Probably. You'd have a bunch of clients periodically checking for the lock file before they do work. You'll lose some time to idling when one application finishes, before another application gets the lock and starts working. You may be better off having one process scheduling work, starting up or signaling an application when the data is available. You'll avoid the complexity of the locking, and possibly reduce idle time. So, you are suggesting: One process on the nfs server that knows about all the files in question and waits for clients to ask for a file (any file), returns back to that nfs client an nfs handle for some specific file chosen by the server, creates some flag somewhere (on the server) that the file has been served to username xyz, uid cde, on client a.b.c. When is finished with that file, does something like ask the server to delete that file, or says to the server I am done with that file; at which point server might delete the file per client request, or simply remove the flag that the file is in use by username xyz, uid cde, on client a.b.c, ... and so on. Well, now you have another wait problem for each request for some unserved file. In such a serial service of requests - clients might time out, and can err out or simply retry. What if the list of files is in the thousands? Serially serving said files would be very time consuming. Parallel server processes: The server process might be coded so that for each request, it forks a child process to serve the request. Now, child processes have to compete for a lock on the list of files to be served, one of the children will succeed, and set the flag (mark username xyz, uid cde, on client a.b.c has file /filename/), serve the file, unlock the list and exit. Same client that requested the file must then request the server to delete the file or tell the server it is done with the file - again requiring the child server process (not same child process that served the file, but a new child spawned to handle the request) to lock the list + delete the file or simply remove the flag of the client. On the surface of it, it sounds like this is a slow process. But it is not. A sever process that spawns children that compete for the list is a good way to serve said files. - Hey JD! Thanks for the reply. I thought about having some sort of "nfs/server" side process that iterates through the list of files, and then determines when a "client" is ready, and pushes the file to the client. In having the clients, pull the file (set the pidFILE), there's the possibility that the pidFile, could be messed up, along with a few other minor issues, but it would be simple to implement. Thanks -b Hi Bruce, Client's .pid files are not shared. They only exist on the client. So, client A would inevitable end up processing the same file as client B. I suggest you write a small daemon on the nfs server that does what I suggest. The "list" is simply an array of strings (array of pointers to null terminated character strings.). This array should be in an include file that also exists on the client that will build the client application code. Once built it can be installed on all clients. the server process will also include this file. The server process should 1. setup the socket for listening to requests. 2. loop listen for incoming requests accept the request (which contains data of the nature of the request) spawn a thread and pass to it the acceptance socket (the data of which the thread will process). endloop 3. The thread will parse the request to see if it is a request for a file or a request to close a file or a request to delete a file. The thread will use sychronization primitives to lock the list mentioned above and perform the request. If the request is for a file, then it needs to open that file for reading, and return an nfs handle to the client, do some overhead, like create the flag of client info (username, uid, client ip), then unlock the list and return. The parent process will not wait for the child thread, as it must simply accept connections and spawn the same thread to handle the request. Multiply (I do not mean mathematical 'multiply', I mean more than one), spawned threads are independent of each other. They only share the the list and the list's lock primitives. The client will use that handle as a file descriptor and process the file. When client is done, client will request to a: close the file or b: close and delete the file. That's it. ___ users mailing list -- users@list
