Re: Shell script loop runs out of memory

2012-06-11 Thread Andrey Repin
Greetings, Buchbinder, Barry (NIH/NIAID) [E]! > Just to complete this topic ... > This gets rid of all the fork-execs in the inner loop except > for sleep. Instead of comparing file contents, it uses > the test builtin to compare time stamps. I /never ever/ rely on timestamps, except for casual

RE: Shell script loop runs out of memory

2012-06-10 Thread Buchbinder, Barry (NIH/NIAID) [E]
Just to complete this topic ... This gets rid of all the fork-execs in the inner loop except for sleep. Instead of comparing file contents, it uses the test builtin to compare time stamps. #!/bin/dash FILE_TO_CHECK=/mypath/style.less COMPARE_FILE=/mypath/compare_file.tm

Re: Shell script loop runs out of memory

2012-06-01 Thread Eric Blake
On 05/31/2012 11:42 AM, Jordan wrote: > Hi folks, > > I've written a shell script running under CygWin, the purpose of which is to > monitor a file for changes. If the MD5 hash fails to match the previous hash, > it > will execute a command to process the file. I used a 1-second delay between

Re: Shell script loop runs out of memory

2012-06-01 Thread Eric Blake
On 06/01/2012 03:20 AM, Adam Dinwoodie wrote: > Buchbinder, Barry wrote: >> You might try changing >> [[ condition ]] >> to >> [ condition ] >> Perhaps single brackets use memory differently than double brackets. > > They do: [[ condition ]] is interpreted by the shell; [ condition ] forks

Re: Shell script loop runs out of memory

2012-06-01 Thread Corinna Vinschen
On May 31 17:42, Jordan wrote: > Hi folks, > > I've written a shell script running under CygWin, the purpose of which is to > monitor a file for changes. If the MD5 hash fails to match the previous hash, > it > will execute a command to process the file. I used a 1-second delay between > check

RE: Shell script loop runs out of memory

2012-06-01 Thread Adam Dinwoodie
AZ 9901 wrote: > So some things to avoid while (bash)scripting under Cygwin to limit > BLODA effect : > - | : pipe stdout --> stdin > - $(...) : subshell fork > - `...` : same as before, subshell fork > - [ condition ] : prefer [[ condition ]] construction > - anything else ? By my understanding o

Re: Shell script loop runs out of memory

2012-06-01 Thread AZ 9901
2012/6/1 AZ 9901: > 2012/6/1 Adam Dinwoodie: >> Buchbinder, Barry wrote: >>> You might try changing >>>     [[ condition ]] >>> to >>>     [ condition ] >>> Perhaps single brackets use memory differently than double brackets. >> >> They do: [[ condition ]] is interpreted by the shell; [ condition ]

Re: Shell script loop runs out of memory

2012-06-01 Thread AZ 9901
2012/6/1 Adam Dinwoodie: > Buchbinder, Barry wrote: >> You might try changing >>     [[ condition ]] >> to >>     [ condition ] >> Perhaps single brackets use memory differently than double brackets. > > They do: [[ condition ]] is interpreted by the shell; [ condition ] forks to > call /usr/bin/[.

Re: Shell script loop runs out of memory

2012-05-31 Thread wynfield
I will not address the memory management problem, if there is one here. I am addressing your method of determing a difference. No offense, but it seems extremely inefficient, and the larger the file you are computnig the md5sum on the more inefficient. Why not use the file system's "modified"

Re: Shell script loop runs out of memory

2012-05-31 Thread Jordan
Jordan yahoo.com> writes: > This works great for several hours, but then gives an "out > of memory" error and actually brings Windows 7 to its knees. > I can provide the exact memory error if > requested I reproduced it again. The error messages are as follows: ./myscript.sh: line 32: /usr/bi

Re: Shell script loop runs out of memory

2012-05-31 Thread AZ 9901
2012/5/31 Jordan : > AZ 9901 : >> >> Make an infinite loop with no fork, and look at the memory usage. >> Then, make an infinite loop with one fork and look at the memory >> >> I really hope a solution will be found one day >> > > Argh!  And I really like CygWin, so I was hoping to learn that this

Re: Shell script loop runs out of memory

2012-05-31 Thread Jordan
AZ 9901 gmail.com> writes: > > Make an infinite loop with no fork, and look at the memory usage. > Then, make an infinite loop with one fork and look at the memory > > I really hope a solution will be found one day > Argh! And I really like CygWin, so I was hoping to learn that this is reso

Re: Shell script loop runs out of memory

2012-05-31 Thread AZ 9901
2012/5/31 Eliot Moss : >> 1.  Does "fewer forks" mean that some forks are still occurring, thus the >> same >> memory crash will still happen, but not right away?  Just delaying the >> inevitable, for longer than my original script does? > > > I think so, assuming the problem is that forks are not

RE: Shell script loop runs out of memory

2012-05-31 Thread Buchbinder, Barry (NIH/NIAID) [E]
The following are just ideas - totally untested. You might try changing [[ condition ]] to [ condition ] Perhaps single brackets use memory differently than double brackets. If that doesn't work, try changing #!/bin/sh (which calls bash) to #!/bin/dash You will have to have retain

Re: Shell script loop runs out of memory

2012-05-31 Thread Eliot Moss
1. Does "fewer forks" mean that some forks are still occurring, thus the same memory crash will still happen, but not right away? Just delaying the inevitable, for longer than my original script does? I think so, assuming the problem is that forks are not getting fully reclaimed. 2. What is

Re: Shell script loop runs out of memory

2012-05-31 Thread Jordan
Thrall, Bryan flightsafety.com> writes: > > AZ 9901 wrote on 2012-05-31: > > 2012/5/31 Jordan : > > Then, when (bash) scripting under Cygwin, you must take care to avoid > > forking as much as possible. > > > > You could try to improve the "sleep 1" loop with the following one : > > > > while

RE: Shell script loop runs out of memory

2012-05-31 Thread Thrall, Bryan
AZ 9901 wrote on 2012-05-31: > 2012/5/31 Jordan : > Then, when (bash) scripting under Cygwin, you must take care to avoid > forking as much as possible. > > You could try to improve the "sleep 1" loop with the following one : > > while md5sum $FILE_TO_CHECK | cut -d " " -f1 | grep -q "^$MD5PRINT

Re: Shell script loop runs out of memory

2012-05-31 Thread AZ 9901
2012/5/31 Jordan : > I am just wondering why the loops here are consuming increasing amounts of > memory over time?  I'm assigning new MD5 values into existing variables over > and > over, not allocating new variables for each MD5 assignment. (Right??) Is 1 > second perhaps too short a delay... do