thank you very match for response , i Will try your advices and i ll
tel you.
Anas . A
CIMUT
Le 29 janv. 2010 à 18:10, Sharon Lucas <luc...@us.ibm.com> a écrit :
Actually in re-reading your note, I think you were saying that the
high CPU usage was occurring when the Python code you wrote (not
STAF/STAX code) is does something with a process's large stdout
file. If so, not enough information was provided. I don't know
what you are trying to to do. Only snippets of code was provided,
etc. This is probably more of a Python coding question than a STAF/
STAX question.
--------------------------------------------------------------
Sharon Lucas
IBM Austin, luc...@us.ibm.com
(512) 286-7313 or Tieline 363-7313
----- Forwarded by Sharon Lucas/Austin/IBM on 01/29/2010 11:06 AM
-----
Sharon Lucas/Austin/i...@ibmus
01/29/2010 10:34 AM
To
anas ameziane <anas.amezi...@gmail.com>
cc
staf-users@lists.sourceforge.net
Subject
Re: [staf-users] Stack Overflow ??
First, you need to understand that if you return a file using the
RETURNSTDOUT (or RETURNSTDERR/RETURNFILE) options on a PROCESS START
request (or a <process> element in a STAX job), the contents of the
file are read and then sent as a string string. If the file is
large, this can take up a lot of memory and can consure a lot of CPU
by the unmarshall process. Sub-section "8.12,2 Start" that talks
about the PROCESS service in the STAF User's Guide at http://staf.sourceforge.net/current/STAFUG.htm#HDRPROCSRV
contains the following information that recommends that you don't
use the RETURNxxx options to return large files and talks about how
you can specify a maximum size for a returned file:
Since the entire contents of returned files are stored in the result
string, if you attempt to return the contents of a very large file,
you may run out of memory so it is not recommended that you use the
RETURNSTDOUT, RETURNSTDERR, or RETURNFILE options to return large
files. To help prevent this problem, you can specify a maximum size
for a file returned by this request by setting the MAXRETURNFILESIZE
operational parameter in the STAF configuration file on the machine
where the process is run, or by setting the STAF/MaxReturnFileSize
variable in the request variable pool of the handle that submitted
the request. The lowest of these two values is used as the maximum
return file size (not including 0 which indicates no limit).
So, for large files created by processes, instead of returning their
contents this way, you may want to use the FS COPY request to copy
the contents to the STAX service machine after the process completes
(if you really need the file to reside on the STAX service machine)
as the FS COPY request copies the files in "chunks" so it doesn't
take up much CPU/memory. Then you could read the local file using
Python via a <script> element if you needed to. Or, it might be
more efficient if you can leave the process stdout file on the
remote machine and if you simply need to check the contents of the
file for some information, you couldrun another process on the
remote machine that does this.
--------------------------------------------------------------
Sharon Lucas
IBM Austin, luc...@us.ibm.com
(512) 286-7313 or Tieline 363-7313
anas ameziane <anas.amezi...@gmail.com>
01/29/2010 08:50 AM
To
staf-users@lists.sourceforge.net
cc
Subject
[staf-users] Stack Overflow ??
Hello to all
I use STAF / STAX to automate some tasks.
I developed a python class that allows me to set the stdout of each
process node in XML.
the problem that it works well, except when stdout is large (7 mega
octects).
java.exe consumes 99.9% of my CPU, and I see no error in the logs
(and staf jvm).
----------------
<script>
proc = Process(procName)
</script>
...
<process name="'copygrep'">
<location>location</location>
<command mode="'shell'">'%s/copygrep.sh'%(dir)</command>
<parms>'%s'%(params)</parms>
<console use="'same'"/>
<stderr mode="'stdout'"/>
<returnstdout/>
</process>
<script>
proc._end([RC,STAXResult[0][1]])
</script>
...
-----------------------------------------------------
<script>
class Process:
def __init__(self,name):
self.name = name
self.stdout = 'NaN'
self.rc ='-1'
...
def _end(self,resultTab):
if resultTab[0] != '' :
self.rc = str(resultTab[0])
if resultTab[1] != '' :
import re
self.stdout = resultTab[1]
...
def getXMLResult(self):
from xml.dom.minidom import Document
xmlDoc = Document()
...
stdout = xmlDoc.createElement("stdout")
stdout.appendChild(xmlDoc.createTextNode(self.stdout))
process.appendChild(stdout)
...
</script>
it is an overflow , a bug or something else ?
any ideas ?
Anas A.
Thanks,
---
---
---
---------------------------------------------------------------------
The Planet: dedicated and managed hosting, cloud storage, colocation
Stay online with enterprise data centers and the best network in the
business
Choose flexible plans and management services without long-term
contracts
Personal 24x7 support from experience hosting pros just a phone call
away.
http://p.sf.net/sfu/theplanet-com_______________________________________________
staf-users mailing list
staf-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/staf-users
---
---
---
---------------------------------------------------------------------
The Planet: dedicated and managed hosting, cloud storage, colocation
Stay online with enterprise data centers and the best network in the
business
Choose flexible plans and management services without long-term
contracts
Personal 24x7 support from experience hosting pros just a phone call
away.
http://p.sf.net/sfu/theplanet-com_______________________________________________
staf-users mailing list
staf-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/staf-users
---
---
---
---------------------------------------------------------------------
The Planet: dedicated and managed hosting, cloud storage, colocation
Stay online with enterprise data centers and the best network in the
business
Choose flexible plans and management services without long-term
contracts
Personal 24x7 support from experience hosting pros just a phone call
away.
http://p.sf.net/sfu/theplanet-com
_______________________________________________
staf-users mailing list
staf-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/staf-users
------------------------------------------------------------------------------
The Planet: dedicated and managed hosting, cloud storage, colocation
Stay online with enterprise data centers and the best network in the business
Choose flexible plans and management services without long-term contracts
Personal 24x7 support from experience hosting pros just a phone call away.
http://p.sf.net/sfu/theplanet-com
_______________________________________________
staf-users mailing list
staf-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/staf-users
