Note that specifying an IP address for the location of a <process> in a
STAX job does not mean that the STAF/Process/Complete message will first
be attempted to be sent to the IP address of the STAX service machine.
STAF/Process/Complete messages are always first sent to the host name of
the STAX service machine (that was determined by STAF and provided to the
target process machine) which is why you want to make sure that your DNS
is configured correctly, etc. If that QUEUE request fails with RC 16 and
if the target process machine is running STAF V3.4.8 or later, the process
target machine will then attempt to re-send the STAF/Process/Complete
message to the IP address of the STAX service machine.
Increasing queue sizes won't increase throughput.
You might be able to increase throughput for a STAX job by increasing the
number of physical threads that the STAX service uses by specifying the
NUMTHREADS option when registering the STAX service in the STAF.cfg file.
This is because your STAX job is running lots of tasks in parallel. The
default value for NUMTHREADS is 5 if this option is not specified when
registering the STAX service. For example, to specify to use 30 physical
threads for the STAX service:
SERVICE STAX LIBRARY JSTAF EXECUTE C:\STAF\services\stax\STAX.jar PARMS
"NUMTHREADS 30"
However, you won't be able to control the exact interval at which a remote
STAF service request is executed, particularly if the interval is short,
e.g. 1 second, due to factors such as network speed/traffic, available
memory, etc.
--------------------------------------------------------------
Sharon Lucas
IBM Austin, luc...@us.ibm.com
(512) 286-7313 or Tieline 363-7313
From: Nitzan Zaifman <nitz...@algotec.co.il>
To: Sharon Lucas/Austin/IBM@IBMUS,
Cc: "staf-users@lists.sourceforge.net"
<staf-users@lists.sourceforge.net>
Date: 01/05/2014 12:51 AM
Subject: Re: [staf-users] slow remote commands
Hi, Thanks,
All OSs (currently) are just Windows 2k8r2 machines
I've read the FAQ, and mentioned in my first mail it's not the issue
(besides I'm working with IPs).
Is it possible in increase threads and perhaps queue sizes to increase
remote throughput?
Otherwise, I have to change the design considerably:
The proof-of-concept system is to send multiple commands from various
remote computers at a set interval, for example each second.
The current approach is one STAX system running a <parallel> loop to run
all those commands on remote STAF computers at the correct interval (i.e.
spread the load on all participating "workers" such that the total rate is
once a second, the request can last longer, say a minute, and whenever a
worker is free he's available for another request from STAX).
The thing is, due to the issue described – it's not that the actual
command will take longer, but rather STAX will lack resources as commands
will "seem" to take much longer.
So… another approach is needed.
From: Sharon Lucas [mailto:luc...@us.ibm.com]
Sent: Friday, January 03, 2014 1:03 AM
To: Nitzan Zaifman
Cc: staf-users@lists.sourceforge.net
Subject: Re: [staf-users] slow remote commands
Yes, it will take longer for the STAX service to handle all the incoming
service request completion messages on a STAX job handle's queue when a
STAX job submits lots of STAF service requests simultaneously. When a
STAF service command completes on a remote system, it sends a message to
the STAX service machine that the the command completed and the STAX
service handles that and then the STAX service sends a message to the STAX
Monitor to notify it. So, when there are lots of STAF service messages on
the queue, it will take longer to process if there are lots messages on
the queue. Also, if the STAX Monitor machine isn't the same as the STAX
service machine (e.g. if "local" isn't specified for the STAX Machine in
the STAX Monitor's configuration), then it will take much longer since the
messages from the STAX service machine to the STAX Monitor are being sent
remotely.
However, note that the actual time that it took each STAF service request
to run on the remote system via STAF will be close to the time you
specified for the delay (e.g. 25 milliseconds).
So, checking this type of performance via the STAX Monitor is really
telling you the total time it took between when the STAX Monitor was
notified that a STAF service request was submitted and when the STAX
Monitor is notified that a STAF service request completed -- not the
actual time that it took for a STAF service request to complete on the
remote system.
Note that if your goal is to see how long it takes a remote STAF service
request to complete, then the STAFLoop command can be useful in providing
information on the average time it actually takes a remote STAF service
request to complete (without involving STAX). A remote STAF service
request does take longer than a local STAF service request to complete and
part of that depends on your network speed. For example, in my
environment:
C:\>STAFLoop
Usage: STAFLoop <# loops> <Where> <Service> <Request>
C:\>STAFLoop 1 local DELAY DELAY 5
Total loops: 1
Total time : 0.016
Avg. time : 0.016 <== Local request when delaying 5ms
Loops/sec : 62.5
C:\>STAFLoop 1 local DELAY DELAY 25
Total loops: 1
Total time : 0.032
Avg. time : 0.032 <== Local request when delaying 25ms
Loops/sec : 31.25
C:\>STAFLoop 1 myRemoteSystem DELAY DELAY 5
Total loops: 1
Total time : 2.859
Avg. time : 2.859 <== Remote request when delaying 5ms
Loops/sec : 0.349773
C:\>STAFLoop 1 myRemoteSystem DELAY DELAY 25
Total loops: 1
Total time : 2.875
Avg. time : 2.875 <== Remote request when delaying 25ms
Loops/sec : 0.347826
Also, note that the network speed may also be slower when there is a lot
of network traffic (e.g. a lot of simultaneous requests), as it appears to
be your case.
What operating system is on the systems involved?
Also, did you read FAQ entry "3.1.15 Why is the performance slow when
sending a STAF PING request to a remote machine?" at
http://staf.sourceforge.net/current/STAFFAQ.htm#Slow%20performance%20sending%20a%20STAF%20PING%20request%20to%20a%20remote%20machine
. It describes a host name resolution issue that can effect performance
and how to debug it.
--------------------------------------------------------------
Sharon Lucas
IBM Austin, luc...@us.ibm.com
(512) 286-7313 or Tieline 363-7313
From: Nitzan Zaifman <nitz...@algotec.co.il>
To: "staf-users@lists.sourceforge.net" <
staf-users@lists.sourceforge.net>,
Date: 01/02/2014 08:56 AM
Subject: [staf-users] slow remote commands
Hi list,
(I've checked the FAQ and it doesn't seems to be the resolve issue).
I (will) have a system in which I'm (to) run multiple remote commands
simultaneously.
For the sake of test, I've done "DELAY 2S" for the commands.
On local delays, this indeed takes roughly 2 seconds, while on remote
invocations it can very anywhere between 2 to over a minute to complete.
Attached example stafmonitor screenshot which visualizes the issue.
Any thoughts?
[attachment "slowRemoteSTAF.png" deleted by Sharon Lucas/Austin/IBM]
------------------------------------------------------------------------------
Rapidly troubleshoot problems before they affect your business. Most IT
organizations don't have a clear picture of how application performance
affects their revenue. With AppDynamics, you get 100% visibility into your
Java,.NET, & PHP application. Start your 15-day FREE TRIAL of AppDynamics
Pro!
http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.clktrk
_______________________________________________
staf-users mailing list
staf-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/staf-users
------------------------------------------------------------------------------
Rapidly troubleshoot problems before they affect your business. Most IT
organizations don't have a clear picture of how application performance
affects their revenue. With AppDynamics, you get 100% visibility into your
Java,.NET, & PHP application. Start your 15-day FREE TRIAL of AppDynamics
Pro!
http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.clktrk
_______________________________________________
staf-users mailing list
staf-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/staf-users
------------------------------------------------------------------------------
Rapidly troubleshoot problems before they affect your business. Most IT
organizations don't have a clear picture of how application performance
affects their revenue. With AppDynamics, you get 100% visibility into your
Java,.NET, & PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro!
http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.clktrk
_______________________________________________
staf-users mailing list
staf-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/staf-users
