What evidence do you have that the number of threads is what leads to application slowness? You've stated that conclusion as a fact, but modem JVMs are generally pretty efficient at handling threads, and I wouldn't expect them to have issues with a few hundred that are doing occasional I/O. If it really is a problem, the NIO transport could help, but I'd suggest you do more investigation before you conclude that threading really is the issue.
For any performance problem, I'd always start by characterizing which resource is the bottleneck. If the CPU is pegged, then your application code (including libraries) is doing lots of work, or the JVM id doing lots of work (thread context switches or garbage collection, typically), or some combination of the two. At that point you'll want to attach a profiler or a sampler to the process to see where the various threads are spending their time, as well as finding out if you're spending an unreasonable amount of time doing full GCs. If CPU usage is low, something else is your bottleneck: possibly memory swapping, possibly disk throughput, possibly network throughput, possibly slow resources at the other end of a network connection, and you'll have to dig into that. Tim On Jun 30, 2016 10:22 PM, "Zhou, Yuding (Nokia - CN/Chengdu)" < yuding.z...@nokia.com> wrote: Hi, there The application is running in Servcemix contains as a SA. The application has two components : DataCollector and DataConverter. DataCollector uses fileAlternativeMonitor to monitor directory and copy file to DataConverter. DataConverter parse raw file, mostly it is CSV file. The application doesn't use Servicemix or ActiveMQ, but only boot by servicemix. Xbean attached. xbean.xml<http://activemq.2283324.n4.nabble.com/file/n4713535/xbean.xml> Symptom: 1. DataCollector uses fileAlternativeMonitor to monitor directory and copy file to DataConverter. It basically have 1 thread for whole application. 2. DataConverter has around 80 threads, around 75 threads are in WAITING, since no raw data for converting. 3. [Suspicion] ActiveMQ transportation threads amount is huge, around 60 ~ 80, most of them are parking on socketRead0 () method ActiveMQ Transport: tcp:///127.0.0.1:56204" - Thread t@279 java.lang.Thread.State: RUNNABLE at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.read(Unknown Source) at org.apache.activemq.transport.tcp.TcpBufferedInputStream.fill(TcpBufferedInputStream.java:50) at org.apache.activemq.transport.tcp.TcpTransport$2.fill(TcpTransport.java:576) at org.apache.activemq.transport.tcp.TcpBufferedInputStream.read(TcpBufferedInputStream.java:58) at org.apache.activemq.transport.tcp.TcpTransport$2.read(TcpTransport.java:561) at java.io.DataInputStream.readInt(Unknown Source) at org.apache.activemq.openwire.OpenWireFormat.unmarshal(OpenWireFormat.java:269) at org.apache.activemq.transport.tcp.TcpTransport.readCommand(TcpTransport.java:227) at org.apache.activemq.transport.tcp.TcpTransport.doRun(TcpTransport.java:219) at org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:202) at java.lang.Thread.run(Unknown Source) Locked ownable synchronizers: - None 4. [Suspicion] ActiveMQConnection threads amount is huge, about 30~ 40. All of them are WAITING on "ActiveMQConnection[ID:gccs1-52411-1466641430250-2:26] Scheduler" - Thread t@228 java.lang.Thread.State: WAITING at java.lang.Object.wait(Native Method) - waiting on <71db1e> (a java.util.TaskQueue) at java.lang.Object.wait(Object.java:485) at java.util.TimerThread.mainLoop(Unknown Source) at java.util.TimerThread.run(Unknown Source) Locked ownable synchronizers: - None Thread dump and analyze attached thread_sort_running.JPG< http://activemq.2283324.n4.nabble.com/file/n4713535/thread_sort_running.JPG> threaddump-1467175636951.JPG< http://activemq.2283324.n4.nabble.com/file/n4713535/threaddump-1467175636951.JPG > threaddump-1467175636951.tdump< http://activemq.2283324.n4.nabble.com/file/n4713535/threaddump-1467175636951.tdump > I'm new to ActiveMQ and servicemix, please let me know what is the suspicion and any clues available. Appreciate for your support, thanks! Regards Yuding
