RFR: 8299518: HotSpotVirtualMachine shared code across different platforms

2023-01-03 Thread Yi Yang
harmless refactor to share code across different platforms of VirtualMachineImpl: 1. Shared code to process command response after requesting a command execution 2. Read functionality in SocketInputStream can be reused - Commit messages: - update year - 8299518: HotSpotVirtualMachi

Re: RFR: 8299518: HotSpotVirtualMachine shared code across different platforms

2023-01-03 Thread Yi Yang
On Wed, 4 Jan 2023 02:05:38 GMT, David Holmes wrote: >> harmless refactor to share code across different platforms of >> VirtualMachineImpl: >> 1. Shared code to process command response after requesting a command >> execution >> 2. Read functionality in SocketInputStream can be reused > > src/

Re: RFR: 8299518: HotSpotVirtualMachine shared code across different platforms [v2]

2023-01-04 Thread Yi Yang
> harmless refactor to share code across different platforms of > VirtualMachineImpl: > 1. Shared code to process command response after requesting a command > execution > 2. Read functionality in SocketInputStream can be reused Yi Yang has refreshed the contents of this pu

Re: RFR: 8299518: HotSpotVirtualMachine shared code across different platforms [v2]

2023-01-04 Thread Yi Yang
On Wed, 4 Jan 2023 04:20:11 GMT, David Holmes wrote: > In that case the `int pid` determined by the superclass should be made > available to the subclasses so they can use it and not need to re-parse the > `vmid`. Hi @dholmes-ora, this is feasible, but in fact, the super class of VirtualMachi

Re: RFR: 8299518: HotSpotVirtualMachine shared code across different platforms [v2]

2023-01-04 Thread Yi Yang
On Thu, 5 Jan 2023 02:40:22 GMT, David Holmes wrote: >> src/jdk.attach/linux/classes/sun/tools/attach/VirtualMachineImpl.java line >> 195: >> >>> 193: * InputStream for the socket connection to get target VM >>> 194: */ >>> 195: private static class SocketInputStreamImpl extends

Re: RFR: 8299518: HotSpotVirtualMachine shared code across different platforms [v2]

2023-01-04 Thread Yi Yang
On Thu, 5 Jan 2023 02:36:59 GMT, David Holmes wrote: >> Yi Yang has refreshed the contents of this pull request, and previous >> commits have been removed. The incremental views will show differences >> compared to the previous content of the PR. The pull request contains t

Re: RFR: 8299518: HotSpotVirtualMachine shared code across different platforms [v3]

2023-01-04 Thread Yi Yang
> harmless refactor to share code across different platforms of > VirtualMachineImpl: > 1. Shared code to process command response after requesting a command > execution > 2. Read functionality in SocketInputStream can be reused Yi Yang has updated the pull request increme

Re: RFR: 8299518: HotSpotVirtualMachine shared code across different platforms [v3]

2023-01-04 Thread Yi Yang
On Thu, 5 Jan 2023 03:06:20 GMT, David Holmes wrote: >> All Posix OS platforms call the same VirtualMachineImpl.read/write/etc, but >> Windows is an unusual guy, it calls VirtualMachineImpl.readPipe/writePipe/etc > > That is a shame, though perhaps we could just rename those methods on Windows?

Re: RFR: 8299518: HotSpotVirtualMachine shared code across different platforms [v3]

2023-01-05 Thread Yi Yang
On Thu, 5 Jan 2023 05:44:23 GMT, David Holmes wrote: >> Good suggestion. >> However, I wonder why the `fd` is casted from `long` to `int` on Unix. The >> Unix versions of SocketInputStream had `int fd`. >> So, the following code also needs to cast `fd` to `int`: >> >> +public synchroniz

Re: RFR: 8299518: HotSpotVirtualMachine shared code across different platforms [v3]

2023-01-05 Thread Yi Yang
On Fri, 6 Jan 2023 05:50:05 GMT, David Holmes wrote: > As long as all the platform-specific `VirtualMachineImpl` classes define a > `read` and `close` method then we can simply have the shared socket stream > class call those methods. Windows will need an extra level of indirection > because i

Re: RFR: 8299518: HotSpotVirtualMachine shared code across different platforms [v3]

2023-01-08 Thread Yi Yang
On Fri, 6 Jan 2023 22:01:45 GMT, Chris Plummer wrote: > I would also like to request that the variable renames be omitted from this > PR. They create too much noise and would best left to a PR that focuses on > just the renames and possibly other style changes. Okay, reverted. -

Re: RFR: 8299518: HotSpotVirtualMachine shared code across different platforms [v4]

2023-01-08 Thread Yi Yang
> harmless refactor to share code across different platforms of > VirtualMachineImpl: > 1. Shared code to process command response after requesting a command > execution > 2. Read functionality in SocketInputStream can be reused Yi Yang has updated the pull request increme

Re: RFR: 8299518: HotSpotVirtualMachine shared code across different platforms [v4]

2023-01-08 Thread Yi Yang
On Thu, 5 Jan 2023 04:28:46 GMT, David Holmes wrote: > I was thinking more about describing what the parameters are - in particular > it is unclear why the actual IOE can be replaced by a passed in one. If we > wanted a custom message then I would at least expect to chain the actual IOE > to t

Re: RFR: 8299518: HotSpotVirtualMachine shared code across different platforms [v5]

2023-01-09 Thread Yi Yang
> harmless refactor to share code across different platforms of > VirtualMachineImpl: > 1. Shared code to process command response after requesting a command > execution > 2. Read functionality in SocketInputStream can be reused Yi Yang has updated the pull request increme

Re: RFR: 8299518: HotSpotVirtualMachine shared code across different platforms [v6]

2023-01-10 Thread Yi Yang
> harmless refactor to share code across different platforms of > VirtualMachineImpl: > 1. Shared code to process command response after requesting a command > execution > 2. Read functionality in SocketInputStream can be reused Yi Yang has updated the pull request increment

Re: RFC: regarding metaspace(metadata?) dump

2023-01-11 Thread Yi Yang
d VM.classes, etc. The Serviceability Agent can also be used to analyze the contents of the class metadata. Dd you look at the existing tools and see how they match up with your requirements? I'd be interested in seeing your implementation and compare it with the existing tools. On 1/11/2023

Re: RFR: 8299518: HotSpotVirtualMachine shared code across different platforms [v6]

2023-02-14 Thread Yi Yang
On Wed, 11 Jan 2023 19:13:20 GMT, Chris Plummer wrote: >> Yi Yang has updated the pull request incrementally with three additional >> commits since the last revision: >> >> - format >> - Merge branch 'jdk_virtualmachienimpl' of github.com:y1yang0/jdk

Re: RFR: 8299518: HotSpotVirtualMachine shared code across different platforms [v6]

2023-02-26 Thread Yi Yang
On Tue, 10 Jan 2023 10:49:52 GMT, Yi Yang wrote: >> harmless refactor to share code across different platforms of >> VirtualMachineImpl: >> 1. Shared code to process command response after requesting a command >> execution >> 2. Read functionality in SocketInputSt

Re: RFR: 8299518: HotSpotVirtualMachine shared code across different platforms [v7]

2023-02-27 Thread Yi Yang
> harmless refactor to share code across different platforms of > VirtualMachineImpl: > 1. Shared code to process command response after requesting a command > execution > 2. Read functionality in SocketInputStream can be reused Yi Yang has updated the pull request increme

Re: RFR: 8299518: HotSpotVirtualMachine shared code across different platforms [v2]

2023-03-02 Thread Yi Yang
On Thu, 5 Jan 2023 03:01:22 GMT, Serguei Spitsyn wrote: >> Yi Yang has refreshed the contents of this pull request, and previous >> commits have been removed. The incremental views will show differences >> compared to the previous content of the PR. The pull request co

Re: RFR: 8299518: HotSpotVirtualMachine shared code across different platforms [v8]

2023-03-02 Thread Yi Yang
> harmless refactor to share code across different platforms of > VirtualMachineImpl: > 1. Shared code to process command response after requesting a command > execution > 2. Read functionality in SocketInputStream can be reused Yi Yang has updated the pull request increme

Re: RFR: 8303151: DCmd framework cleanups

2023-03-02 Thread Yi Yang
On Fri, 3 Mar 2023 04:59:44 GMT, David Holmes wrote: > Whilst working on the DCmd code I noticed two items that could be cleaned up: > > 1. The `NMTDCmd` is registered after the call to `register_dcmds()` instead > of inside it. > > 2. The "extension" mechanism to define external DCmds (as add

Integrated: 8299518: HotSpotVirtualMachine shared code across different platforms

2023-03-05 Thread Yi Yang
On Tue, 3 Jan 2023 09:34:55 GMT, Yi Yang wrote: > harmless refactor to share code across different platforms of > VirtualMachineImpl: > 1. Shared code to process command response after requesting a command > execution > 2. Read functionality in SocketInputStream can be re

Re: RFR: 8303151: DCmd framework cleanups [v3]

2023-03-05 Thread Yi Yang
The message from this sender included one or more files which could not be scanned for virus detection; do not open these files unless you are certain of the sender's intent. -- On Sun, 5 Mar 2023 05:46:37 GMT, David Holmes wrote

RFR: JDK-8306441: Segmented heap dump

2023-04-26 Thread Yi Yang
Hi, heap dump brings about pauses for application's execution(STW), this is a well-known pain. JDK-8252842 have added parallel support to heapdump in an attempt to alleviate this issue. However, all concurrent threads competitively write heap data to the same file, and more memory is required to

Re: RFR: JDK-8306441: Segmented heap dump [v2]

2023-04-26 Thread Yi Yang
g forward to > hearing comments and discussions about this solution. > > - Client parser support for segmented heap dump > This patch provides a possibility that whether heap dump needs to be complete > or not, can the VM directly generate segmented heapdump, and let the cl

Re: RFR: JDK-8306441: Segmented heap dump [v3]

2023-04-28 Thread Yi Yang
g forward to > hearing comments and discussions about this solution. > > - Client parser support for segmented heap dump > This patch provides a possibility that whether heap dump needs to be complete > or not, can the VM directly generate segmented heapdump, and let the client > p

Re: RFR: JDK-8306441: Segmented heap dump [v4]

2023-05-04 Thread Yi Yang
g forward to > hearing comments and discussions about this solution. > > - Client parser support for segmented heap dump > This patch provides a possibility that whether heap dump needs to be complete > or not, can the VM directly generate segmented heapdump, and let the client >

Re: RFR: JDK-8306441: Segmented heap dump [v4]

2023-05-07 Thread Yi Yang
On Thu, 4 May 2023 08:40:10 GMT, Yi Yang wrote: >> Hi, heap dump brings about pauses for application's execution(STW), this is >> a well-known pain. JDK-8252842 have added parallel support to heapdump in an >> attempt to alleviate this issue. However, all concurrent th

Re: RFR: JDK-8306441: Segmented heap dump [v5]

2023-05-10 Thread Yi Yang
g forward to > hearing comments and discussions about this solution. > > - Client parser support for segmented heap dump > This patch provides a possibility that whether heap dump needs to be complete > or not, can the VM directly generate segmented heapdump, and let the client >

Re: RFR: JDK-8306441: Segmented heap dump [v5]

2023-05-14 Thread Yi Yang
On Wed, 10 May 2023 08:29:43 GMT, Yi Yang wrote: >> Hi, heap dump brings about pauses for application's execution(STW), this is >> a well-known pain. JDK-8252842 have added parallel support to heapdump in an >> attempt to alleviate this issue. However, all concurrent th

Re: RFR: JDK-8306441: Segmented heap dump [v5]

2023-05-17 Thread Yi Yang
On Mon, 15 May 2023 02:16:43 GMT, Yi Yang wrote: >> Yi Yang has updated the pull request incrementally with one additional >> commit since the last revision: >> >> execute VM_HeapDumper directly > > Hi, can I have a review for this patch? > @y1yang0 Sorry no

Re: RFR: JDK-8306441: Segmented heap dump [v6]

2023-05-17 Thread Yi Yang
l > time for heapdump(See table above). However, considering the reduction of STW > time, I think it is an acceptable trade-off. Furthermore, there is still room > for optimization in the second merge stage(e.g. > sendfile/splice/copy_file_range instead of read+write combina

Re: RFR: JDK-8306441: Enhance parallel heap dump [v6]

2023-05-18 Thread Yi Yang
On Thu, 18 May 2023 09:35:36 GMT, Kevin Walls wrote: > I'm interested in if it is faster in STW time with the parallel write to use > compression or not? (Does not compressing take time, or does compressing mean > we write fewer bytes, so ends up being faster.) Hi @kevinjwalls, so far all benc

Re: RFR: JDK-8306441: Enhance parallel heap dump [v6]

2023-05-21 Thread Yi Yang
On Thu, 18 May 2023 11:02:21 GMT, Yi Yang wrote: > I'm interested in if it is faster in STW time with the parallel write to use > compression or not? (Does not compressing take time, or does compressing mean > we write fewer bytes, so ends up being faster.) I updated the bench

Re: RFR: JDK-8306441: Enhance parallel heap dump [v6]

2023-06-05 Thread Yi Yang
On Mon, 5 Jun 2023 23:44:04 GMT, Alex Menkov wrote: > A lot of code movement is caused by moving AbstractDumpWriter class from > heapDumper.cpp to heapDumpCompression.hpp/cpp > I'm not happy with huge heapDumper.cpp file, but this refactoring does not > look good to me. > Currently all logic ab

Re: RFR: JDK-8306441: Two-stage Segmented Heap Dump [v7]

2023-06-05 Thread Yi Yang
1 secs | 9.885 secs | Compress1 | > | 32g | 96 thread | 38.9900931 secs | 80.574 secs | Compress2 | > | 64g | 1 thread | 100.583 secs | 100.583 secs | N | > | 64g | 32 thread | 20.9233744 secs | 134.701 secs | N | > | 64g | 32 thread | 18.5023784 secs | 19.358 secs | Compress1 | > | 64g |

Re: RFR: JDK-8306441: Two-stage Segmented Heap Dump [v8]

2023-06-05 Thread Yi Yang
1 secs | 9.885 secs | Compress1 | > | 32g | 96 thread | 38.9900931 secs | 80.574 secs | Compress2 | > | 64g | 1 thread | 100.583 secs | 100.583 secs | N | > | 64g | 32 thread | 20.9233744 secs | 134.701 secs | N | > | 64g | 32 thread | 18.5023784 secs | 19.358 secs | Compress1 | > | 64g |

Re: RFR: JDK-8306441: Two-stage Segmented Heap Dump [v5]

2023-06-11 Thread Yi Yang
On Tue, 16 May 2023 18:41:26 GMT, Chris Plummer wrote: >> Hi, can I have a review for this patch? > > @y1yang0 Sorry no one has been able to review this so far. The serviceability > team is very busy for the next few weeks finishing up JDK 21 changes before > RDP1. It's unlikely we'll find time

Re: RFR: JDK-8306441: Two-stage Segmented Heap Dump [v9]

2023-06-13 Thread Yi Yang
| N | > | 32g | 32 thread | 10.7734677 secs | 61.643 secs | N | > | 32g | 32 thread | 10.1642097 secs | 10.903 secs | Compress1 | > | 32g | 32 thread | 43.8407607 secs | 88.152 secs | Compress2 | > | 32g | 96 thread | 13.1522042 secs | 61.432 secs | N | > | 32g | 96 thread | 9.09546

Re: RFR: JDK-8306441: Two-stage Segmented Heap Dump [v10]

2023-06-13 Thread Yi Yang
| N | > | 32g | 32 thread | 10.7734677 secs | 61.643 secs | N | > | 32g | 32 thread | 10.1642097 secs | 10.903 secs | Compress1 | > | 32g | 32 thread | 43.8407607 secs | 88.152 secs | Compress2 | > | 32g | 96 thread | 13.1522042 secs | 61.432 secs | N | > | 32g | 96 thread | 9

Re: RFR: JDK-8306441: Two-stage Segmented Heap Dump [v10]

2023-06-18 Thread Yi Yang
On Sat, 17 Jun 2023 01:22:54 GMT, Alex Menkov wrote: > This change adds new option. most likely this requires CSR Hi @alexmenkov, it seems that adding new parameters to jcmd does not require a CSR, only changes like those to jmap do. - PR Review Comment: https://git.openjdk.org/jd

Re: RFR: JDK-8306441: Two-stage Segmented Heap Dump [v10]

2023-06-19 Thread Yi Yang
On Sat, 17 Jun 2023 01:20:21 GMT, Alex Menkov wrote: >> Yi Yang has updated the pull request incrementally with one additional >> commit since the last revision: >> >> whitespace > > Could you please add info about what testing have been done Hi @alexmenkov

Re: RFR: JDK-8306441: Two-stage Segmented Heap Dump [v11]

2023-06-19 Thread Yi Yang
| N | > | 32g | 32 thread | 10.7734677 secs | 61.643 secs | N | > | 32g | 32 thread | 10.1642097 secs | 10.903 secs | Compress1 | > | 32g | 32 thread | 43.8407607 secs | 88.152 secs | Compress2 | > | 32g | 96 thread | 13.1522042 secs | 61.432 secs | N | > | 32g | 96 thread | 9.0954641

Re: RFR: JDK-8306441: Two-stage Segmented Heap Dump [v12]

2023-06-19 Thread Yi Yang
| N | > | 32g | 32 thread | 10.7734677 secs | 61.643 secs | N | > | 32g | 32 thread | 10.1642097 secs | 10.903 secs | Compress1 | > | 32g | 32 thread | 43.8407607 secs | 88.152 secs | Compress2 | > | 32g | 96 thread | 13.1522042 secs | 61.432 secs | N | > | 32g | 96 thread | 9.0954641 sec

Re: RFR: JDK-8306441: Two-stage Segmented Heap Dump [v13]

2023-06-19 Thread Yi Yang
| N | > | 32g | 32 thread | 10.7734677 secs | 61.643 secs | N | > | 32g | 32 thread | 10.1642097 secs | 10.903 secs | Compress1 | > | 32g | 32 thread | 43.8407607 secs | 88.152 secs | Compress2 | > | 32g | 96 thread | 13.1522042 secs | 61.432 secs | N | > | 32g | 96 thread | 9.0954641 s

Re: RFR: JDK-8306441: Two-stage Segmented Heap Dump [v14]

2023-06-28 Thread Yi Yang
| N | > | 32g | 32 thread | 10.7734677 secs | 61.643 secs | N | > | 32g | 32 thread | 10.1642097 secs | 10.903 secs | Compress1 | > | 32g | 32 thread | 43.8407607 secs | 88.152 secs | Compress2 | > | 32g | 96 thread | 13.1522042 secs | 61.432 secs | N | > | 32g | 96 thread | 9.

Re: RFR: JDK-8306441: Two-stage Segmented Heap Dump [v5]

2023-06-28 Thread Yi Yang
On Tue, 16 May 2023 18:41:26 GMT, Chris Plummer wrote: >> Hi, can I have a review for this patch? > > @y1yang0 Sorry no one has been able to review this so far. The serviceability > team is very busy for the next few weeks finishing up JDK 21 changes before > RDP1. It's unlikely we'll find time

Re: RFR: JDK-8306441: Two-stage Segmented Heap Dump [v14]

2023-07-02 Thread Yi Yang
On Fri, 30 Jun 2023 22:23:19 GMT, Alex Menkov wrote: >> Yi Yang has updated the pull request incrementally with one additional >> commit since the last revision: >> >> memory leak > > I think commit "use HandshakeClosure instead of VMOperation" is a w

Re: RFR: JDK-8306441: Two-stage Segmented Heap Dump [v15]

2023-07-05 Thread Yi Yang
| N | > | 32g | 32 thread | 10.7734677 secs | 61.643 secs | N | > | 32g | 32 thread | 10.1642097 secs | 10.903 secs | Compress1 | > | 32g | 32 thread | 43.8407607 secs | 88.152 secs | Compress2 | > | 32g | 96 thread | 13.1522042 secs | 61.432 secs | N | > | 32g | 96 thread | 9.0954641 s

Re: RFR: 8287135: Calculation of jmm_GetMemoryUsage is wrong

2022-06-17 Thread Yi Yang
On Fri, 27 May 2022 07:22:11 GMT, Thomas Stuefe wrote: >>> I think the right fix is to just convert the MetaspacePool into >>> NonClassMetaspacePool and only report the non-class values. >> >> Yes, it's okay for me. But I have another concern. >> >> The compressed class pool is not directly us

Re: RFR: 8287135: Calculation of jmm_GetMemoryUsage is wrong [v4]

2022-06-17 Thread Yi Yang
t; > In this way, getNonHeapMemoryUsage is larger than it ought to be, it should > be `NonHeapUsage = CodeCache + Metaspace`. Yi Yang has updated the pull request incrementally with one additional commit since the last revision: address Ioi's comments; fix LowMemoryTest2.sh fa

Re: RFR: 8287135: Calculation of jmm_GetMemoryUsage is wrong [v3]

2022-06-17 Thread Yi Yang
On Wed, 8 Jun 2022 05:25:28 GMT, Ioi Lam wrote: >> Yi Yang has updated the pull request incrementally with one additional >> commit since the last revision: >> >> update > > src/hotspot/share/services/management.cpp line 743: > >> 741: } else { >&

Re: RFR: 8287135: Calculation of jmm_GetMemoryUsage is wrong [v4]

2022-06-17 Thread Yi Yang
On Fri, 17 Jun 2022 08:00:44 GMT, Thomas Stuefe wrote: >> Yi Yang has updated the pull request incrementally with one additional >> commit since the last revision: >> >> address Ioi's comments; fix LowMemoryTest2.sh failure > > test/jdk/java/lang/manage

Re: RFR: 8287135: Calculation of jmm_GetMemoryUsage is wrong [v4]

2022-06-20 Thread Yi Yang
On Fri, 17 Jun 2022 07:02:47 GMT, Yi Yang wrote: >> It seems that calculation of >> MemoryMXBean.getNonHeapMemoryUsage(jmm_GetMemoryUsage) is wrong. >> >> Currently, >> `NonHeapUsage=CodeCache+Metaspace(ClassTypeSpace+NonClassTypeSpace)+Compr

Re: RFR: 8287135: Calculation of jmm_GetMemoryUsage is wrong [v4]

2022-06-21 Thread Yi Yang
On Tue, 21 Jun 2022 05:23:58 GMT, Thomas Stuefe wrote: > > > I would have thought that since we don't have the pool anymore, we can > > > just remove this test line. The lines above already > > > test against MaxMetaspaceSize. > > > > > > Okay. > > > I think you may be right, we need a replace

Re: RFR: 8287135: Calculation of jmm_GetMemoryUsage is wrong [v4]

2022-06-27 Thread Yi Yang
On Tue, 21 Jun 2022 13:44:23 GMT, Thomas Stuefe wrote: >>> > > I would have thought that since we don't have the pool anymore, we can >>> > > just remove this test line. The lines above already >>> > > test against MaxMetaspaceSize. >>> > >>> > >>> > Okay. >>> > > I think you may be right, we

Re: RFR: 8287135: Calculation of jmm_GetMemoryUsage is wrong [v5]

2022-06-27 Thread Yi Yang
t; > In this way, getNonHeapMemoryUsage is larger than it ought to be, it should > be `NonHeapUsage = CodeCache + Metaspace`. Yi Yang has updated the pull request incrementally with one additional commit since the last revision: address @tstuefe's comments; remove unnecessary declar

Re: RFR: JDK-8306441: Two phase segmented heap dump [v14]

2023-07-09 Thread Yi Yang
On Wed, 5 Jul 2023 21:10:56 GMT, Alex Menkov wrote: >> I think commit "use HandshakeClosure instead of VMOperation" is a wrong way >> to go. >> It restricts use of parallel dumping only to attach case. >> I have pending changes in heap dumper to support virtual threads and I'm >> going switch h

Re: RFR: JDK-8306441: Two phase segmented heap dump [v16]

2023-07-09 Thread Yi Yang
| 80.574 | > | 64g | 1 T | N | 100.583 | 100.583 | > | 64g | 32 T | N | 20.9233744 | 134.701 | > | 64g | 32 T | C1 | 18.5023784 | 19.358 | > | 64g | 32 T | C2 | 86.4748377 | 172.707 | > | 64g | 96 T | N | 26.7374116 | 126.08 | > | 64g | 96 T | C1 | 16.8101551 | 17.938 |

Re: RFR: JDK-8306441: Two phase segmented heap dump [v17]

2023-07-12 Thread Yi Yang
| 80.574 | > | 64g | 1 T | N | 100.583 | 100.583 | > | 64g | 32 T | N | 20.9233744 | 134.701 | > | 64g | 32 T | C1 | 18.5023784 | 19.358 | > | 64g | 32 T | C2 | 86.4748377 | 172.707 | > | 64g | 96 T | N | 26.7374116 | 126.08 | > | 64g | 96 T | C1 | 16.8101551 | 17.938 | > |

Re: RFR: JDK-8306441: Two phase segmented heap dump [v17]

2023-07-16 Thread Yi Yang
On Wed, 12 Jul 2023 07:58:47 GMT, Yi Yang wrote: >> ### Motivation and proposal >> Hi, heap dump brings about pauses for application's execution(STW), this is >> a well-known pain. JDK-8252842 have added parallel support to heapdump in an >> attempt to allev

Re: RFR: JDK-8306441: Two phase segmented heap dump [v17]

2023-07-16 Thread Yi Yang
On Mon, 17 Jul 2023 04:11:54 GMT, David Holmes wrote: > > Because I need the overloaded check is_AttachListener_thread(), which can > > avoid using the VM thread to execute the dump file merge as much as > > possible. > > I'm unclear what the set of candidate threads is for executing the code

Re: RFR: JDK-8306441: Two phase segmented heap dump [v18]

2023-07-17 Thread Yi Yang
| 80.574 | > | 64g | 1 T | N | 100.583 | 100.583 | > | 64g | 32 T | N | 20.9233744 | 134.701 | > | 64g | 32 T | C1 | 18.5023784 | 19.358 | > | 64g | 32 T | C2 | 86.4748377 | 172.707 | > | 64g | 96 T | N | 26.7374116 | 126.08 | > | 64g | 96 T | C1 | 16.8101551 | 17.938 | > | 64

Re: RFR: JDK-8306441: Two phase segmented heap dump [v19]

2023-07-18 Thread Yi Yang
| 80.574 | > | 64g | 1 T | N | 100.583 | 100.583 | > | 64g | 32 T | N | 20.9233744 | 134.701 | > | 64g | 32 T | C1 | 18.5023784 | 19.358 | > | 64g | 32 T | C2 | 86.4748377 | 172.707 | > | 64g | 96 T | N | 26.7374116 | 126.08 | > | 64g | 96 T | C1 | 16.8101551 | 17.938 | > | 64g

Re: RFR: JDK-8306441: Two phase segmented heap dump [v20]

2023-07-19 Thread Yi Yang
| 80.574 | > | 64g | 1 T | N | 100.583 | 100.583 | > | 64g | 32 T | N | 20.9233744 | 134.701 | > | 64g | 32 T | C1 | 18.5023784 | 19.358 | > | 64g | 32 T | C2 | 86.4748377 | 172.707 | > | 64g | 96 T | N | 26.7374116 | 126.08 | > | 64g | 96 T | C1 | 16.8101551 | 17.938 | > | 6

Re: RFR: JDK-8306441: Two phase segmented heap dump [v16]

2023-07-28 Thread Yi Yang
On Tue, 11 Jul 2023 20:49:35 GMT, Chris Plummer wrote: >> Yi Yang has updated the pull request incrementally with two additional >> commits since the last revision: >> >> - whitespace >> - add test > > BTW, `serviceability/dcmd/gc/HeapDumpAllTest.java`

Re: RFR: JDK-8306441: Two phase segmented heap dump [v20]

2023-07-28 Thread Yi Yang
On Fri, 28 Jul 2023 17:18:05 GMT, Kevin Walls wrote: > I think we might be best off leaving the long dump as you have it in the > AttachListener for now. 8-) Consideing a very large heap size, the VM used to pause all threads and spend 1 hour to execute a heap dump, during which time jcmd/jsta

Re: RFR: JDK-8306441: Two phase segmented heap dump [v21]

2023-07-31 Thread Yi Yang
| 80.574 | > | 64g | 1 T | N | 100.583 | 100.583 | > | 64g | 32 T | N | 20.9233744 | 134.701 | > | 64g | 32 T | C1 | 18.5023784 | 19.358 | > | 64g | 32 T | C2 | 86.4748377 | 172.707 | > | 64g | 96 T | N | 26.7374116 | 126.08 | > | 64g | 96 T | C1 | 16.8101551 | 17.938 | > | 6

Re: RFR: JDK-8306441: Two phase segmented heap dump [v20]

2023-07-31 Thread Yi Yang
On Wed, 19 Jul 2023 12:42:27 GMT, Yi Yang wrote: >> ### Motivation and proposal >> Hi, heap dump brings about pauses for application's execution(STW), this is >> a well-known pain. JDK-8252842 have added parallel support to heapdump in an >> attempt to allev

Re: RFR: JDK-8306441: Two phase segmented heap dump [v21]

2023-08-02 Thread Yi Yang
On Tue, 1 Aug 2023 14:41:53 GMT, Kevin Walls wrote: > I just saw a run with no -parallel option which got num_active_workers = 2, > so it did a parallel dump, so the default is being set as intended, but the > availability of worker threads is not always predictable! Yes, now I've added new lo

Re: RFR: JDK-8306441: Two phase segmented heap dump [v22]

2023-08-02 Thread Yi Yang
| 80.574 | > | 64g | 1 T | N | 100.583 | 100.583 | > | 64g | 32 T | N | 20.9233744 | 134.701 | > | 64g | 32 T | C1 | 18.5023784 | 19.358 | > | 64g | 32 T | C2 | 86.4748377 | 172.707 | > | 64g | 96 T | N | 26.7374116 | 126.08 | > | 64g | 96 T | C1 | 16.8101551 | 17.938 | > | 64g

Re: RFR: JDK-8306441: Two phase segmented heap dump [v21]

2023-08-02 Thread Yi Yang
On Tue, 1 Aug 2023 17:48:30 GMT, Chris Plummer wrote: >> test/hotspot/jtreg/serviceability/HeapDump/IntegrityHeapDumpTest.java line >> 84: >> >>> 82: .addToolArg(heapDumpFile.getAbsolutePath()); >>> 83: >>> 84: ProcessBuilder processBuilder = new >>> ProcessBuilder(lau

Re: RFR: JDK-8306441: Two phase segmented heap dump [v23]

2023-08-02 Thread Yi Yang
| 80.574 | > | 64g | 1 T | N | 100.583 | 100.583 | > | 64g | 32 T | N | 20.9233744 | 134.701 | > | 64g | 32 T | C1 | 18.5023784 | 19.358 | > | 64g | 32 T | C2 | 86.4748377 | 172.707 | > | 64g | 96 T | N | 26.7374116 | 126.08 | > | 64g | 96 T | C1 | 16.8101551 | 17.938 | > |

Re: RFR: JDK-8306441: Two phase segmented heap dump [v23]

2023-08-02 Thread Yi Yang
On Wed, 2 Aug 2023 11:33:17 GMT, Yi Yang wrote: >> ### Motivation and proposal >> Hi, heap dump brings about pauses for application's execution(STW), this is >> a well-known pain. JDK-8252842 have added parallel support to heapdump in an >> attempt to allev

Re: RFR: JDK-8306441: Two phase segmented heap dump [v24]

2023-08-02 Thread Yi Yang
| 80.574 | > | 64g | 1 T | N | 100.583 | 100.583 | > | 64g | 32 T | N | 20.9233744 | 134.701 | > | 64g | 32 T | C1 | 18.5023784 | 19.358 | > | 64g | 32 T | C2 | 86.4748377 | 172.707 | > | 64g | 96 T | N | 26.7374116 | 126.08 | > | 64g | 96 T | C1 | 16.8101551 | 17.938 |

Re: RFR: JDK-8306441: Two phase segmented heap dump [v25]

2023-08-02 Thread Yi Yang
| 80.574 | > | 64g | 1 T | N | 100.583 | 100.583 | > | 64g | 32 T | N | 20.9233744 | 134.701 | > | 64g | 32 T | C1 | 18.5023784 | 19.358 | > | 64g | 32 T | C2 | 86.4748377 | 172.707 | > | 64g | 96 T | N | 26.7374116 | 126.08 | > | 64g | 96 T | C1 | 16.8101551 | 17.938 |

Re: RFR: JDK-8306441: Two phase segmented heap dump [v23]

2023-08-02 Thread Yi Yang
On Wed, 2 Aug 2023 13:11:51 GMT, Kevin Walls wrote: > > > > Yes, dividing testing by feature so we have HeapDumpTest, > HeapDumpCompressedTest, HeapDumpParallelTest... that sounds good. > > Currently the HeapDumpTest does more parallel testing than > IntegrityHeapDumpTest, so I didn't unders

Re: RFR: JDK-8306441: Two phase segmented heap dump [v21]

2023-08-02 Thread Yi Yang
On Wed, 2 Aug 2023 18:02:22 GMT, Chris Plummer wrote: > so I'm still wondering how this use of SATestUtils.createProcessBuilder() > got in this new test in the first place I simply grep existing tests and found this snippet to create java process. There is no special intention, it is an obvio

Re: RFR: JDK-8306441: Two phase segmented heap dump [v25]

2023-08-03 Thread Yi Yang
On Thu, 3 Aug 2023 13:24:19 GMT, Kevin Walls wrote: > Also the variable num_requested_dump_thread is not needed? It's just a copy > of _num_dumper_threads which we don't change. This is needed because _num_dumper_threads will change later, the requested dump thread may not equal to actual _num

Re: RFR: JDK-8306441: Two phase segmented heap dump [v26]

2023-08-04 Thread Yi Yang
| 80.574 | > | 64g | 1 T | N | 100.583 | 100.583 | > | 64g | 32 T | N | 20.9233744 | 134.701 | > | 64g | 32 T | C1 | 18.5023784 | 19.358 | > | 64g | 32 T | C2 | 86.4748377 | 172.707 | > | 64g | 96 T | N | 26.7374116 | 126.08 | > | 64g | 96 T | C1 | 16.8101551 | 17.938 | > |

Re: RFR: JDK-8306441: Two phase segmented heap dump [v26]

2023-08-04 Thread Yi Yang
On Fri, 4 Aug 2023 16:31:28 GMT, Kevin Walls wrote: >> Yi Yang has updated the pull request incrementally with one additional >> commit since the last revision: >> >> new can_parallel_dump > > test/hotspot/jtreg/serviceability/dcmd/gc/HeapDumpParal

Re: RFR: JDK-8306441: Two phase segmented heap dump [v26]

2023-08-08 Thread Yi Yang
On Tue, 8 Aug 2023 07:41:23 GMT, Kevin Walls wrote: > It's not too bad to change the test to launch a LingeredApp and test the > output, I made an attempt at that. I'll share a testcase suggestion here, see > what you think. Thank you for the share! It makes sense to me. This test will start J

Re: RFR: JDK-8306441: Two phase segmented heap dump [v27]

2023-08-08 Thread Yi Yang
| 80.574 | > | 64g | 1 T | N | 100.583 | 100.583 | > | 64g | 32 T | N | 20.9233744 | 134.701 | > | 64g | 32 T | C1 | 18.5023784 | 19.358 | > | 64g | 32 T | C2 | 86.4748377 | 172.707 | > | 64g | 96 T | N | 26.7374116 | 126.08 | > | 64g | 96 T | C1 | 16.8101551 | 17.938 | > |

Re: RFR: JDK-8306441: Two phase segmented heap dump [v28]

2023-08-08 Thread Yi Yang
| 80.574 | > | 64g | 1 T | N | 100.583 | 100.583 | > | 64g | 32 T | N | 20.9233744 | 134.701 | > | 64g | 32 T | C1 | 18.5023784 | 19.358 | > | 64g | 32 T | C2 | 86.4748377 | 172.707 | > | 64g | 96 T | N | 26.7374116 | 126.08 | > | 64g | 96 T | C1 | 16.8101551 | 17.938 | > | 64

Re: RFR: JDK-8306441: Two phase segmented heap dump [v28]

2023-08-08 Thread Yi Yang
On Tue, 8 Aug 2023 13:09:58 GMT, Kevin Walls wrote: >> Yi Yang has updated the pull request incrementally with one additional >> commit since the last revision: >> >> aware of serialgc and epsilongc > > Great, thanks for your patience with all of this! 8-)

Integrated: JDK-8306441: Two phase segmented heap dump

2023-08-08 Thread Yi Yang
On Wed, 26 Apr 2023 09:37:46 GMT, Yi Yang wrote: > ### Motivation and proposal > Hi, heap dump brings about pauses for application's execution(STW), this is a > well-known pain. JDK-8252842 have added parallel support to heapdump in an > attempt to alleviate this issue. Howeve

RFR: 8311775: [TEST] duplicate verifyHeapDump in several tests

2023-08-09 Thread Yi Yang
This is a follow-up patch of #13667. verifyHeapDump is duplicated in several tests, this patch tries to consolidate them into one method. - Commit messages: - 8311775: [TEST] duplicate verifyHeapDump in several tests Changes: https://git.openjdk.org/jdk/pull/15202/files Webrev: ht

Re: RFR: 8311775: [TEST] duplicate verifyHeapDump in several tests [v2]

2023-08-10 Thread Yi Yang
> This is a follow-up patch of #13667. verifyHeapDump is duplicated in several > tests, this patch tries to consolidate them into one method. Yi Yang has updated the pull request incrementally with one additional commit since the last revision: fix TestHeapDumpForInvokD

Re: RFR: 8311775: [TEST] duplicate verifyHeapDump in several tests [v2]

2023-08-10 Thread Yi Yang
On Thu, 10 Aug 2023 09:41:29 GMT, Yi Yang wrote: >> This is a follow-up patch of #13667. verifyHeapDump is duplicated in several >> tests, this patch tries to consolidate them into one method. > > Yi Yang has updated the pull request incrementally with one additional commit

Re: RFR: 8311775: [TEST] duplicate verifyHeapDump in several tests [v3]

2023-08-10 Thread Yi Yang
> This is a follow-up patch of #13667. verifyHeapDump is duplicated in several > tests, this patch tries to consolidate them into one method. Yi Yang has updated the pull request incrementally with one additional commit since the last revision: review from Alex - C

Re: RFR: 8311775: [TEST] duplicate verifyHeapDump in several tests [v2]

2023-08-10 Thread Yi Yang
On Thu, 10 Aug 2023 19:29:27 GMT, Alex Menkov wrote: >> Yi Yang has updated the pull request incrementally with one additional >> commit since the last revision: >> >> fix TestHeapDumpForInvokDynamic > > test/hotspot/jtreg/serviceability/sa/TestHeapDumpFo

Re: RFR: 8314021: HeapDump: Optimize segmented heap file merging phase

2023-08-23 Thread Yi Yang
On Fri, 11 Aug 2023 09:31:56 GMT, Yi Yang wrote: > This patch reduce ~16%(24s->20s) pahse 2 merge time during dumping 32g heap > with 96threads and fixes a memory leak of compressor > > You might argue why this is Linux-only optimization, because sendfile > requires at

Re: RFR: 8311775: [TEST] duplicate verifyHeapDump in several tests [v4]

2023-08-30 Thread Yi Yang
> This is a follow-up patch of #13667. verifyHeapDump is duplicated in several > tests, this patch tries to consolidate them into one method. Yi Yang has updated the pull request incrementally with one additional commit since the last revision: Update HeapDumpTes

Re: RFR: 8311775: [TEST] duplicate verifyHeapDump in several tests [v5]

2023-08-31 Thread Yi Yang
> This is a follow-up patch of #13667. verifyHeapDump is duplicated in several > tests, this patch tries to consolidate them into one method. Yi Yang has updated the pull request incrementally with one additional commit since the last revision: Update HprofParser.java - C

Integrated: 8311775: [TEST] duplicate verifyHeapDump in several tests

2023-09-03 Thread Yi Yang
On Wed, 9 Aug 2023 07:02:34 GMT, Yi Yang wrote: > This is a follow-up patch of #13667. verifyHeapDump is duplicated in several > tests, this patch tries to consolidate them into one method. This pull request has now been integrated. Changeset: 75d4ac26 Author: Yi Yang URL:

Re: RFR: 8314021: HeapDump: Optimize segmented heap file merging phase

2023-09-05 Thread Yi Yang
On Fri, 11 Aug 2023 09:31:56 GMT, Yi Yang wrote: > This patch reduce ~16%(24s->20s) pahse 2 merge time during dumping 32g heap > with 96threads and fixes a memory leak of compressor > > You might argue why this is Linux-only optimization, because sendfile > requires at

Re: RFR: 8314021: HeapDump: Optimize segmented heap file merging phase [v2]

2023-09-06 Thread Yi Yang
i/mswsock/nf-mswsock-transmitfile)), > while [only Linux](https://man7.org/linux/man-pages/man2/sendfile.2.html) > supports both two file descriptors. Yi Yang has updated the pull request incrementally with one additional commit since the last revision: one merge_file for all

Re: RFR: 8314021: HeapDump: Optimize segmented heap file merging phase

2023-09-06 Thread Yi Yang
On Thu, 7 Sep 2023 00:56:40 GMT, Alex Menkov wrote: > The fix looks good to me in general, but I'm not sure about code > organization. src/hotspot/share/runtime/os.hpp describes rules for os* files. > It states: // Platform-independent source files should not include these > header files // (a

Re: RFR: 8314021: HeapDump: Optimize segmented heap file merging phase

2023-09-10 Thread Yi Yang
On Thu, 7 Sep 2023 00:56:40 GMT, Alex Menkov wrote: >> This patch reduce ~16%(24s->20s) pahse 2 merge time during dumping 32g heap >> with 96threads and fixes a memory leak of compressor >> >> You might argue why this is Linux-only optimization, because sendfile >> requires at least socket fd

  1   2   >