On 19/02/18 9:23 AM, Jaikiran Pai wrote:
On 19/02/18 1:37 AM, Stefan Bodewig wrote:
On 2018-02-15, Jaikiran Pai wrote:
We have 3 pre-defined formatters all of which are capable of receiving
this streamed sysout/syserr data. Each of these do _not_ hold on to
this sysout/syserr data in-memory during the execution and instead
stream it out a temporary file. Once the execution completes and it's
time to present the result, these formatters stream/read back the
content from the temp file and write it out in a formatted manner to
the target report files.
Sounds expensive.
Indeed. In 2 ways actually:
1. The creation of these files.
2. The fact that we write it out to these files (although in a
separate thread) and then read it while formatting the result, finally.
It's only applicable for result formatters which want to have the
sysout and syserr content sent to them, but I expect that most users
will want this enabled anyway. So yes, it's expensive.
The alternate approach is perhaps to have this content maintained
in-memory till a fixed size limit (relatively large enough but not too
much that it will cause OOM) and only create and stream into temporary
files if that size limit is exceeded. That way the temp file creation
and writing/reading perhaps can be avoided/controlled in most of the
cases.
I just updated the PR to include an additional commit which introduces
this idea. The (common internal) formatter class now has a "store" which
is used to store the sysout/syserr content. The store is initially
backed by a fixed size ByteBuffer and the incoming sysout/syserr content
is pushed to that. When/if the size limit is reached, the store is
switched to create a new temporary file and write out the already held
data and any new incoming data. Once the file is created, the in-memory
buffer is destroyed. I've intentionally kept this as a completely
internal implementation detail which means that the size limit of this
internal buffer isn't configurable. I (somewhat randomly) decided to use
50KB as the size limit for (each) sysout/syserr store but am willing to
change it to anything lower (or maybe even higher) that might make sense.
The PR is the same one, that we have been using so far for review
here[1] and the specific commit is this one[2]. I'll be squashing all
those commits into one before merging, once the review is done.
Thanks Stefan for the review so far, it has been of real help and
uncovered some bugs as well as my lack of knowledge of some of the Ant
task creation guidelines.
[1] https://github.com/apache/ant/pull/60
[2]
https://github.com/apache/ant/pull/60/commits/641b873b9e9db5d78c7cd7953da135fb0b9c097e
-Jaikiran
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@ant.apache.org
For additional commands, e-mail: dev-h...@ant.apache.org
