> On Feb. 26, 2015, 3:07 a.m., Gwen Shapira wrote:
> > Ah, now I get what "dedupe buffer" refers to :)
> >
> > I never managed to reproduce the oom, but this looks like the right fix.
>
> Ewen Cheslack-Postava wrote:
> This happened consistently for me, which provided good motivation to
> figure out how to fix it :) It looks like the tests use the default JVM
> settings (build.gradle has overrides for compile, but not for test), so the
> 500 MB allocation was always failing since it obviously exceeds the default
> 128 MB heap. Now I'm wondering if other people are running with different
> settings. Maybe if you have a gradle daemon running that was started with a
> bigger max heap size?
>
> Gwen Shapira wrote:
> My gradle.properties has:
> org.gradle.jvmargs=-XX:MaxPermSize=512m -Xmx1024m
>
> I take it that yours doesn't?
>
> The builds in Apache's jenkins didn't seem to fail on that test either...
>
> Ewen Cheslack-Postava wrote:
> The one checked into the repo does, and I double checked that I'm not
> overriding that setting in my ~/.gradle/gradle.properties. But as far as I
> can tell that only applies to the gradle daemon process. It forks off another
> process to run the tests, which doesn't inherit those settings. If I add this
> to the build.gradle file next to the overrides for ScalaCompile tasks:
>
> tasks.withType(JavaForkOptions) {
> jvmArgs = ['-XX:MaxPermSize=512m', '-Xmx1024m']
> }
>
> then the forked process gets those settings and the test passes without
> this patch.
>
> Gwen Shapira wrote:
> huh. so I have no idea why the tests pass on my setup...
>
> Ewen Cheslack-Postava wrote:
> I checked my setup by grabbing info from visualvm while the tests were
> running. Maybe double check the processes running for you when you run the
> tests, and what settings those processes have had passed to them?
>
> I don't think it's critical since the immediate issue is resolved by the
> patch, but it would be nice to know why our test environments are
> inconsistent.
First, I didn't know about visualvm. Its very sweet :)
Looking at it, the GradleDaemon had the flags I specified above and it had 500M
RAM allocated throughout the test. Used mem for this process grew gradually to
250M during the test execution.
GradleWorkerMain actually ran the tests. It had no memory-related JVM flags. It
started with 250M allocated and jumped to 750M when testDeleteTopicWithCleaner
started (with 522M used).
With that info, I suspect that the default JVM settings on my Mac OS X
Yosemite, JVM 1.7.0.67 were different.
I looked at: java -XX:+PrintFlagsFinal -version
and found:
uintx MaxHeapSize := 4294967296 {product}
Thats pretty large :)
- Gwen
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31447/#review74200
-----------------------------------------------------------
On Feb. 25, 2015, 11:27 p.m., Ewen Cheslack-Postava wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/31447/
> -----------------------------------------------------------
>
> (Updated Feb. 25, 2015, 11:27 p.m.)
>
>
> Review request for kafka.
>
>
> Bugs: KAFKA-1881
> https://issues.apache.org/jira/browse/KAFKA-1881
>
>
> Repository: kafka
>
>
> Description
> -------
>
> KAFKA-1881: Fix out of memory error in testDeleteTopicWithCleaner by making
> the dedupe buffer much smaller.
>
>
> Diffs
> -----
>
> core/src/test/scala/unit/kafka/admin/DeleteTopicTest.scala
> 0cbd72684a0cc6fe9f1fba2dc834de675b2eb994
>
> Diff: https://reviews.apache.org/r/31447/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Ewen Cheslack-Postava
>
>
