On Wed, Oct 19, 2011 at 8:24 AM, Richard Yunhua Sang
<yunhua.s...@gmail.com> wrote:
> After more testing, I've seen the WeakReference impl. failed when JVM was
> very busy in multi-threaded operations. Although the SoftReference impl.
> survived in my test cases, it may fails too in radical situation of JVM.
>
> The Guava does support Lazy operations, but its API is not that friendly.
Well, it's nice to know I'm doing something right! Guava is too
general purpose, whereas tapestry-func fits a specific common purpose
inside Tapestry's code. Bring on the JDK closures!
>
> Thanks,
> Richard
> On Wed, Oct 19, 2011 at 2:40 AM, Howard Lewis Ship <hls...@gmail.com> wrote:
>
>> I'm curious what Guava (the google Java functional programming
>> library) does in this situation? I'm not sure they even try for
>> laziness.
>>
>> On Tue, Oct 18, 2011 at 5:57 PM, Richard Yunhua Sang
>> <yunhua.s...@gmail.com> wrote:
>> > Thanks for the new method, it's more easier to create a flow now.
>> >
>> > I have tried to use WeakReference/SoftReference to replace the rest
>> property
>> > in LazyFlow, both worked even in low memory scenario (-Xmx128m). The
>> problem
>> > is that the resolved Flow cannot be reused, it'll cause NPE since the
>> > references may have been GCed, though it's rare that the Flow will be
>> used
>> > again. Can Tapestry introduce a new type of flow which is particular for
>> > large dataset?
>> >
>> > Another interesting thing is creating a new object in WeakReference seems
>> > much faster than usual situation, at least on Mac.
>> >
>> > Thanks,
>> > Richard
>> >
>> > Oct 14, 2011 at 4:49 PM, Howard Lewis Ship <hls...@gmail.com> wrote:
>> >
>> >> On Fri, Oct 14, 2011 at 1:22 PM, Richard Yunhua Sang
>> >> <yunhua.s...@gmail.com> wrote:
>> >> > Tapestry 5 version: 5.3-beta19
>> >> >
>> >> > I love the API of tapestry-func, Today when I use tapestry-func to
>> >> process a
>> >> > file with 3 million lines, I get following exception:
>> >> >
>> >> > Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
>> >> > at java.util.Arrays.copyOfRange(Arrays.java:3209)
>> >> > at java.lang.String.<init>(String.java:216)
>> >> > at java.io.BufferedReader.readLine(BufferedReader.java:331)
>> >> > at java.io.BufferedReader.readLine(BufferedReader.java:362)
>> >> > at org.apache.commons.io.LineIterator.hasNext(LineIterator.java:97)
>> >> > at org.apache.tapestry5.func.LazyIterator.next(LazyIterator.java:33)
>> >> > at org.apache.tapestry5.func.LazyFlow.resolve(LazyFlow.java:78)
>> >> > at org.apache.tapestry5.func.LazyFlow.isEmpty(LazyFlow.java:61)
>> >> > at
>> >> org.apache.tapestry5.func.AbstractFlow$1.hasNext(AbstractFlow.java:63)
>> >> > at org.apache.tapestry5.func.AbstractFlow.each(AbstractFlow.java:103)
>> >> > at org.apache.tapestry5.func.AbstractFlow.each(AbstractFlow.java:32)
>> >> > at
>> >> com.datastream.services.flow.FileProcessor.main(FileProcessor.java:29)
>> >> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> >> > at
>> >> >
>> >>
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>> >> > at
>> >> >
>> >>
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>> >> > at java.lang.reflect.Method.invoke(Method.java:597)
>> >> > at
>> com.intellij.rt.execution.application.AppMain.main(AppMain.java:90)
>> >> >
>> >> > Java code: (using common.io)
>> >> >
>> >> > public static void main(String[] args) {
>> >> > try {
>> >> > final LineIterator lineIterator =
>> >> > FileUtils.lineIterator(FileUtils.getFile("c:\\tmp\\largefile.txt"));
>> >> > Iterable<String> lineIterable = new Iterable<String>() {
>> >> > public Iterator<String> iterator() {
>> >> > return lineIterator;
>> >> > }
>> >> > };
>> >> > F.flow(lineIterable)
>> >> > .each(new Worker<String>() {
>> >> > public void work(String s) {
>> >> > }
>> >> > });
>> >> > LineIterator.closeQuietly(lineIterator);
>> >> > } catch (IOException e) {
>> >> > e.printStackTrace();
>> >> > }
>> >> >
>> >> > }
>> >> >
>> >> > I have looked at the code, looks like Tapestry created a lot of
>> >> > chained LazyFlow objects that used up memories. Is it a bug?
>> >> >
>> >>
>> >> Certainly, that's not the intention; it's supposed to work that if you
>> >> don't retain the head of the Flow, it should be able to release nodes
>> >> to the GC. Unfortunately, there's a problem: the head is accessible to
>> >> the GC on the stack as you invoke the each() method. Clojure handles
>> >> this using semantics not available to Java (it sets the local variable
>> >> to null AFTER pushing it onto the stack). I'm not sure there's a
>> >> solution to this in pure Java ... I'm racking my brains right now to
>> >> think of an approach involving WeakReferences that would get the job
>> >> done.
>> >>
>> >> BTW, I'm adding a static F.flow(Iterable) method cause why not?
>> >>
>> >> > Thanks,
>> >> > Richard
>> >> >
>> >>
>> >>
>> >>
>> >> --
>> >> Howard M. Lewis Ship
>> >>
>> >> Creator of Apache Tapestry
>> >>
>> >> The source for Tapestry training, mentoring and support. Contact me to
>> >> learn how I can get you up and productive in Tapestry fast!
>> >>
>> >> (971) 678-5210
>> >> http://howardlewisship.com
>> >>
>> >> ---------------------------------------------------------------------
>> >> To unsubscribe, e-mail: users-unsubscr...@tapestry.apache.org
>> >> For additional commands, e-mail: users-h...@tapestry.apache.org
>> >>
>> >>
>> >
>>
>>
>>
>> --
>> Howard M. Lewis Ship
>>
>> Creator of Apache Tapestry
>>
>> The source for Tapestry training, mentoring and support. Contact me to
>> learn how I can get you up and productive in Tapestry fast!
>>
>> (971) 678-5210
>> http://howardlewisship.com
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscr...@tapestry.apache.org
>> For additional commands, e-mail: users-h...@tapestry.apache.org
>>
>>
>
--
Howard M. Lewis Ship
Creator of Apache Tapestry
The source for Tapestry training, mentoring and support. Contact me to
learn how I can get you up and productive in Tapestry fast!
(971) 678-5210
http://howardlewisship.com
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscr...@tapestry.apache.org
For additional commands, e-mail: users-h...@tapestry.apache.org
