Hi Anton, Let's debug this a big. I suspect that the thresholds are the issue here. Does this script work?
ext1 = 4640 # data extent along 1 ext2 = 4650 # data extent along 2 ext3 = 4650 # data extent along 3 ffile = "./large/1000/zf5.raw" # fracture file imsize1 = 1200 # size, in pixels, of the resulting image along 1 imsize2 = 1200 # size, in pixels, of the resulting image along 2 cor1 = 0.5 * ext1 cor2 = 0.5 * ext2 cor3 = 0.5 * ext3 from paraview.simple import * # the extents start from zero, so need to lower # the upper extents by 1 cracks = ImageReader( FilePrefix= ffile ) cracks.DataExtent=[ 0, ext1-1, 0, ext2-1, 0, ext3-1 ] cracks.DataByteOrder = 'LittleEndian' cracks.DataScalarType = 'int' RenderView1 = GetRenderView() DataRepresentation1 = Show() If this works, let's discuss how to avoid using Threshold but get the same result. The issue with Threshold is that it creates a volume as an unstructured grid, which is heavy-weight. For example, a ~40KB volume (one float variable) turns into a ~800KB if I extract all of it as an unstructured grid. It seems like all you need is to extract the interfaces between the grains, correct? Best, -berk On Mon, Sep 21, 2015 at 10:50 AM, Anton Shterenlikht <me...@bris.ac.uk> wrote: > >From mexas Mon Sep 21 14:29:16 2015 > >To: paraview@paraview.org > >Subject: 400GB dataset out of memory on 1k cores? > >Reply-To: me...@bris.ac.uk > > > >I get OOM on a ~400GB raw dataset > >that I process with a script below. > >I use 24 core nodes with 64GB per node. > >I tried using up to 1008 nodes. > > error... 1008 cores (42 nodes). > > Anton > _______________________________________________ > Powered by www.kitware.com > > Visit other Kitware open-source projects at > http://www.kitware.com/opensource/opensource.html > > Please keep messages on-topic and check the ParaView Wiki at: > http://paraview.org/Wiki/ParaView > > Search the list archives at: http://markmail.org/search/?q=ParaView > > Follow this link to subscribe/unsubscribe: > http://public.kitware.com/mailman/listinfo/paraview >
_______________________________________________ Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the ParaView Wiki at: http://paraview.org/Wiki/ParaView Search the list archives at: http://markmail.org/search/?q=ParaView Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/paraview
