Hi Ufuk,
I'm guessing the issue is that the calls to Catalyst are not consistent.
Could you share your Python scripts? Also, did you modify them manually?
I tried with PV 5.3. with
the ../ParaView-v5.3.0/Examples/Catalyst/CxxFullExample/ example with the
attached scripts simultaneously by running with "./CxxFullExample
doubleoutputs.py output3.py image11.py" and got the correct output and no
warnings.
Cheers,
Andy
On Tue, Jul 4, 2017 at 7:04 AM, Ufuk Utku Turuncoglu (BE) <
[email protected]> wrote:
> Hi Andy,
>
> I tested you suggestion about using multiple script in co-processing. In
> this case, i used following code in the adaptor side to add multiple
> pipeline
>
> for (int i = 0; i < *nscript; i++) {
> pipeline->Initialize(pythonScriptNames[i]);
> g_coprocessor->AddPipeline(pipeline);
> }
>
> When i run the simulation, i am getting following warning
>
> Warning: In /okyanus/users/uturuncoglu/progs/paraview-5.3.0/src/
> ParaViewCore/VTKExtensions/Core/vtkPVTrivialProducer.cxx, line 66
> vtkPVTrivialProducer (0x13816760): New time step is not after last time
> step.
>
> the output seems not correct and it is zoom out version of second pipeline
> (png file). The first pipeline is not even triggered. Do i missing
> something in here? BTW, i am using PV 5.3.
>
> Thanks,
>
> --ufuk
>
>
> On 16/05/2017 16:08, Andy Bauer wrote:
>
> Hi Ufuk,
>
> If you create a vtkCPythonScriptPipeline, when you initialize it with the
> script file name (which has to be done on each process) everything will be
> taken care of with respect to broadcasting the file contents from process 0
> to the others. We aren't sophisticated enough to parse the Python script to
> see if it imports other scripts that are not part of ParaView (e.g.
> paraview.simple) or Python (e.g. sys). That is why I recommended the first
> approach as opposed to the second approach above. Depending on the compute
> platform and how many MPI processes are in the run the difference may be
> negligible but having 100K processes or more trying to access the same file
> can seriously slow down an HPC machine.
>
> Cheers,
> Andy
>
> On Tue, May 16, 2017 at 8:24 AM, Ufuk Utku Turuncoglu (BE) <
> [email protected]> wrote:
>
>> Thanks Andy. That is exactly what i am looking for. The broadcasting
>> mechanism is not clear to me yet. Do i need to broadcast only the file
>> names? Anyway, i will try to implement it and see what is going on there.
>>
>> Thanks again,
>> Regards,
>>
>> --ufuk
>>
>>
>> On 16/05/2017 14:58, Andy Bauer wrote:
>>
>> Hi Ufuk,
>>
>> Unless I'm not understanding your question correctly, I think you can get
>> what you want by adding in multiple vtkCPPythonScriptPipelines to your
>> vtkCPProcessor object in your adaptor. Alternatively if you want to have a
>> single, master Catalyst script handling other Catalyst scripts you can do
>> something like the following:
>> ================
>> import script_a
>> import script_b
>> import script_c
>>
>> def RequestDataDescription(datadescription):
>> script_a.RequestDataDescription(datadescription)
>> script_b.RequestDataDescription(datadescription)
>> script_c.RequestDataDescription(datadescription)
>>
>> def DoCoProcessing(datadescription):
>> script_a.DoCoProcessing(datadescription)
>> script_b.DoCoProcessing(datadescription)
>> script_c.DoCoProcessing(datadescription)
>> ===================
>>
>> The first way is the recommended way though as that should be more
>> efficient by having process 0 read the scripts and broadcasting the script
>> contents to the other processes for use. The second method will only do
>> that for the master script.
>>
>> Please let me know if this doesn't answer your question.
>>
>> Cheers,
>> Andy
>>
>> On Tue, May 16, 2017 at 5:46 AM, Ufuk Utku Turuncoglu (BE) <
>> [email protected]> wrote:
>>
>>> Hi All,
>>>
>>> I just wonder that is it possible to trigger multiple visualization
>>> pipeline in the same time with co-processing. The co-processing script
>>> generator plugin mainly outputs only single pipeline at a time and that is
>>> fine but what about combining multiple Python script (generated by plugin)
>>> using higher level Python script to trigger multiple pipelines. So, i think
>>> that this will be much efficient way to look at different part of the data
>>> without writing to the disk. I am not sure but somebody else might do it
>>> before.
>>>
>>> Regards,
>>>
>>> --ufuk
>>>
>>> _______________________________________________
>>> Powered by www.kitware.com
>>>
>>> Visit other Kitware open-source projects at
>>> http://www.kitware.com/opensource/opensource.html
>>>
>>> Please keep messages on-topic and check the ParaView Wiki at:
>>> http://paraview.org/Wiki/ParaView
>>>
>>> Search the list archives at: http://markmail.org/search/?q=ParaView
>>>
>>> Follow this link to subscribe/unsubscribe:
>>> http://public.kitware.com/mailman/listinfo/paraview
>>>
>>
>>
>>
>
>
from paraview.simple import *
from paraview import coprocessing
#--------------------------------------------------------------
# Code generated from cpstate.py to create the CoProcessor.
# ParaView 5.3.0 64 bits
# ----------------------- CoProcessor definition -----------------------
def CreateCoProcessor():
def _CreatePipeline(coprocessor, datadescription):
class Pipeline:
# state file generated using paraview version 5.3.0
# ----------------------------------------------------------------
# setup the data processing pipelines
# ----------------------------------------------------------------
#### disable automatic camera reset on 'Show'
paraview.simple._DisableFirstRenderCameraReset()
# create a new 'XML Partitioned Unstructured Grid Reader'
# create a producer from a simulation input
filename_99pvtu = coprocessor.CreateProducer(datadescription, 'input')
# create a new 'Slice'
slice1 = Slice(Input=filename_99pvtu)
slice1.SliceType = 'Plane'
slice1.SliceOffsetValues = [0.0]
# init the 'Plane' selected for 'SliceType'
slice1.SliceType.Origin = [34.5, 32.45, 27.95]
# create a new 'Parallel PolyData Writer'
parallelPolyDataWriter1 = servermanager.writers.XMLPPolyDataWriter(Input=slice1)
# register the writer with coprocessor
# and provide it with information such as the filename to use,
# how frequently to write the data, etc.
coprocessor.RegisterWriter(parallelPolyDataWriter1, filename='output_5_%t.pvtp', freq=5)
# create a new 'Slice'
slice2 = Slice(Input=filename_99pvtu)
slice2.SliceType = 'Plane'
slice2.SliceOffsetValues = [0.0]
# init the 'Plane' selected for 'SliceType'
slice2.SliceType.Origin = [34.5, 32.45, 27.95]
slice2.SliceType.Normal = [0.0, 1.0, 0.0]
# create a new 'Parallel PolyData Writer'
parallelPolyDataWriter2 = servermanager.writers.XMLPPolyDataWriter(Input=slice2)
# register the writer with coprocessor
# and provide it with information such as the filename to use,
# how frequently to write the data, etc.
coprocessor.RegisterWriter(parallelPolyDataWriter2, filename='output8_%t.pvtp', freq=8)
# ----------------------------------------------------------------
# finally, restore active source
SetActiveSource(parallelPolyDataWriter2)
# ----------------------------------------------------------------
return Pipeline()
class CoProcessor(coprocessing.CoProcessor):
def CreatePipeline(self, datadescription):
self.Pipeline = _CreatePipeline(self, datadescription)
coprocessor = CoProcessor()
# these are the frequencies at which the coprocessor updates.
freqs = {'input': [5, 8]}
coprocessor.SetUpdateFrequencies(freqs)
return coprocessor
#--------------------------------------------------------------
# Global variables that will hold the pipeline for each timestep
# Creating the CoProcessor object, doesn't actually create the ParaView pipeline.
# It will be automatically setup when coprocessor.UpdateProducers() is called the
# first time.
coprocessor = CreateCoProcessor()
#--------------------------------------------------------------
# Enable Live-Visualizaton with ParaView
coprocessor.EnableLiveVisualization(False, 1)
# ---------------------- Data Selection method ----------------------
def RequestDataDescription(datadescription):
"Callback to populate the request for current timestep"
global coprocessor
if datadescription.GetForceOutput() == True:
# We are just going to request all fields and meshes from the simulation
# code/adaptor.
for i in range(datadescription.GetNumberOfInputDescriptions()):
datadescription.GetInputDescription(i).AllFieldsOn()
datadescription.GetInputDescription(i).GenerateMeshOn()
return
# setup requests for all inputs based on the requirements of the
# pipeline.
coprocessor.LoadRequestedData(datadescription)
# ------------------------ Processing method ------------------------
def DoCoProcessing(datadescription):
"Callback to do co-processing for current timestep"
global coprocessor
# Update the coprocessor by providing it the newly generated simulation data.
# If the pipeline hasn't been setup yet, this will setup the pipeline.
coprocessor.UpdateProducers(datadescription)
# Write output data, if appropriate.
coprocessor.WriteData(datadescription);
# Write image capture (Last arg: rescale lookup table), if appropriate.
coprocessor.WriteImages(datadescription, rescale_lookuptable=False)
# Live Visualization, if enabled.
coprocessor.DoLiveVisualization(datadescription, "localhost", 22222)
from paraview.simple import *
from paraview import coprocessing
#--------------------------------------------------------------
# Code generated from cpstate.py to create the CoProcessor.
# ParaView 5.3.0 64 bits
# ----------------------- CoProcessor definition -----------------------
def CreateCoProcessor():
def _CreatePipeline(coprocessor, datadescription):
class Pipeline:
# state file generated using paraview version 5.3.0
# ----------------------------------------------------------------
# setup the data processing pipelines
# ----------------------------------------------------------------
#### disable automatic camera reset on 'Show'
paraview.simple._DisableFirstRenderCameraReset()
# create a new 'XML Partitioned Unstructured Grid Reader'
# create a producer from a simulation input
filename_99pvtu = coprocessor.CreateProducer(datadescription, 'input')
# create a new 'Slice'
slice1 = Slice(Input=filename_99pvtu)
slice1.SliceType = 'Plane'
slice1.SliceOffsetValues = [0.0]
# init the 'Plane' selected for 'SliceType'
slice1.SliceType.Origin = [34.5, 32.45, 27.95]
# create a new 'Parallel PolyData Writer'
parallelPolyDataWriter1 = servermanager.writers.XMLPPolyDataWriter(Input=slice1)
# register the writer with coprocessor
# and provide it with information such as the filename to use,
# how frequently to write the data, etc.
coprocessor.RegisterWriter(parallelPolyDataWriter1, filename='output_3_%t.pvtp', freq=3)
# ----------------------------------------------------------------
# finally, restore active source
SetActiveSource(parallelPolyDataWriter1)
# ----------------------------------------------------------------
return Pipeline()
class CoProcessor(coprocessing.CoProcessor):
def CreatePipeline(self, datadescription):
self.Pipeline = _CreatePipeline(self, datadescription)
coprocessor = CoProcessor()
# these are the frequencies at which the coprocessor updates.
freqs = {'input': [3]}
coprocessor.SetUpdateFrequencies(freqs)
return coprocessor
#--------------------------------------------------------------
# Global variables that will hold the pipeline for each timestep
# Creating the CoProcessor object, doesn't actually create the ParaView pipeline.
# It will be automatically setup when coprocessor.UpdateProducers() is called the
# first time.
coprocessor = CreateCoProcessor()
#--------------------------------------------------------------
# Enable Live-Visualizaton with ParaView
coprocessor.EnableLiveVisualization(False, 1)
# ---------------------- Data Selection method ----------------------
def RequestDataDescription(datadescription):
"Callback to populate the request for current timestep"
global coprocessor
if datadescription.GetForceOutput() == True:
# We are just going to request all fields and meshes from the simulation
# code/adaptor.
for i in range(datadescription.GetNumberOfInputDescriptions()):
datadescription.GetInputDescription(i).AllFieldsOn()
datadescription.GetInputDescription(i).GenerateMeshOn()
return
# setup requests for all inputs based on the requirements of the
# pipeline.
coprocessor.LoadRequestedData(datadescription)
# ------------------------ Processing method ------------------------
def DoCoProcessing(datadescription):
"Callback to do co-processing for current timestep"
global coprocessor
# Update the coprocessor by providing it the newly generated simulation data.
# If the pipeline hasn't been setup yet, this will setup the pipeline.
coprocessor.UpdateProducers(datadescription)
# Write output data, if appropriate.
coprocessor.WriteData(datadescription);
# Write image capture (Last arg: rescale lookup table), if appropriate.
coprocessor.WriteImages(datadescription, rescale_lookuptable=False)
# Live Visualization, if enabled.
coprocessor.DoLiveVisualization(datadescription, "localhost", 22222)
from paraview.simple import *
from paraview import coprocessing
#--------------------------------------------------------------
# Code generated from cpstate.py to create the CoProcessor.
# ParaView 5.3.0 64 bits
# ----------------------- CoProcessor definition -----------------------
def CreateCoProcessor():
def _CreatePipeline(coprocessor, datadescription):
class Pipeline:
# state file generated using paraview version 5.3.0
# ----------------------------------------------------------------
# setup views used in the visualization
# ----------------------------------------------------------------
#### disable automatic camera reset on 'Show'
paraview.simple._DisableFirstRenderCameraReset()
# Create a new 'Render View'
renderView1 = CreateView('RenderView')
renderView1.ViewSize = [1302, 1162]
renderView1.AxesGrid = 'GridAxes3DActor'
renderView1.CenterOfRotation = [34.5, 32.45, 27.95]
renderView1.StereoType = 0
renderView1.CameraPosition = [-63.70051803648709, 138.21873161577062, 183.89650547729363]
renderView1.CameraFocalPoint = [34.5, 32.45, 27.95]
renderView1.CameraViewUp = [0.159830273735241, 0.8608059992371918, -0.4831845561221694]
renderView1.CameraParallelScale = 54.99504523136608
renderView1.Background = [0.32, 0.34, 0.43]
# register the view with coprocessor
# and provide it with information such as the filename to use,
# how frequently to write the images, etc.
coprocessor.RegisterView(renderView1,
filename='image_%t.png', freq=11, fittoscreen=0, magnification=1, width=1302, height=1162, cinema={})
renderView1.ViewTime = datadescription.GetTime()
# ----------------------------------------------------------------
# setup the data processing pipelines
# ----------------------------------------------------------------
# create a new 'XML Partitioned Unstructured Grid Reader'
# create a producer from a simulation input
filename_99pvtu = coprocessor.CreateProducer(datadescription, 'input')
# ----------------------------------------------------------------
# setup color maps and opacity mapes used in the visualization
# note: the Get..() functions create a new object, if needed
# ----------------------------------------------------------------
# get color transfer function/color map for 'velocity'
velocityLUT = GetColorTransferFunction('velocity')
velocityLUT.RGBPoints = [0.0, 0.231373, 0.298039, 0.752941, 321.25500000000005, 0.865003, 0.865003, 0.865003, 642.5100000000001, 0.705882, 0.0156863, 0.14902]
velocityLUT.NumberOfTableValues = 620
velocityLUT.ScalarRangeInitialized = 1.0
velocityLUT.IndexedColors = [0.0, 0.0, 0.0]
# get opacity transfer function/opacity map for 'velocity'
velocityPWF = GetOpacityTransferFunction('velocity')
velocityPWF.Points = [0.0, 0.0, 0.5, 0.0, 642.5100000000001, 0.9342105388641357, 0.5, 0.0]
velocityPWF.ScalarRangeInitialized = 1
# ----------------------------------------------------------------
# setup the visualization in view 'renderView1'
# ----------------------------------------------------------------
# show data from filename_99pvtu
filename_99pvtuDisplay = Show(filename_99pvtu, renderView1)
# trace defaults for the display properties.
filename_99pvtuDisplay.Representation = 'Surface'
filename_99pvtuDisplay.ColorArrayName = ['POINTS', 'velocity']
filename_99pvtuDisplay.LookupTable = velocityLUT
filename_99pvtuDisplay.OSPRayScaleArray = 'velocity'
filename_99pvtuDisplay.OSPRayScaleFunction = 'PiecewiseFunction'
filename_99pvtuDisplay.SelectOrientationVectors = 'None'
filename_99pvtuDisplay.ScaleFactor = 6.9
filename_99pvtuDisplay.SelectScaleArray = 'None'
filename_99pvtuDisplay.GlyphType = 'Arrow'
filename_99pvtuDisplay.PolarAxes = 'PolarAxesRepresentation'
filename_99pvtuDisplay.ScalarOpacityUnitDistance = 1.966212140042607
filename_99pvtuDisplay.GaussianRadius = 3.45
filename_99pvtuDisplay.SetScaleArray = [None, '']
filename_99pvtuDisplay.ScaleTransferFunction = 'PiecewiseFunction'
filename_99pvtuDisplay.OpacityArray = [None, '']
filename_99pvtuDisplay.OpacityTransferFunction = 'PiecewiseFunction'
# init the 'PiecewiseFunction' selected for 'OSPRayScaleFunction'
filename_99pvtuDisplay.OSPRayScaleFunction.Points = [0.0, 0.0, 0.5, 0.0, 1.0, 0.9342105388641357, 0.5, 0.0]
# init the 'PiecewiseFunction' selected for 'ScaleTransferFunction'
filename_99pvtuDisplay.ScaleTransferFunction.Points = [0.0, 0.0, 0.5, 0.0, 1.0, 0.9342105388641357, 0.5, 0.0]
# init the 'PiecewiseFunction' selected for 'OpacityTransferFunction'
filename_99pvtuDisplay.OpacityTransferFunction.Points = [0.0, 0.0, 0.5, 0.0, 1.0, 0.9342105388641357, 0.5, 0.0]
# show color legend
filename_99pvtuDisplay.SetScalarBarVisibility(renderView1, True)
# setup the color legend parameters for each legend in this view
# get color legend/bar for velocityLUT in view renderView1
velocityLUTColorBar = GetScalarBar(velocityLUT, renderView1)
velocityLUTColorBar.Title = 'velocity'
velocityLUTColorBar.ComponentTitle = 'Magnitude'
# ----------------------------------------------------------------
# finally, restore active source
SetActiveSource(filename_99pvtu)
# ----------------------------------------------------------------
return Pipeline()
class CoProcessor(coprocessing.CoProcessor):
def CreatePipeline(self, datadescription):
self.Pipeline = _CreatePipeline(self, datadescription)
coprocessor = CoProcessor()
# these are the frequencies at which the coprocessor updates.
freqs = {'input': [11]}
coprocessor.SetUpdateFrequencies(freqs)
return coprocessor
#--------------------------------------------------------------
# Global variables that will hold the pipeline for each timestep
# Creating the CoProcessor object, doesn't actually create the ParaView pipeline.
# It will be automatically setup when coprocessor.UpdateProducers() is called the
# first time.
coprocessor = CreateCoProcessor()
#--------------------------------------------------------------
# Enable Live-Visualizaton with ParaView
coprocessor.EnableLiveVisualization(False, 1)
# ---------------------- Data Selection method ----------------------
def RequestDataDescription(datadescription):
"Callback to populate the request for current timestep"
global coprocessor
if datadescription.GetForceOutput() == True:
# We are just going to request all fields and meshes from the simulation
# code/adaptor.
for i in range(datadescription.GetNumberOfInputDescriptions()):
datadescription.GetInputDescription(i).AllFieldsOn()
datadescription.GetInputDescription(i).GenerateMeshOn()
return
# setup requests for all inputs based on the requirements of the
# pipeline.
coprocessor.LoadRequestedData(datadescription)
# ------------------------ Processing method ------------------------
def DoCoProcessing(datadescription):
"Callback to do co-processing for current timestep"
global coprocessor
# Update the coprocessor by providing it the newly generated simulation data.
# If the pipeline hasn't been setup yet, this will setup the pipeline.
coprocessor.UpdateProducers(datadescription)
# Write output data, if appropriate.
coprocessor.WriteData(datadescription);
# Write image capture (Last arg: rescale lookup table), if appropriate.
coprocessor.WriteImages(datadescription, rescale_lookuptable=False)
# Live Visualization, if enabled.
coprocessor.DoLiveVisualization(datadescription, "localhost", 22222)
_______________________________________________
Powered by www.kitware.com
Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html
Please keep messages on-topic and check the ParaView Wiki at:
http://paraview.org/Wiki/ParaView
Search the list archives at: http://markmail.org/search/?q=ParaView
Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/paraview
