I think would be "easy" in a scripted pipeline compared to a declarative pipeline. The reason I say that is because you can use loops directly in your pipeline, so if you knew the number of frames, you could iterate over the number of frames with 4 as an increment and create parallel steps for each iteration (https://jenkins.io/doc/pipeline/examples/#parallel-from-list). You'd use the iterator variable as the start index and the increment as the number of frames. You could then even make the number of frames a parameter to your job. I am not sure how this could be done with declarative, I am still new to using declarative.
On Tue, Dec 18, 2018 at 1:56 PM 'Mark Raynsford' via Jenkins Users < jenkinsci-users@googlegroups.com> wrote: > Hello! > > I've been using Jenkins to build code from hundreds of git repositories > using declarative pipelines. In the setup I'm using, I have a small > number of nodes, and all nodes must successfully build and run all > tests in order for the job as a whole to have been considered > successful. > > I'd now like to try using Jenkins to render animations using Blender[0]. > > The problem that I'm actually trying to solve is that Blender (like all > 3D renderers) generally takes a long time to render images. When > rendering an animation, each frame is rendered individually and saved > to a PNG file. When all frames have been rendered, the PNG files can be > stitched together with something like ffmpeg to produce a video file. > Given that there's no data dependency between frames, it's possible to > distribute the workload over many machines by giving each machine a set > of frame numbers to render and then collecting the images from all > machines when they've all finished rendering. > > Blender has a full command-line interface, so the 'sh' declaration > in the pipelines will be enough to launch instances of Blender for > rendering. I can place the data to be rendered in a Git repository so > that all nodes can pull from that repository to start building. I would > include a pipeline script in the repository that each node can use to > build. This is how I've been handling building code, and it's been > working very well. > > The problem: It's not clear to me how I could assign a set of frame > numbers to each node in the pipeline. I could hard code the numbers > into something that the pipeline code reads in the first stage of > building, but it seems like to do that effectively, I'd need to know > all of the nodes ahead of time (meaning I'd have to rewrite the data if > I add or remove a node later). > > What I would really like is for the pipeline in each node to be told > "You are node 1 of 4" "You are node 3 of 4", and so on. Node 3 could > then, for example, use this information to tell Blender "Start > rendering at frame 3, and step forward by 4 frames each time" (4 > because there are 4 nodes in total). If each node did this, the entire > range of frames would be covered and the work would be split equally > between the four nodes. The nodes I have are homogeneous, so there's no > need to try to give powerful nodes more work, and weaker nodes less. > > Any advice would be appreciated! > > [0] https://www.blender.org/ > > -- > Mark Raynsford | http://www.io7m.com > > -- > You received this message because you are subscribed to the Google Groups > "Jenkins Users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to jenkinsci-users+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/jenkinsci-users/20181218205530.767b8097%40almond.int.arc7.info > . > For more options, visit https://groups.google.com/d/optout. > -- Website: http://earl-of-code.com -- You received this message because you are subscribed to the Google Groups "Jenkins Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/CAPiUgVdbvm8oz%2BWt5k1KuJtKcH8J3LfFw2q5Hd0bXr20rXt4zA%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
