Hello, Why don't use task-array instead of separate jobs?
You can do something like: qsub -t 1-1000 script.sh In the script, you will get $SGE_TASK_ID environment variable with task number in it, and you may use it to select input tgz file. Best Regards, Mikhail Serkov On Thu, Jun 13, 2019 at 12:54 PM VG <gupta567va...@gmail.com> wrote: > Hi Daniel, > Will give it a try. If I am not mistaken, there should be another *done *in > the code snippet. > > Regards > Varun > > On Thu, Jun 13, 2019 at 12:45 PM Daniel Povey <dpo...@gmail.com> wrote: > >> >> for i in *tar.gz; >> do >> while true; do >> if [ $(qstat -u $USER | wc -l) -lt 900 ]; then break; fi; >> sleep 60; >> done >> qsub -l h_vmem=4G -cwd -j y -b y -N tar -R y -q all.q,gpu.q "tar -xzf >> $i" >> done >> >> On Thu, Jun 13, 2019 at 12:39 PM Skylar Thompson <skyl...@uw.edu> wrote: >> >>> We've used resource quota sets to accomplish that on a per-queue or >>> per-project basis. I don't know that you can limit on jobs in RQSs but >>> you >>> certainly can on slots; the sge_resource_quota(5) man page has some >>> examples. >>> >>> On Thu, Jun 13, 2019 at 12:32:51PM -0400, VG wrote: >>> > I have a scripting question regarding submitting jobs to the cluster. >>> > There is a limitation per user of 1000 jobs only. >>> > Let's say I have 1200 tar.gz files >>> > >>> > >>> > I tried to submit all the jobs together but after 1000 jobs it gave me >>> an >>> > error message saying per user limit is 1000 and after that it did not >>> > queued the remaining jobs. >>> > I want to write a script where if the submitted jobs goes below >>> > 1000(because they finished running), then next jobs are submitted in >>> the >>> > queue. How can I do that? >>> > I have written something like this: >>> > >>> > for i in *tar.gz >>> > do >>> > qsub -l h_vmem=4G -cwd -j y -b y -N tar -R y -q all.q,gpu.q "tar >>> -xzf $i" >>> > done >>> > >>> > Hope to hear from you soon. >>> > >>> > Regards >>> > Varun >>> >>> > _______________________________________________ >>> > users mailing list >>> > users@gridengine.org >>> > https://gridengine.org/mailman/listinfo/users >>> >>> >>> -- >>> -- Skylar Thompson (skyl...@u.washington.edu) >>> -- Genome Sciences Department, System Administrator >>> -- Foege Building S046, (206)-685-7354 >>> -- University of Washington School of Medicine >>> _______________________________________________ >>> users mailing list >>> users@gridengine.org >>> https://gridengine.org/mailman/listinfo/users >>> >> _______________________________________________ >> users mailing list >> users@gridengine.org >> https://gridengine.org/mailman/listinfo/users >> > _______________________________________________ > users mailing list > users@gridengine.org > https://gridengine.org/mailman/listinfo/users >
_______________________________________________ users mailing list users@gridengine.org https://gridengine.org/mailman/listinfo/users
