8 slots:
# qstat -f
queuename qtype resv/used/tot. load_avg arch
states
---------------------------------------------------------------------------------
all.q@ibm021 BIP 0/0/8 0.02 lx-amd64
---------------------------------------------------------------------------------
all.q@ibm037 BIP 0/0/8 0.00 lx-amd64
---------------------------------------------------------------------------------
all.q@ibm038 BIP 0/0/8 0.00 lx-amd64
---------------------------------------------------------------------------------
pc.q@ibm021 BIP 0/0/1 0.02 lx-amd64
---------------------------------------------------------------------------------
sim.q@ibm021 BIP 0/0/1 0.02 lx-amd64
############################################################################
- PENDING JOBS - PENDING JOBS - PENDING JOBS - PENDING JOBS - PENDING JOBS
############################################################################
89 0.55500 xclock johnt qw 12/09/2016 15:14:25 2
-----Original Message-----
From: Reuti [mailto:re...@staff.uni-marburg.de]
Sent: Friday, December 09, 2016 3:46
To: John_Tai
Cc: users@gridengine.org
Subject: Re: [gridengine users] CPU complex
Hi,
Am 09.12.2016 um 08:20 schrieb John_Tai:
> I've setup PE but I'm having problems submitting jobs.
>
> - Here's the PE I created:
>
> # qconf -sp cores
> pe_name cores
> slots 999
> user_lists NONE
> xuser_lists NONE
> start_proc_args /bin/true
> stop_proc_args /bin/true
> allocation_rule $pe_slots
> control_slaves FALSE
> job_is_first_task TRUE
> urgency_slots min
> accounting_summary FALSE
> qsort_args NONE
>
> - I've then added this to all.q:
>
> qconf -aattr queue pe_list cores all.q
How many "slots" were defined in there queue definition for all.q?
-- Reuti
> - Now I submit a job:
>
> # qsub -V -b y -cwd -now n -pe cores 2 -q all.q@ibm038 xclock Your job
> 89 ("xclock") has been submitted # qstat
> job-ID prior name user state submit/start at queue
> slots ja-task-ID
> -----------------------------------------------------------------------------------------------------------------
> 89 0.00000 xclock johnt qw 12/09/2016 15:14:25
> 2
> # qalter -w p 89
> Job 89 cannot run in PE "cores" because it only offers 0 slots
> verification: no suitable queues
> # qstat -f
> queuename qtype resv/used/tot. load_avg arch
> states
> ---------------------------------------------------------------------------------
> all.q@ibm038 BIP 0/0/8 0.00 lx-amd64
>
> ######################################################################
> ######
> - PENDING JOBS - PENDING JOBS - PENDING JOBS - PENDING JOBS - PENDING
> JOBS
> ############################################################################
> 89 0.55500 xclock johnt qw 12/09/2016 15:14:25 2
>
>
> ----------------------------------------------------
>
> It looks like all.q@ibm038 should have 8 free slots, so why is it only
> offering 0?
>
> Hope you can help me.
> Thanks
> John
>
>
>
>
>
>
> -----Original Message-----
> From: Reuti [mailto:re...@staff.uni-marburg.de]
> Sent: Monday, December 05, 2016 6:32
> To: John_Tai
> Cc: users@gridengine.org
> Subject: Re: [gridengine users] CPU complex
>
> Hi,
>
>> Am 05.12.2016 um 09:36 schrieb John_Tai <john_...@smics.com>:
>>
>> Thank you so much for your reply!
>>
>>>> Will you use the consumable virtual_free here instead mem?
>>
>> Yes I meant to write virtual_free, not mem. Apologies.
>>
>>>> For parallel jobs you need to configure a (or some) so called PE (Parallel
>>>> Environment).
>>
>> My jobs are actually just one process which uses multiple cores, so for
>> example in top one process "simv" is currently using 2 cpu cores (200%).
>
> Yes, then it's a parallel job for SGE. Although the entries for
> start_proc_args resp. stop_proc_args can be left untouched to the default, a
> PE is the paradigm in SGE for a parallel job.
>
>
>> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
>> 3017 kelly 20 0 3353m 3.0g 165m R 200.0 0.6 15645:46 simv
>>
>> So I'm not sure PE is suitable for my case, since it is not multiple
>> parallel processes running at the same time. Am I correct?
>>
>> If so, I am trying to find a way to get SGE to keep track of the number of
>> cores used, but I believe it only keeps track of the total CPU usage in %. I
>> guess I could use this and and the <total num cores> to get the <num of
>> cores in use>, but how to integrate it in SGE?
>
> You can specify a necessary number of cores for your job in the -pe
> parameter, which can also be a range. The granted allocation by SGE you can
> check in the job script $NHOSTS, $NSLOTS, $PE_HOSTFILE.
>
> Having this setup, SGE will track the number of used cores per machine. The
> available ones you define in the queue definition. In case you have more than
> one queue per exechost, we need to setup in addition an overall limit of
> cores which can be used at the same time to avoid oversubscription.
>
> -- Reuti
>
>> Thank you again for your help.
>>
>> John
>>
>> -----Original Message-----
>> From: Reuti [mailto:re...@staff.uni-marburg.de]
>> Sent: Monday, December 05, 2016 4:21
>> To: John_Tai
>> Cc: users@gridengine.org
>> Subject: Re: [gridengine users] CPU complex
>>
>> Hi,
>>
>> Am 05.12.2016 um 08:00 schrieb John_Tai:
>>
>>> Newbie here, hope to understand SGE usage.
>>>
>>> I've successfully configured virtual_free as a complex for telling SGE how
>>> much memory is needed when submitting a job, as described here:
>>>
>>> https://docs.oracle.com/cd/E19957-01/820-0698/6ncdvjclk/index.html#i
>>> 1000029
>>>
>>> How do I do the same for telling SGE how many CPU cores a job needs? For
>>> example:
>>>
>>> qsub -l mem=24G,cpu=4 myjob
>>
>> Will you use the consumable virtual_free here instead mem?
>>
>>
>>> Obviously I'd need for SGE to keep track of the actual CPU utilization in
>>> the host, just as virtual_free is being tracked independently of the SGE
>>> jobs.
>>
>> For parallel jobs you need to configure a (or some) so called PE (Parallel
>> Environment). Purpose of this is, to make preparations for the parallel jobs
>> like rearranging the list of granted slots, prepare shared directories
>> between the nodes,...
>>
>> These PEs were of higher importance in former times, when parallel libraries
>> were not programmed to integrate automatically in SGE for a tight
>> integration. Your submissions could read:
>>
>> qsub -pe smp 4 myjob # allocation_rule $peslots, control_slaves true
>> qsub -pe orte 16 myjob # allovation_rule $round_robin,
>> control_slaves tue
>>
>> where smp resp. orte is the chosen parallel environment for OpenMP resp.
>> Open MPI. Its settings are explained in `man sge_pe`, the "-pe" parameter to
>> in the submission command in `man qsub`.
>>
>> -- Reuti
>> ________________________________
>>
>> This email (including its attachments, if any) may be confidential and
>> proprietary information of SMIC, and intended only for the use of the named
>> recipient(s) above. Any unauthorized use or disclosure of this email is
>> strictly prohibited. If you are not the intended recipient(s), please notify
>> the sender immediately and delete this email from your computer.
>>
>
> ________________________________
>
> This email (including its attachments, if any) may be confidential and
> proprietary information of SMIC, and intended only for the use of the named
> recipient(s) above. Any unauthorized use or disclosure of this email is
> strictly prohibited. If you are not the intended recipient(s), please notify
> the sender immediately and delete this email from your computer.
>
________________________________
This email (including its attachments, if any) may be confidential and
proprietary information of SMIC, and intended only for the use of the named
recipient(s) above. Any unauthorized use or disclosure of this email is
strictly prohibited. If you are not the intended recipient(s), please notify
the sender immediately and delete this email from your computer.
_______________________________________________
users mailing list
users@gridengine.org
https://gridengine.org/mailman/listinfo/users
