On Dec 11, 2016 5:11 PM, "John_Tai" <john_...@smics.com> wrote:
I associated the queue with the PE:
qconf -aattr queue pe_list cores all.q
The only slots were defined in the all.q queue, and just the total slots in
the PE:
>> # qconf -sp cores
>> pe_name cores
>> slots 999
>> user_lists NONE
>> xuser_lists NONE
Do I need to define slots in another way for each exec host? Is there a way
to check the current free slots for a host, other than the qstat -f below?
> # qstat -f
> queuename qtype resv/used/tot. load_avg arch
states
> ------------------------------------------------------------
---------------------
> all.q@ibm021 BIP 0/0/8 0.02 lx-amd64
> ------------------------------------------------------------
---------------------
> all.q@ibm037 BIP 0/0/8 0.00 lx-amd64
> ------------------------------------------------------------
---------------------
> all.q@ibm038 BIP 0/0/8 0.00 lx-amd64
What is the output of the command
qconf -sq all.q
? (I think that's right one)
Chris
-----Original Message-----
From: Reuti [mailto:re...@staff.uni-marburg.de]
Sent: Saturday, December 10, 2016 5:40
To: John_Tai
Cc: users@gridengine.org
Subject: Re: [gridengine users] CPU complex
Am 09.12.2016 um 10:36 schrieb John_Tai:
> 8 slots:
>
> # qstat -f
> queuename qtype resv/used/tot. load_avg arch
states
> ------------------------------------------------------------
---------------------
> all.q@ibm021 BIP 0/0/8 0.02 lx-amd64
> ------------------------------------------------------------
---------------------
> all.q@ibm037 BIP 0/0/8 0.00 lx-amd64
> ------------------------------------------------------------
---------------------
> all.q@ibm038 BIP 0/0/8 0.00 lx-amd64
> ------------------------------------------------------------
---------------------
> pc.q@ibm021 BIP 0/0/1 0.02 lx-amd64
> ------------------------------------------------------------
---------------------
> sim.q@ibm021 BIP 0/0/1 0.02 lx-amd64
Is there any limit of slots in the exechost defined, or in an RQS?
-- Reuti
>
> ######################################################################
> ######
> - PENDING JOBS - PENDING JOBS - PENDING JOBS - PENDING JOBS - PENDING
> JOBS ############################################################
################
> 89 0.55500 xclock johnt qw 12/09/2016 15:14:25 2
>
>
>
> -----Original Message-----
> From: Reuti [mailto:re...@staff.uni-marburg.de]
> Sent: Friday, December 09, 2016 3:46
> To: John_Tai
> Cc: users@gridengine.org
> Subject: Re: [gridengine users] CPU complex
>
> Hi,
>
> Am 09.12.2016 um 08:20 schrieb John_Tai:
>
>> I've setup PE but I'm having problems submitting jobs.
>>
>> - Here's the PE I created:
>>
>> # qconf -sp cores
>> pe_name cores
>> slots 999
>> user_lists NONE
>> xuser_lists NONE
>> start_proc_args /bin/true
>> stop_proc_args /bin/true
>> allocation_rule $pe_slots
>> control_slaves FALSE
>> job_is_first_task TRUE
>> urgency_slots min
>> accounting_summary FALSE
>> qsort_args NONE
>>
>> - I've then added this to all.q:
>>
>> qconf -aattr queue pe_list cores all.q
>
> How many "slots" were defined in there queue definition for all.q?
>
> -- Reuti
>
>
>> - Now I submit a job:
>>
>> # qsub -V -b y -cwd -now n -pe cores 2 -q all.q@ibm038 xclock Your
>> job
>> 89 ("xclock") has been submitted # qstat
>> job-ID prior name user state submit/start at queue
slots ja-task-ID
>> ------------------------------------------------------------
-----------------------------------------------------
>> 89 0.00000 xclock johnt qw 12/09/2016 15:14:25
2
>> # qalter -w p 89
>> Job 89 cannot run in PE "cores" because it only offers 0 slots
>> verification: no suitable queues
>> # qstat -f
>> queuename qtype resv/used/tot. load_avg arch
states
>> ------------------------------------------------------------
---------------------
>> all.q@ibm038 BIP 0/0/8 0.00 lx-amd64
>>
>> #####################################################################
>> #
>> ######
>> - PENDING JOBS - PENDING JOBS - PENDING JOBS - PENDING JOBS - PENDING
>> JOBS ############################################################
################
>> 89 0.55500 xclock johnt qw 12/09/2016 15:14:25 2
>>
>>
>> ----------------------------------------------------
>>
>> It looks like all.q@ibm038 should have 8 free slots, so why is it only
offering 0?
>>
>> Hope you can help me.
>> Thanks
>> John
>>
>>
>>
>>
>>
>>
>> -----Original Message-----
>> From: Reuti [mailto:re...@staff.uni-marburg.de]
>> Sent: Monday, December 05, 2016 6:32
>> To: John_Tai
>> Cc: users@gridengine.org
>> Subject: Re: [gridengine users] CPU complex
>>
>> Hi,
>>
>>> Am 05.12.2016 um 09:36 schrieb John_Tai <john_...@smics.com>:
>>>
>>> Thank you so much for your reply!
>>>
>>>>> Will you use the consumable virtual_free here instead mem?
>>>
>>> Yes I meant to write virtual_free, not mem. Apologies.
>>>
>>>>> For parallel jobs you need to configure a (or some) so called PE
(Parallel Environment).
>>>
>>> My jobs are actually just one process which uses multiple cores, so for
example in top one process "simv" is currently using 2 cpu cores (200%).
>>
>> Yes, then it's a parallel job for SGE. Although the entries for
start_proc_args resp. stop_proc_args can be left untouched to the default,
a PE is the paradigm in SGE for a parallel job.
>>
>>
>>> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
>>> 3017 kelly 20 0 3353m 3.0g 165m R 200.0 0.6 15645:46 simv
>>>
>>> So I'm not sure PE is suitable for my case, since it is not multiple
parallel processes running at the same time. Am I correct?
>>>
>>> If so, I am trying to find a way to get SGE to keep track of the number
of cores used, but I believe it only keeps track of the total CPU usage in
%. I guess I could use this and and the <total num cores> to get the <num
of cores in use>, but how to integrate it in SGE?
>>
>> You can specify a necessary number of cores for your job in the -pe
parameter, which can also be a range. The granted allocation by SGE you can
check in the job script $NHOSTS, $NSLOTS, $PE_HOSTFILE.
>>
>> Having this setup, SGE will track the number of used cores per machine.
The available ones you define in the queue definition. In case you have
more than one queue per exechost, we need to setup in addition an overall
limit of cores which can be used at the same time to avoid oversubscription.
>>
>> -- Reuti
>>
>>> Thank you again for your help.
>>>
>>> John
>>>
>>> -----Original Message-----
>>> From: Reuti [mailto:re...@staff.uni-marburg.de]
>>> Sent: Monday, December 05, 2016 4:21
>>> To: John_Tai
>>> Cc: users@gridengine.org
>>> Subject: Re: [gridengine users] CPU complex
>>>
>>> Hi,
>>>
>>> Am 05.12.2016 um 08:00 schrieb John_Tai:
>>>
>>>> Newbie here, hope to understand SGE usage.
>>>>
>>>> I've successfully configured virtual_free as a complex for telling SGE
how much memory is needed when submitting a job, as described here:
>>>>
>>>> https://docs.oracle.com/cd/E19957-01/820-0698/6ncdvjclk/index.html#
>>>> i
>>>> 1000029
>>>>
>>>> How do I do the same for telling SGE how many CPU cores a job needs?
For example:
>>>>
>>>> qsub -l mem=24G,cpu=4 myjob
>>>
>>> Will you use the consumable virtual_free here instead mem?
>>>
>>>
>>>> Obviously I'd need for SGE to keep track of the actual CPU utilization
in the host, just as virtual_free is being tracked independently of the SGE
jobs.
>>>
>>> For parallel jobs you need to configure a (or some) so called PE
(Parallel Environment). Purpose of this is, to make preparations for the
parallel jobs like rearranging the list of granted slots, prepare shared
directories between the nodes,...
>>>
>>> These PEs were of higher importance in former times, when parallel
libraries were not programmed to integrate automatically in SGE for a tight
integration. Your submissions could read:
>>>
>>> qsub -pe smp 4 myjob # allocation_rule $peslots, control_slaves
true
>>> qsub -pe orte 16 myjob # allovation_rule $round_robin,
control_slaves tue
>>>
>>> where smp resp. orte is the chosen parallel environment for OpenMP
resp. Open MPI. Its settings are explained in `man sge_pe`, the "-pe"
parameter to in the submission command in `man qsub`.
>>>
>>> -- Reuti
>>> ________________________________
>>>
>>> This email (including its attachments, if any) may be confidential and
proprietary information of SMIC, and intended only for the use of the named
recipient(s) above. Any unauthorized use or disclosure of this email is
strictly prohibited. If you are not the intended recipient(s), please
notify the sender immediately and delete this email from your computer.
>>>
>>
>> ________________________________
>>
>> This email (including its attachments, if any) may be confidential and
proprietary information of SMIC, and intended only for the use of the named
recipient(s) above. Any unauthorized use or disclosure of this email is
strictly prohibited. If you are not the intended recipient(s), please
notify the sender immediately and delete this email from your computer.
>>
>
> ________________________________
>
> This email (including its attachments, if any) may be confidential and
proprietary information of SMIC, and intended only for the use of the named
recipient(s) above. Any unauthorized use or disclosure of this email is
strictly prohibited. If you are not the intended recipient(s), please
notify the sender immediately and delete this email from your computer.
>
________________________________
This email (including its attachments, if any) may be confidential and
proprietary information of SMIC, and intended only for the use of the named
recipient(s) above. Any unauthorized use or disclosure of this email is
strictly prohibited. If you are not the intended recipient(s), please
notify the sender immediately and delete this email from your computer.
_______________________________________________
users mailing list
users@gridengine.org
https://gridengine.org/mailman/listinfo/users
_______________________________________________
users mailing list
users@gridengine.org
https://gridengine.org/mailman/listinfo/users
