You've found it in the dream, interesting!
Tetsuya Mishima
> Thanks - hit me in the middle of the night over here that we had missed
some options, but nice to find you had also seen it. Slightly modified
patch will be applied and brought over to 1.7.5
>
>
> On Feb 13, 2014, at 10:16 PM, tmish...@jcity.maeda.co.jp wrote:
>
> >
> >
> >
> > Please try attached patch - from r30723.
> >
> > (See attached file: patch.rmaps_base_frame.from_r30723)
> >
> > Tetsuya Mishima
> >
> >> Thanks for prompt help.
> >> Could you please resent the patch as attachment which can be applied
with
> > "patch" command, my mail client messes long lines.
> >>
> >>
> >> On Fri, Feb 14, 2014 at 7:40 AM, <tmish...@jcity.maeda.co.jp>wrote:
> >>
> >>
> >> Thanks. I'm not familiar with mindist mapper. But obviously
> >> checking for ORTE_MAPPING_BYDIST is missing. In addition,
> >> ORTE_MAPPING_PPR is missing again by my mistake.
> >>
> >> Please try this patch.
> >>
> >> if OPAL_HAVE_HWLOC
> >> } else if (ORTE_MAPPING_BYCORE == ORTE_GET_MAPPING_POLICY
> >> (mapping)) {
> >> ORTE_SET_RANKING_POLICY(tmp, ORTE_RANK_BY_CORE);
> >> } else if (ORTE_MAPPING_BYL1CACHE ==
ORTE_GET_MAPPING_POLICY
> >> (mapping)) {
> >> ORTE_SET_RANKING_POLICY(tmp, ORTE_RANK_BY_L1CACHE);
> >> } else if (ORTE_MAPPING_BYL2CACHE ==
ORTE_GET_MAPPING_POLICY
> >> (mapping)) {
> >> ORTE_SET_RANKING_POLICY(tmp, ORTE_RANK_BY_L2CACHE);
> >> } else if (ORTE_MAPPING_BYL3CACHE ==
ORTE_GET_MAPPING_POLICY
> >> (mapping)) {
> >> ORTE_SET_RANKING_POLICY(tmp, ORTE_RANK_BY_L3CACHE);
> >> } else if (ORTE_MAPPING_BYSOCKET ==
ORTE_GET_MAPPING_POLICY
> >> (mapping)) {
> >> ORTE_SET_RANKING_POLICY(tmp, ORTE_RANK_BY_SOCKET);
> >> } else if (ORTE_MAPPING_BYNUMA == ORTE_GET_MAPPING_POLICY
> >> (mapping)) {
> >> ORTE_SET_RANKING_POLICY(tmp, ORTE_RANK_BY_NUMA);
> >> } else if (ORTE_MAPPING_BYBOARD == ORTE_GET_MAPPING_POLICY
> >> (mapping)) {
> >> ORTE_SET_RANKING_POLICY(tmp, ORTE_RANK_BY_BOARD);
> >> } else if (ORTE_MAPPING_BYHWTHREAD ==
ORTE_GET_MAPPING_POLICY
> >> (mapping)) {
> >> ORTE_SET_RANKING_POLICY(tmp, ORTE_RANK_BY_HWTHREAD);
> >> } else if (ORTE_MAPPING_PPR == ORTE_GET_MAPPING_POLICY
> >> (mapping)) {
> >> ORTE_SET_RANKING_POLICY(tmp, ORTE_RANK_BY_SLOT);
> >> } else if (ORTE_MAPPING_BYDIST == ORTE_GET_MAPPING_POLICY
> >> (mapping)) {
> >> ORTE_SET_RANKING_POLICY(tmp, ORTE_RANK_BY_SLOT);
> >> #endif
> >>
> >> Tetsuya Mishima
> >>
> >>> Hi,
> >>> after this patch we get this in jenkins:
> >>>
> >>> 07:03:15 [vegas12.mtr.labs.mlnx:01646] [[26922,0],0] ORTE_ERROR_LOG:
> > Not
> >> implemented in file rmaps_mindist_module.c at line 39107:03:15
> >> [vegas12.mtr.labs.mlnx:01646] [[26922,0],0] ORTE_ERROR_LOG: Not
> >>> implemented in file base/rmaps_base_map_job.c at line 285
> >>>
> >>>
> >>>
> >>>
> >>>
> >>> On Fri, Feb 14, 2014 at 6:35 AM, <tmish...@jcity.maeda.co.jp>wrote:
> >>>
> >>>
> >>> Sorry, one more shot - byslot was dropped!
> >>>
> >>> if (NULL == spec) {
> >>> /* check for map-by object directives - we set the
> >>> * ranking to match if one was given
> >>> */
> >>> if (ORTE_MAPPING_GIVEN & ORTE_GET_MAPPING_DIRECTIVE(mapping))
{
> >>> if (ORTE_MAPPING_BYSLOT == ORTE_GET_MAPPING_POLICY
> > (mapping))
> >> {
> >>> ORTE_SET_RANKING_POLICY(tmp, ORTE_RANK_BY_SLOT);
> >>> } else if (ORTE_MAPPING_BYNODE == ORTE_GET_MAPPING_POLICY
> >>> (mapping)) {
> >>> ORTE_SET_RANKING_POLICY(tmp, ORTE_RANK_BY_NODE);
> >>> #if OPAL_HAVE_HWLOC
> >>> } else if (ORTE_MAPPING_BYCORE == ORTE_GET_MAPPING_POLICY
> >>> (mapping)) {
> >>> ORTE_SET_RANKING_POLICY(tmp, ORTE_RANK_BY_CORE);
> >>> } else if (ORTE_MAPPING_BYL1CACHE ==
> > ORTE_GET_MAPPING_POLICY
> >>> (mapping)) {
> >>> ORTE_SET_RANKING_POLICY(tmp, ORTE_RANK_BY_L1CACHE);
> >>> } else if (ORTE_MAPPING_BYL2CACHE ==
> > ORTE_GET_MAPPING_POLICY
> >>> (mapping)) {
> >>> ORTE_SET_RANKING_POLICY(tmp, ORTE_RANK_BY_L2CACHE);
> >>> } else if (ORTE_MAPPING_BYL3CACHE ==
> > ORTE_GET_MAPPING_POLICY
> >>> (mapping)) {
> >>> ORTE_SET_RANKING_POLICY(tmp, ORTE_RANK_BY_L3CACHE);
> >>> } else if (ORTE_MAPPING_BYSOCKET ==
ORTE_GET_MAPPING_POLICY
> >>> (mapping)) {
> >>> ORTE_SET_RANKING_POLICY(tmp, ORTE_RANK_BY_SOCKET);
> >>> } else if (ORTE_MAPPING_BYNUMA == ORTE_GET_MAPPING_POLICY
> >>> (mapping)) {
> >>> ORTE_SET_RANKING_POLICY(tmp, ORTE_RANK_BY_NUMA);
> >>> } else if (ORTE_MAPPING_BYBOARD ==
ORTE_GET_MAPPING_POLICY
> >>> (mapping)) {
> >>> ORTE_SET_RANKING_POLICY(tmp, ORTE_RANK_BY_BOARD);
> >>> } else if (ORTE_MAPPING_BYHWTHREAD ==
> > ORTE_GET_MAPPING_POLICY
> >>> (mapping)) {
> >>> ORTE_SET_RANKING_POLICY(tmp, ORTE_RANK_BY_HWTHREAD);
> >>> #endif
> >>>
> >>> Tetusya Mishima
> >>>
> >>>> I've found it. Please add 2 lines(770, 771) in rmaps_base_frame.c:
> >>>>
> >>>> 747 if (NULL == spec) {
> >>>> 748 /* check for map-by object directives - we set the
> >>>> 749 * ranking to match if one was given
> >>>> 750 */
> >>>> 751 if (ORTE_MAPPING_GIVEN & ORTE_GET_MAPPING_DIRECTIVE
> >>>> (mapping)) {
> >>>> 752 if (ORTE_MAPPING_BYCORE == ORTE_GET_MAPPING_POLICY
> >>>> (mapping)) {
> >>>> 753 ORTE_SET_RANKING_POLICY(tmp,
ORTE_RANK_BY_CORE);
> >>>> 754 } else if (ORTE_MAPPING_BYNODE ==
> >>>> ORTE_GET_MAPPING_POLICY(mapping)) {
> >>>> 755 ORTE_SET_RANKING_POLICY(tmp,
ORTE_RANK_BY_NODE);
> >>>> 756 } else if (ORTE_MAPPING_BYL1CACHE ==
> >>>> ORTE_GET_MAPPING_POLICY(mapping)) {
> >>>> 757 ORTE_SET_RANKING_POLICY(tmp,
> >> ORTE_RANK_BY_L1CACHE);
> >>>> 758 } else if (ORTE_MAPPING_BYL2CACHE ==
> >>>> ORTE_GET_MAPPING_POLICY(mapping)) {
> >>>> 759 ORTE_SET_RANKING_POLICY(tmp,
> >> ORTE_RANK_BY_L2CACHE);
> >>>> 760 } else if (ORTE_MAPPING_BYL3CACHE ==
> >>>> ORTE_GET_MAPPING_POLICY(mapping)) {
> >>>> 761 ORTE_SET_RANKING_POLICY(tmp,
> >> ORTE_RANK_BY_L3CACHE);
> >>>> 762 } else if (ORTE_MAPPING_BYSOCKET ==
> >>>> ORTE_GET_MAPPING_POLICY(mapping)) {
> >>>> 763 ORTE_SET_RANKING_POLICY(tmp,
> > ORTE_RANK_BY_SOCKET);
> >>>> 764 } else if (ORTE_MAPPING_BYNUMA ==
> >>>> ORTE_GET_MAPPING_POLICY(mapping)) {
> >>>> 765 ORTE_SET_RANKING_POLICY(tmp,
ORTE_RANK_BY_NUMA);
> >>>> 766 } else if (ORTE_MAPPING_BYBOARD ==
> >>>> ORTE_GET_MAPPING_POLICY(mapping)) {
> >>>> 767 ORTE_SET_RANKING_POLICY(tmp,
> > ORTE_RANK_BY_BOARD);
> >>>> 768 } else if (ORTE_MAPPING_BYHWTHREAD ==
> >>>> ORTE_GET_MAPPING_POLICY(mapping)) {
> >>>> 769 ORTE_SET_RANKING_POLICY(tmp,
> >>>> ORTE_RANK_BY_HWTHREAD);
> >>>> 770 } else if (ORTE_MAPPING_PPR ==
> > ORTE_GET_MAPPING_POLICY
> >>>> (mapping)) {
> >>>> 771 ORTE_SET_RANKING_POLICY(tmp,
ORTE_RANK_BY_SLOT);
> >>>> 772 }
> >>>>
> >>>> Tetsuya Mishima
> >>>>
> >>>>> You are welcome, Ralph.
> >>>>>
> >>>>> But, after fixing it, I'm facing another problem whin I use ppr
> >> option:
> >>>>> [mishima@manage openmpi-1.7.4]$ mpirun -np 2 -map-by ppr:1:socket
> >>>> -bind-to
> >>>>> socket -report-bindings ~/mis/openmpi/demos/m
> >>>>> yprog
> >>>>> [manage.cluster:28057] [[25570,0],0] ORTE_ERROR_LOG: Not
> > implemented
> >> in
> >>>>> file rmaps_ppr.c at line 389
> >>>>> [manage.cluster:28057] [[25570,0],0] ORTE_ERROR_LOG: Not
> > implemented
> >> in
> >>>>> file base/rmaps_base_map_job.c at line 285
> >>>>>
> >>>>> I confirmed it worked when it reverted back.
> >>>>> I'm a little bit confused. Could you take a look?
> >>>>>
> >>>>> Tetsuya Mishima
> >>>>>
> >>>>>> Thanks - these used to be bitmaps, but changed when we started
> >>> getting
> >>>> so
> >>>>> many options. Sadly, they are very rarely used, so bugs like this
> > can
> >>> go
> >>>>> unnoticed for long times. Appreciate you taking such
> >>>>>> a close look at them.
> >>>>>>
> >>>>>> Ralph
> >>>>>>
> >>>>>> On Feb 13, 2014, at 4:55 PM, tmish...@jcity.maeda.co.jp wrote:
> >>>>>>
> >>>>>>>
> >>>>>>> Hi Ralph,
> >>>>>>>
> >>>>>>> I would report one more finding in openmpi-1.7.5a1.
> >>>>>>>
> >>>>>>> Because ORTE_MAPPING_BY...s are not a bit field expression,
> >>>>>>> at orte_rmaps_base_set_ranking_policy in rmaps_base_frame.c
> >>>>>>> you should not use "&" to compare them:
> >>>>>>>
> >>>>>>> 747 if (NULL == spec) {
> >>>>>>> 748 /* check for map-by object directives - we set
> > the
> >>>>>>> 749 * ranking to match if one was given
> >>>>>>> 750 */
> >>>>>>> 751 if (ORTE_MAPPING_GIVEN &
> >> ORTE_GET_MAPPING_DIRECTIVE
> >>>>>>> (mapping)) {
> >>>>>>> 752 if (ORTE_MAPPING_BYCORE ==
> >>> ORTE_GET_MAPPING_POLICY
> >>>>>>> (mapping)) {
> >>>>>>> 753 ORTE_SET_RANKING_POLICY(tmp,
> >>>> ORTE_RANK_BY_CORE);
> >>>>>>> 754 } else if (ORTE_MAPPING_BYNODE ==
> >>>>>>> ORTE_GET_MAPPING_POLICY(mapping)) {
> >>>>>>> 755 ORTE_SET_RANKING_POLICY(tmp,
> >>>> ORTE_RANK_BY_NODE);
> >>>>>>> 756 } else if (ORTE_MAPPING_BYL1CACHE ==
> >>>>>>> ORTE_GET_MAPPING_POLICY(mapping)) {
> >>>>>>> 757 ORTE_SET_RANKING_POLICY(tmp,
> >>>>> ORTE_RANK_BY_L1CACHE);
> >>>>>>> 758 } else if (ORTE_MAPPING_BYL2CACHE ==
> >>>>>>> ORTE_GET_MAPPING_POLICY(mapping)) {
> >>>>>>> 759 ORTE_SET_RANKING_POLICY(tmp,
> >>>>> ORTE_RANK_BY_L2CACHE);
> >>>>>>> 760 } else if (ORTE_MAPPING_BYL3CACHE ==
> >>>>>>> ORTE_GET_MAPPING_POLICY(mapping)) {
> >>>>>>> 761 ORTE_SET_RANKING_POLICY(tmp,
> >>>>> ORTE_RANK_BY_L3CACHE);
> >>>>>>> 762 } else if (ORTE_MAPPING_BYSOCKET ==
> >>>>>>> ORTE_GET_MAPPING_POLICY(mapping)) {
> >>>>>>> 763 ORTE_SET_RANKING_POLICY(tmp,
> >>>>> ORTE_RANK_BY_SOCKET);
> >>>>>>> 764 } else if (ORTE_MAPPING_BYNUMA ==
> >>>>>>> ORTE_GET_MAPPING_POLICY(mapping)) {
> >>>>>>> 765 ORTE_SET_RANKING_POLICY(tmp,
> >>>> ORTE_RANK_BY_NUMA);
> >>>>>>> 766 } else if (ORTE_MAPPING_BYBOARD ==
> >>>>>>> ORTE_GET_MAPPING_POLICY(mapping)) {
> >>>>>>> 767 ORTE_SET_RANKING_POLICY(tmp,
> >>>>> ORTE_RANK_BY_BOARD);
> >>>>>>> 768 } else if (ORTE_MAPPING_BYHWTHREAD ==
> >>>>>>> ORTE_GET_MAPPING_POLICY(mapping)) {
> >>>>>>> 769 ORTE_SET_RANKING_POLICY(tmp,
> >>>>>>> ORTE_RANK_BY_HWTHREAD);
> >>>>>>> 770 }
> >>>>>>>
> >>>>>>> Regards,
> >>>>>>> Tetsuya Mishima
> >>>>>>>
> >>>>>>> _______________________________________________
> >>>>>>> users mailing list
> >>>>>>> us...@open-mpi.org
> >>>>>>> http://www.open-mpi.org/mailman/listinfo.cgi/users
> >>>>>>
> >>>>>> _______________________________________________
> >>>>>> users mailing list
> >>>>>> us...@open-mpi.org
> >>>>>> http://www.open-mpi.org/mailman/listinfo.cgi/users
> >>>>>
> >>>>> _______________________________________________
> >>>>> users mailing list
> >>>>> us...@open-mpi.org
> >>>>> http://www.open-mpi.org/mailman/listinfo.cgi/users
> >>>>
> >>>> _______________________________________________
> >>>> users mailing list
> >>>> us...@open-mpi.org
> >>>> http://www.open-mpi.org/mailman/listinfo.cgi/users
> >>>
> >>> _______________________________________________
> >>> users mailing list
> >>> us...@open-mpi.org
> >>>
> >>
> >
http://www.open-mpi.org/mailman/listinfo.cgi/users_______________________________________________
> >
> >>
> >>> users mailing list
> >>> users@open-mpi.orghttp://www.open-mpi.org/mailman/listinfo.cgi/users
> >>
> >> _______________________________________________
> >> users mailing list
> >> us...@open-mpi.org
> >>
> >
http://www.open-mpi.org/mailman/listinfo.cgi/users_______________________________________________
> >
> >> users mailing list
> >> users@open-mpi.orghttp://www.open-mpi.org/mailman/listinfo.cgi/users
> >
<patch.rmaps_base_frame.from_r30723>_______________________________________________
> > users mailing list
> > us...@open-mpi.org
> > http://www.open-mpi.org/mailman/listinfo.cgi/users
>
> _______________________________________________
> users mailing list
> us...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/users
