On Friday 14 July 2017 03:09 PM, Hemant Agrawal wrote:
> On 7/14/2017 2:00 PM, santosh wrote:
>> On Friday 14 July 2017 01:37 PM, Hemant Agrawal wrote:
>>
>>> On 7/11/2017 11:46 AM, Santosh Shukla wrote:
>>>> API(rte_bus_get_iommu_class) helps to automatically detect and select
>>>> appropriate iova mapping scheme for iommu capable device on that bus.
>>>>
>>>> Algorithm for iova scheme selection for bus:
>>>> 0. Iterate through bus_list.
>>>> 1. Collect each bus iova mode value and update into 'mode' var.
>>>> 2. Here value '1' is _pa and value '2' is _va mode.
>>>> So mode selection scheme is like:
>>>> if mode == 2 then iova mode is _va.
>>>> if mode == 1 then iova mode is _pa
>>>> if mode == 3 then iova mode ia _pa.
>>>>
>>>> So mode !=2 will be default iova mode.
>>>>
>>>> Signed-off-by: Santosh Shukla <santosh.shu...@caviumnetworks.com>
>>>> Signed-off-by: Jerin Jacob <jerin.ja...@caviumnetworks.com>
>>>> ---
>>>> lib/librte_eal/bsdapp/eal/rte_eal_version.map | 1 +
>>>> lib/librte_eal/common/eal_common_bus.c | 23
>>>> +++++++++++++++++++++++
>>>> lib/librte_eal/common/eal_common_pci.c | 1 +
>>>> lib/librte_eal/common/include/rte_bus.h | 22
>>>> ++++++++++++++++++++++
>>>> lib/librte_eal/linuxapp/eal/rte_eal_version.map | 1 +
>>>> 5 files changed, 48 insertions(+)
>>>>
>>>> diff --git a/lib/librte_eal/bsdapp/eal/rte_eal_version.map
>>>> b/lib/librte_eal/bsdapp/eal/rte_eal_version.map
>>>> index 33c2c32c0..a2dd65a33 100644
>>>> --- a/lib/librte_eal/bsdapp/eal/rte_eal_version.map
>>>> +++ b/lib/librte_eal/bsdapp/eal/rte_eal_version.map
>>>> @@ -202,6 +202,7 @@ DPDK_17.08 {
>>>> rte_bus_find_by_name;
>>>> rte_pci_match;
>>>> rte_pci_get_iommu_class;
>>>> + rte_bus_get_iommu_class;
>>>>
>>>> } DPDK_17.05;
>>>>
>>>> diff --git a/lib/librte_eal/common/eal_common_bus.c
>>>> b/lib/librte_eal/common/eal_common_bus.c
>>>> index 08bec2d93..5d5753ac9 100644
>>>> --- a/lib/librte_eal/common/eal_common_bus.c
>>>> +++ b/lib/librte_eal/common/eal_common_bus.c
>>>> @@ -222,3 +222,26 @@ rte_bus_find_by_device_name(const char *str)
>>>> c[0] = '\0';
>>>> return rte_bus_find(NULL, bus_can_parse, name);
>>>> }
>>>> +
>>>> +
>>>> +/*
>>>> + * Get iommu class of devices on the bus.
>>>> + */
>>>> +enum rte_iova_mode
>>>> +rte_bus_get_iommu_class(void)
>>>> +{
>>>> + int mode = 0;
>>>> + struct rte_bus *bus;
>>>> +
>>>> + TAILQ_FOREACH(bus, &rte_bus_list, next) {
>>>> +
>>>> + if (bus->get_iommu_class)
>>>> + mode |= bus->get_iommu_class();
>>>> + }
>>>> +
>>>
>>> If you change the default return as '0' for buses. This code will work.
>>> e.g. PCI will return '0' - when no device is probed. FSL MC will return VA.
>>> the default mode will be 'VA'
>>>
>> I'm confused why it won't work for fslmc case?
>>
>> Let me walk through the code:
>>
>> If no-pci device Or (future) no-platform device probed then bus opt
>> to use default mapping scheme .. which is iova_pa(default scheme).
>>
>> Lets take PCI_bus example:
>> bus->get_iommu_class()
>> ---> bus->_pci_get_iommu_class()
>> * Now consider that no interface bound to any of PCI device, then
>> it will return RTE_IOVA_PA mode to rte_bus layer (aka
>> bus->get_iommu_class).
>> So the iova mapping result from iommu_class scan is RTE_IOVA_PA
>> (default).
>> It works for PCI_bus case, tested for both iova_va and iova_pa
>> case, no-pci device case.
>>
>> Now in fslmc bus case:
>> bus->get_iommu_class()
>> ---> bus->_fslmc_get_iommu_class()
>>
>> * IIUC your comment - You want fslmc bus to return RTE_IOVA_VA if no
>> device
>> detected, Right?
> why?
>
As I didn't understood your previous reply:
`e.g. PCI will return '0' - when no device is probed. FSL MC will return VA.
the default mode will be 'VA'`
So, I'm asking you that in fslmc bus case - if no device found then are you
opting _va scheme or not?
Seems like _not_ per your below comment.
> If bus is just present but no device is in use for dpdk, then bus should
> return 0 and it *should not* participate in the IOMMU class decision.
>
I think, I understand your point..Example if you have no-pci on first PCI bus
but device found on 2nd platform bus then you don't want to fallback to default
(/_pa) mode..
instead you want to use 2nd bus mode for mapping, which is _va. Right?
If so then In my first version - We did introduced the case called _DC.
_DC:0 --> stands for no-device found case.
> Right now there are only two buses. There can be more buses. (e.g. PCI,
> platform, fslmc in case of dpaa2 as well).
>
> If the bus is not being used at all, why it influence the decision of other
> buses.
>
If your referring to above case then I agree, We'll re-introduce _DC state from
v1 in next revision.
That will look like
rte_pci_get_iommu_class() {
int mode = RTE_IOVA_DC; /* '0' */
return _DC; /* if no device found */
}
Right?
> if no bus has any device, the System default is anyway PA.
>
Right, If no bus present then It's also responsibility of
`rte_bus_get_iommu_class`
to use default mapping scheme which is _pa and which It does.
>
>> if so then your fslmc bus handle should do something like below
>> -- If no device on fslmc bus : return RTE_IOVA_VA.
>> -- If device detected on fslmc bus and bound to iommu driver :
>> return RTE_IOVA_VA
>> -- If device detected fslmc but not bound to iommu drv : return
>> RTE_IOVA_PA..
>>
>> make sense? If not then can you describe fslmc mapping scheme?
>>
>>> if fslmc is not present. The default mode will be PA.
>>>
>>>> + if (mode != RTE_IOVA_VA) {
>>>> + /* Use default IOVA mode */
>>>> + mode = RTE_IOVA_PA;
>>>> + }
>
> The system default is anyway PA.
>
No, That check is needed for case like 1st bus return with _PA and 2nd bus
returns with _VA,
then mode = 3 (Mix mode), which we don't support so (as I mentioned before) its
responsibility of
rte_bus_get_iommu_class() to return default mode (_pa). That's why!.
>>>> + return mode;
>>>> +}
>>>> diff --git a/lib/librte_eal/common/eal_common_pci.c
>>>> b/lib/librte_eal/common/eal_common_pci.c
>>>> index 8b6ecebd6..bdf2e7c3a 100644
>>>> --- a/lib/librte_eal/common/eal_common_pci.c
>>>> +++ b/lib/librte_eal/common/eal_common_pci.c
>>>> @@ -552,6 +552,7 @@ struct rte_pci_bus rte_pci_bus = {
>>>> .plug = pci_plug,
>>>> .unplug = pci_unplug,
>>>> .parse = pci_parse,
>>>> + .get_iommu_class = rte_pci_get_iommu_class,
>>>> },
>>>> .device_list = TAILQ_HEAD_INITIALIZER(rte_pci_bus.device_list),
>>>> .driver_list = TAILQ_HEAD_INITIALIZER(rte_pci_bus.driver_list),
>>>> diff --git a/lib/librte_eal/common/include/rte_bus.h
>>>> b/lib/librte_eal/common/include/rte_bus.h
>>>> index 7a0cfb165..8b2805b7f 100644
>>>> --- a/lib/librte_eal/common/include/rte_bus.h
>>>> +++ b/lib/librte_eal/common/include/rte_bus.h
>>>> @@ -181,6 +181,17 @@ struct rte_bus_conf {
>>>> enum rte_bus_scan_mode scan_mode; /**< Scan policy. */
>>>> };
>>>>
>>>> +
>>>> +/**
>>>> + * Get iommu class of devices on the bus.
>>>> + * Check that those devices are attached to iommu driver.
>>>> + *
>>>> + * @return
>>>> + * enum rte_iova_mode value.
>>>> + */
>>>> +typedef enum rte_iova_mode (*rte_bus_get_iommu_class_t)(void);
>>>> +
>>>> +
>>>> /**
>>>> * A structure describing a generic bus.
>>>> */
>>>> @@ -194,6 +205,7 @@ struct rte_bus {
>>>> rte_bus_unplug_t unplug; /**< Remove single device from driver */
>>>> rte_bus_parse_t parse; /**< Parse a device name */
>>>> struct rte_bus_conf conf; /**< Bus configuration */
>>>> + rte_bus_get_iommu_class_t get_iommu_class; /**< Get iommu class */
>>>> };
>>>>
>>>> /**
>>>> @@ -293,6 +305,16 @@ struct rte_bus *rte_bus_find_by_device(const struct
>>>> rte_device *dev);
>>>> */
>>>> struct rte_bus *rte_bus_find_by_name(const char *busname);
>>>>
>>>> +
>>>> +/**
>>>> + * Get iommu class of devices on the bus.
>>>> + * Check that those devices are attached to iommu driver.
>>>> + *
>>>> + * @return
>>>> + * enum rte_iova_mode value.
>>>> + */
>>>> +enum rte_iova_mode rte_bus_get_iommu_class(void);
>>>> +
>>>> /**
>>>> * Helper for Bus registration.
>>>> * The constructor has higher priority than PMD constructors.
>>>> diff --git a/lib/librte_eal/linuxapp/eal/rte_eal_version.map
>>>> b/lib/librte_eal/linuxapp/eal/rte_eal_version.map
>>>> index 044f89c7c..186c7b0fd 100644
>>>> --- a/lib/librte_eal/linuxapp/eal/rte_eal_version.map
>>>> +++ b/lib/librte_eal/linuxapp/eal/rte_eal_version.map
>>>> @@ -207,6 +207,7 @@ DPDK_17.08 {
>>>> rte_bus_find_by_name;
>>>> rte_pci_match;
>>>> rte_pci_get_iommu_class;
>>>> + rte_bus_get_iommu_class;
>>>>
>>>> } DPDK_17.05;
>>>>
>>>>
>>>
>>>
>>
>>
>
>
