Re: [PATCH v2 0/2] scsi: qla2xxx: fixes for FW trace/dump buffers

2019-08-14 Thread Martin K. Petersen
Martin, > The first patch of the series is a fix for a memory corruption we > saw in a test where qla2xxx was loaded/unloaded repeatedly under > memory pressure. The second one is a cleanup/consistency fix. Applied to 5.4/scsi-queue, thanks! -- Martin K. Petersen Oracle Linux Engineering

Re: [PATCH v2] tcmu: avoid use-after-free after command timeout

2019-08-14 Thread Martin K. Petersen
Dmitry, > In tcmu_handle_completion() function, the variable called read_len is > always initialized with a value taken from se_cmd structure. If this > function is called to complete an expired (timed out) out command, the > session command pointed by se_cmd is likely to be already deallocated

Re: [PATCH RESEND] qla2xxx: Fix gnl.l memory leak on adapter init failure

2019-08-14 Thread Martin K. Petersen
Bill, > If HBA initialization fails unexpectedly (exiting via probe_failed:), > we may fail to free vha->gnl.l. So that we don't attempt to double > free, set this pointer to NULL after a free and check for NULL at > probe_failed: so we know whether or not to call dma_free_coherent. Applied to

[PATCH 41/42] lpfc: Merge per-protocol WQ/CQ pairs into single per-cpu pair

2019-08-14 Thread James Smart
Currently, each hardware queue, typically allocated per-cpu, consists of a WQ/CQ pair per protocol. Meaning if both SCSI and NVMe are supported 2 WQ/CQ pairs will exist for the hardware queue. Separate queues are unnecessary. The current implementation wastes memory backing the 2nd set of queues, a

[PATCH 32/42] lpfc: Fix BlockGuard enablement on FCoE adapters

2019-08-14 Thread James Smart
The driver is allowing the user to change lpfc_enable_bg while loading the driver against a FCoE adapter. This is not supported. No check is made for the adapter type when applying the blockguard enablement value. Fix by verifying the adapter type before setting the enablement flag. Signed-off-b

[PATCH 27/42] lpfc: Fix nvme sg_seg_cnt display if HBA does not support NVME

2019-08-14 Thread James Smart
The driver is currently reporting a non-zero nvme sg_seg_cnt value of 256 when nvme is disabled. It should be zero. Fix by ensuring the value is cleared. Signed-off-by: Dick Kennedy Signed-off-by: James Smart --- drivers/scsi/lpfc/lpfc_init.c | 1 + 1 file changed, 1 insertion(+) diff --git a

[PATCH 29/42] lpfc: Fix upcall to bsg done in non-success cases

2019-08-14 Thread James Smart
The scsi transport fc bsg interface does not expect the bsg_job_done() callback to be done if the bsg request call returns failure. Several of the HST_VENDOR cases in the driver unconditionally call bsg_job_done() regardless of the returning value. Fix the code to only call bsg_job_done() if the c

[PATCH 36/42] lpfc: Migrate to %px and %pf in kernel print calls

2019-08-14 Thread James Smart
In order to see real addresses, convert %p with %px for kernel addresses and replace %p with %pf for functions. While converting, standardize on "x%px" throughout (not %px or 0x%px). Signed-off-by: Dick Kennedy Signed-off-by: James Smart --- drivers/scsi/lpfc/lpfc_bsg.c | 2 +- drivers/

[PATCH 08/42] lpfc: Fix discovery when target has no GID_FT information

2019-08-14 Thread James Smart
Some remote ports may be slow in registering their GID_FT protocol information with the fabric. If the remote port is an initiator, it may send PLOGI to the port before the GID_FT logic is complete. Meaning, after accepting the PLOGI, when the driver may see no response to the GID_FT that is issued

[PATCH 37/42] lpfc: Add first and second level hardware revisions to sysfs reporting

2019-08-14 Thread James Smart
To aid better hardware detection when there are issues, report the first and second level hardware revisions from the READ_REV command. Add the elements to the existing hardware id string. Signed-off-by: Dick Kennedy Signed-off-by: James Smart --- drivers/scsi/lpfc/lpfc_attr.c | 3 ++- 1 file c

[PATCH 26/42] lpfc: Fix nvme target mode ABTSing a received ABTS

2019-08-14 Thread James Smart
If an unsolicited ABTS was received, the driver looks up the exchange it references. It it does various searches looking for the exchange context. When one is eventually matched and it is associated with an XRI context, the driver sends an ABORT WQE to terminate the exchange. Current code looks at

[PATCH 28/42] lpfc: Fix sli4 adapter initialization with MSI

2019-08-14 Thread James Smart
When forcing the use of MSI (vs MSI-X) the driver is crashing in pci_irq_get_affinity. The driver was not using the new pci_alloc_irq_vectors interface in the MSI path. Fix by using pci_alloc_irq_vectors() with PCI_RQ_MSI in the MSI path. Signed-off-by: Dick Kennedy Signed-off-by: James Smart

[PATCH 07/42] lpfc: Fix port relogin failure due to GID_FT interaction

2019-08-14 Thread James Smart
In cases of remote-port-side cable pull/replug, there happens to be a target that upon replug will send the port a PLOGI, a PRLI, and a LOGO. When this sequence is received by the driver, the PLOGI accepted and a GFT_ID is issued to find the protocol support for the remote port. While the GFT_ID i

[PATCH 33/42] lpfc: Fix nvme first burst module parameter description

2019-08-14 Thread James Smart
modinfo for lpfc_nvme_enable_fb is incorrect. FirstBurst on lpfc target is not fully supported. Update the attribute description Signed-off-by: Dick Kennedy Signed-off-by: James Smart --- drivers/scsi/lpfc/lpfc_attr.c | 9 +++-- 1 file changed, 3 insertions(+), 6 deletions(-) diff --git a

[PATCH 05/42] lpfc: Fix failure to clear non-zero eq_delay after io rate reduction

2019-08-14 Thread James Smart
Unusually high IO latency can be observed with little IO in progress. The latency may remain high regardless of amount of IO and can only be cleared by forcing lpfc_fcp_imax values to non-zero and then back to zero. The driver's eq_delay mechanism that scales the interrupt coalescing based on io c

[PATCH 31/42] lpfc: Fix reported physical link speed on a disabled trunked link

2019-08-14 Thread James Smart
GetTrunkInfo is displaying an incorrect link speed when the link is a trunk and the link has gone down. The driver is not clearing the logical speed as part of the link down transition. Fix by setting the logical speed to UNKNOWN SPEED when the link goes down. Signed-off-by: Dick Kennedy Signed

[PATCH 30/42] lpfc: Fix Max Frame Size value shown in fdmishow output

2019-08-14 Thread James Smart
Max Frame Size value is shown as 34816 in fdmishow from Switch. The driver uses bbRcvSize in common service param which is obtained from the READ_SPARM mailbox command. The bbRcvSize field which is displayed is a three nibble field but the driver is printing a full four nibbles. Fix by masking of

[PATCH 12/42] lpfc: Fix irq raising in lpfc_sli_hba_down

2019-08-14 Thread James Smart
The adapter reset path (lpfc_sli_hba_down) is taking/releasing a lock with irq. But, the path is already under the hbalock which raised irq so it's unnecessary. Convert to simple lock/unlock. Signed-off-by: Dick Kennedy Signed-off-by: James Smart --- drivers/scsi/lpfc/lpfc_sli.c | 4 ++-- 1 fi

[PATCH 38/42] lpfc: Add MDS driver loopback diagnostics support

2019-08-14 Thread James Smart
Added code to support driver loopback with MDS Diagnostics. This style of diagnostics passes frames from the fabric to the driver who then echo them back out the link. SEND_FRAME WQEs are used to transmit the frames. Added the SOF and EOF field location definitions for use by SEND_FRAME. Also en

[PATCH 35/42] lpfc: Add simple unlikely optimizations to reduce NVME latency

2019-08-14 Thread James Smart
While performing code review, several relatively simple optimizations can be done in the fast path. Add these optimizations (unlikely designators). Signed-off-by: Dick Kennedy Signed-off-by: James Smart --- drivers/scsi/lpfc/lpfc_nvme.c | 8 1 file changed, 4 insertions(+), 4 deletion

[PATCH 02/42] lpfc: Fix PLOGI failure with high remoteport count

2019-08-14 Thread James Smart
When connected to a high number of remote ports, the driver is encountering PLOGI errors. The errors are due to adapter detected failures indicating illegal field values. Turns out the driver was prematurely clearing an RPI bitmask before waiting for an UNREG_RPI mailbox completion. This allowed

[PATCH 34/42] lpfc: Fix coverity warnings

2019-08-14 Thread James Smart
Running on Coverity produced the following errors: - coding style (indentation) - memset size mismatch errors note: comment cases where it is purposely a mismatch Fix the errors Signed-off-by: Dick Kennedy Signed-off-by: James Smart --- drivers/scsi/lpfc/lpfc_els.c | 10 +- drivers

[PATCH 22/42] lpfc: Fix deadlock on host_lock during cable pulls

2019-08-14 Thread James Smart
During cable pull testing a deadlock was seen between lpfc_nlp_counters() vs lpfc_mbox_process_link_up() vs lpfc_work_list_done(). They are all waiting on the shost->host_lock. Issue is all of these cases raise irq when taking out the lock but use spin_unlock_irq() when unlocking. The unlock path

[PATCH 25/42] lpfc: Fix hang when downloading fw on port enabled for nvme

2019-08-14 Thread James Smart
As part of firmware download, the adapter is reset. On the adapter the reset causes the function to stop and all outstanding io is terminated (without responses). The reset path then starts teardown of the adapter, starting with deregistration of the remote ports with the nvme-fc transport. The loc

[PATCH 18/42] lpfc: Fix propagation of devloss_tmo setting to nvme transport

2019-08-14 Thread James Smart
If admin changes the devloss_tmo on an rport via the fc_remote_port rport dev_loss_tmo attribute, the value is on set on scsi stack. The change is not propagated to NVMe. The set routine in the lldd lacks the call to nvme_fc_set_remoteport_devloss() to set the value. Fix by adding the call to the

[PATCH 24/42] lpfc: Fix too many sg segments spamming in kernel log

2019-08-14 Thread James Smart
This issue is specific to SLI-3 adapters, specifically when DIF is used. Once seen, this message floods the logs: 9064 BLKGRD: lpfc_scsi_prep_dma_buf_s3: Too many sg segments from dma_map_sg The driver, upon detecting an error such as too many elements in an sglist, misrepresents the error by t

[PATCH 10/42] lpfc: Fix issuing init_vpi mbox on SLI-3 card

2019-08-14 Thread James Smart
The driver is inadvertantly trying to issue an INIT_VPI mailbox command on an SLI-3 driver. The command is specific to SLI-4. When the call is made to send the command, if on an SLI-3 adapter, an array pointer is NULL and the driver will oops. Fix by restricting the command to SLI-4 adapters only.

[PATCH 21/42] lpfc: Fix error in remote port address change

2019-08-14 Thread James Smart
In a test with high nvme remote port counts connected via a multi-hop FC switch config where switches were systematically reset (e.g. fabric partitioning and re-establishment), the nvme remote ports would switch addresses based on the switch reconfiguration events. The driver would get into a situa

[PATCH 09/42] lpfc: Fix ADISC reception terminating login state if a NVME target

2019-08-14 Thread James Smart
If a target issues an ADISC to the port and the target is a NVME target, the driver is inadvertantly invalidating the login and marking the remote port as logged out. Communication with the target is lost. Revise the ADISC check so that FCP or NVME targets will be marked valid at the end of ADISC

[PATCH 42/42] lpfc: Update lpfc version to 12.4.0.0

2019-08-14 Thread James Smart
Update lpfc version to 12.4.0.0 Signed-off-by: Dick Kennedy Signed-off-by: James Smart --- drivers/scsi/lpfc/lpfc_version.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/scsi/lpfc/lpfc_version.h b/drivers/scsi/lpfc/lpfc_version.h index f7e93aaf1e00..b8aae31ffda3 10

[PATCH 39/42] lpfc: Support dynamic unbounded SGL lists on G7 hardware.

2019-08-14 Thread James Smart
Typical SLI-4 hardware supports up to 2 4KB pages to be registered per XRI to contain the exchanges Scatter/Gather List. This caps the number of SGL elements that can be in the SGL. There are not extensions to extend the list out of the 2 pages. The G7 hardware adds a SGE type that allows the SGL

[PATCH 13/42] lpfc: Fix oops when fewer hdwqs than cpus

2019-08-14 Thread James Smart
When tearing down the adapter for a reset, online/offline, or driver unload, the queue free routine would hit a GPF oops. This only occurs on conditions where the number of hardware queues created is fewer than the number of cpus in the system. In this condition cpus share a hardware queue. And of

[PATCH 00/42] lpfc: Update lpfc to revision 12.4.0.0

2019-08-14 Thread James Smart
Update lpfc to revision 12.4.0.0 This patch set contains a laundry list of fixes for lpfc. Most are small. There are a couple of cleanup patches and several functional additions at the tail of the patch set. The patches were cut against Martin's 5.4/scsi-queue tree James Smart (42): lpfc: Lim

[PATCH 15/42] lpfc: Fix null ptr oops updating lpfc_devloss_tmo via sysfs attribute

2019-08-14 Thread James Smart
If an admin updates lpfc's devloss_tmo sysfs attribute, the kernel will oops. Coding of a loop allowed a new value (rport) to be set/checked for null followed by an older value (remoteport) checked for null to allow progress where the new value, even though null, will be referenced. Rework the lo

[PATCH 40/42] lpfc: Add NVMe sequence level error recovery support

2019-08-14 Thread James Smart
FC-NVMe-2 added support for sequence level error recovery in the FC-NVME protocol. This allows for the detection of errors and lost frames and immediate retransmission of data to avoid exchange termination, which escalates into NVMeoFC connection and association failures. A significant RAS improvem

[PATCH 23/42] lpfc: Fix crash due to port reset racing vs adapter error handling

2019-08-14 Thread James Smart
If the adapter encounters a condition which causes the adapter to fail (driver must detect the failure) simultaneously to a request to the driver to reset the adapter (such as a host_reset), the reset path will be racing with the asynchronously-detect adapter failure path. In the failing situation

[PATCH 19/42] lpfc: Fix sg_seg_cnt for HBAs that don't support NVME

2019-08-14 Thread James Smart
On an SLi-3 adapter which does not support NVMe, but with the driver global attribute to enable nvme on any adapter if it does support NVMe (e.g. module parameter lpfc_enable_fc4_type=3), the SGL and total SGE values are being munged by the protocol enablement when it shouldn't be. Correct by chan

[PATCH 01/42] lpfc: Limit xri count for kdump environment

2019-08-14 Thread James Smart
SCSI-MQ operation inherently performs pre-allocation of resources for blk-mq request queues. Even though the kdump environment reduces the configuration to a single cpu, thus 1 hardware queue, which helps significantly, the resources are still rather large due to the per request allocations. Blk-mq

[PATCH 17/42] lpfc: Fix loss of remote port after devloss due to lack of RPIs

2019-08-14 Thread James Smart
In tests with remote ports contantly logging out/logging coupled with occassional local link bounce, if a remote port is disocnnected for longer than devloss_tmo and then subsequently reconnected, eventually the test will fail to login with the remote port and remote port connectivity is lost. Whe

[PATCH 14/42] lpfc: Fix FLOGI handling across multiple link up/down conditions

2019-08-14 Thread James Smart
It's possible for the driver to initiate an FLOGI and before it completes, another link down/up transition occurs requiring a new FLOGI. Currently, nothing is done to abort/noop the older FLOGI request to the adapter, so if this transition occurs and the FLOGI completion is received after the link

[PATCH 16/42] lpfc: Fix devices that don't return after devloss followed by rediscovery

2019-08-14 Thread James Smart
If a remote port is removed and remains removed for devloss_tmo, if an RSCN is subsequently received indicating the presence of the remte port, the driver does not login to and rediscovery the remote port. Currently, in order to for a port to be rediscovered post an RSCN, the node state must be NP

[PATCH 11/42] lpfc: Fix Oops in nvme_register with target logout/login

2019-08-14 Thread James Smart
lpfc_nvme_register_port hit a null prev_ndlp pointer in a test with lots of target ports swapping addresses. The oldport value was stale, thus it's ndlp (prev_ndlp set to it) was used. Fix by moving oldrport pointer checks, and if used prev_ndlp pointer assignment, to be done while the lock is hel

[PATCH 04/42] lpfc: Fix crash on driver unload in wq free

2019-08-14 Thread James Smart
If a timer routine uses workqueues, it could fire before the workqueue is allocated. Fix by allocating the workqueue before the timer routines are setup Signed-off-by: Dick Kennedy Signed-off-by: James Smart --- drivers/scsi/lpfc/lpfc_init.c | 11 +-- 1 file changed, 5 insertions(+), 6

[PATCH 20/42] lpfc: Fix driver nvme rescan logging

2019-08-14 Thread James Smart
In situations where zoning is not being used, thus NVMe initiators see other NVMe initiators as well as NVMe targets, a link bounce on an initiator will cause the NVMe initiators to spew "6169" State Error messages. The driver is not qualifying whether the remote port is a NVMe targer or not befor

[PATCH 03/42] lpfc: Fix ELS field alignments

2019-08-14 Thread James Smart
After seeing some interoperability issues with ADISC, it was determined the ELS definitions in lpfc were using types that allowed the compiler to add pad to the structure, causing the structure to no longer be per spec. The offending structures are ADISC, FAN, and RNID. This patch implements the s

[PATCH 06/42] lpfc: Fix leak of ELS completions on adapter reset

2019-08-14 Thread James Smart
If the adapter is reset while there are outstanding ELS's, subsequent reinitialization of the adapter will fail as it has not recovered all of the io contexts relative to the ELS's. If an ELS timed out or otherwise failed and an the ELS was attempted to be aborted (which changes the ELS completion

Re: [EXT] [PATCH v2 0/2] scsi: qla2xxx: fixes for FW trace/dump buffers

2019-08-14 Thread Himanshu Madhani
Hi Martin W, On 8/14/19, 8:52 AM, "Martin Wilck" wrote: External Email -- From: Martin Wilck Hi Himanshu, hi Martin, Please consider this series for merging. The first patch of the

TODAY, Wed, Aug 14, 2019 I AM READY FOR COMING TO YOUR ADDRESS WITH THIS ATM CARD

2019-08-14 Thread MS. MARYANNA B. THOMASON
ATTN DEAR PARCEL BENEFICIARY. I AM CATHY JONES,DIPLOMATIC AGENT ASIGNED ON THE DELIVERY OF YOUR ATM CARD THROUGH MS. MARYANNA B. THOMASON, DHL MANAGEMENT DIRECTOR NEW YORK. TODAY, Wed, Aug 14, 2019 I AM READY FOR COMING TO YOUR ADDRESS WITH THIS ATM CARD, So before i deliver I want you to send me.

[Bug 204119] scsi_mod: Could not allocate 4104 bytes percpu data

2019-08-14 Thread bugzilla-daemon
https://bugzilla.kernel.org/show_bug.cgi?id=204119 --- Comment #10 from Bart Van Assche (bvanass...@acm.org) --- This patch has been accepted in Martin's tree as commit dccc96abfb21 ("scsi: core: Reduce memory required for SCSI logging") and is on its way to kernel v5.4. If you need that patch in

[PATCH v5 00/18] compat_ioctl.c removal, part 2/3

2019-08-14 Thread Arnd Bergmann
This is a follow-up to part 1/3 that I posted after -rc2. I hope these are still largely uncontroversial changes, and I would like to get them into linux-5.4. Part 1 was in https://lore.kernel.org/lkml/capcyv4i_nhzv155rcgnaq189aq2lfd2g8pa1d5nbzqo9e_u...@mail.gmail.com/ Part 3 will be one kernel

Re: [PATCH] scsi: qla2xxx: replace snprintf with strscpy

2019-08-14 Thread Himanshu Madhani
On 8/14/19, 10:25 AM, "linux-scsi-ow...@vger.kernel.org on behalf of Bart Van Assche" wrote: On 7/24/19 10:46 PM, Wang Xiayang wrote: > As commit a86028f8e3ee ("staging: most: sound: replace snprintf > with strscpy") suggested, using snprintf without a format specifier > is p

Re: [PATCH RESEND] qla2xxx: Fix gnl.l memory leak on adapter init failure

2019-08-14 Thread Himanshu Madhani
On 8/14/19, 9:25 AM, "linux-scsi-ow...@vger.kernel.org on behalf of Bill Kuzeja" wrote: If HBA initialization fails unexpectedly (exiting via probe_failed:), we may fail to free vha->gnl.l. So that we don't attempt to double free, set this pointer to NULL after a free and check f

Re: [PATCH] scsi: qla2xxx: replace snprintf with strscpy

2019-08-14 Thread Bart Van Assche
On 7/24/19 10:46 PM, Wang Xiayang wrote: As commit a86028f8e3ee ("staging: most: sound: replace snprintf with strscpy") suggested, using snprintf without a format specifier is potentially risky if a0->vendor_name or a0->vendor_pn mistakenly contain format specifiers. In addition, as compared in t

[PATCH RESEND] qla2xxx: Fix gnl.l memory leak on adapter init failure

2019-08-14 Thread Bill Kuzeja
If HBA initialization fails unexpectedly (exiting via probe_failed:), we may fail to free vha->gnl.l. So that we don't attempt to double free, set this pointer to NULL after a free and check for NULL at probe_failed: so we know whether or not to call dma_free_coherent. Signed-off-by: Bill Kuzeja

Re: [PATCH 2/3] scsi: qla2xxx: unset RCE/EFT fields in failure case

2019-08-14 Thread Himanshu Madhani
On 8/14/19, 6:30 AM, "linux-scsi-ow...@vger.kernel.org on behalf of Martin Wilck" wrote: On Wed, 2019-08-14 at 08:24 +0200, Hannes Reinecke wrote: > On 8/13/19 10:31 PM, Martin Wilck wrote: > > From: Martin Wilck > > > > Reset ha->rce, ha->eft and the respective dma fie

Re: [PATCH] scsi: qla2xxx: replace snprintf with strscpy

2019-08-14 Thread Himanshu Madhani
On 7/25/19, 12:54 AM, "linux-scsi-ow...@vger.kernel.org on behalf of Wang Xiayang" wrote: As commit a86028f8e3ee ("staging: most: sound: replace snprintf with strscpy") suggested, using snprintf without a format specifier is potentially risky if a0->vendor_name or a0->vendor_pn mi

Re: [PATCH] qla2xxx: Fix gnl.l memory leak on adapter init failure

2019-08-14 Thread Himanshu Madhani
On 7/16/19, 1:57 PM, "linux-scsi-ow...@vger.kernel.org on behalf of Bill Kuzeja" wrote: If HBA initialization fails unexpectedly (exiting via probe_failed:), we may fail to free vha->gnl.l. So that we don't attempt to double free, set this pointer to NULL after a free and check f

[PATCH v2 0/2] scsi: qla2xxx: fixes for FW trace/dump buffers

2019-08-14 Thread Martin Wilck
From: Martin Wilck Hi Himanshu, hi Martin, Please consider this series for merging. The first patch of the series is a fix for a memory corruption we saw in a test where qla2xxx was loaded/unloaded repeatedly under memory pressure. The second one is a cleanup/consistency fix. Regards, Martin

[PATCH v2 2/2] scsi: qla2xxx: cleanup trace buffer initialization

2019-08-14 Thread Martin Wilck
From: Martin Wilck Avoid code duplication between qla2x00_alloc_offload_mem() and qla2x00_alloc_fw_dump() by moving the FCE and EFT buffer allocation and initialization to separate functions. Cleanly track failure and success by making sure that the ha->eft, ha->fce and respective eft_dma, fce_dm

[PATCH v2 1/2] scsi: qla2xxx: qla2x00_alloc_fw_dump: set ha->eft

2019-08-14 Thread Martin Wilck
From: Martin Wilck In qla2x00_alloc_fw_dump(), an existing EFT buffer (e.g. from previous invocation of qla2x00_alloc_offload_mem()) is freed. The buffer is then re-allocated, but without setting the eft and eft_dma fields to the new values. Fixes: a28d9e4ef997 "scsi: qla2xxx: Add support for mu

Re: [PATCH 2/3] scsi: qla2xxx: unset RCE/EFT fields in failure case

2019-08-14 Thread Martin Wilck
On Wed, 2019-08-14 at 08:24 +0200, Hannes Reinecke wrote: > On 8/13/19 10:31 PM, Martin Wilck wrote: > > From: Martin Wilck > > > > Reset ha->rce, ha->eft and the respective dma fields if > > the buffers aren't mapped for some reason. Also, treat > > both failure cases (allocation and initializat

PLEASE CONFIRM PURCHASE ORDER

2019-08-14 Thread Mr NARESH KUMAR
Could you please confirm if your recieved our purchase order last week. If no please confirm let me resend it to you. NARESH KUMAR Executive Purchase Saiapextrading Ltd Dubai, KSA. (T/F): +96-2667-264 777 / 778 (Mo): +96 94284 02803 Website - http://www.saiapexgeneraltrading.com