Re: nfstimeout on server ISILON storage

2018-09-12 Thread Grant Street
Grant From: ADSM: Dist Stor Manager on behalf of Zoltan Forray Sent: Friday, 7 September 2018 4:22 AM To: ADSM-L@VM.MARIST.EDU Subject: Re: [ADSM-L] nfstimeout on server ISILON storage >>> Are the timeouts repeatable enough that you can get a packet ca

Re: nfstimeout on server ISILON storage

2018-09-12 Thread Grant Street
or experience Grant From: ADSM: Dist Stor Manager on behalf of Frank Kraemer Sent: Friday, 7 September 2018 7:50 AM To: ADSM-L@VM.MARIST.EDU Subject: Re: [ADSM-L] nfstimeout on server ISILON storage Isilon = Slow Performance - Although a parallel filesystem inside (OneFS),

Re: nfstimeout on server ISILON storage

2018-09-06 Thread Frank Kraemer
Isilon = Slow Performance - Although a parallel filesystem inside (OneFS), each client node can only talk to a single Isilon node using standard NAS protocols, which then performs  parallel I/O across the internal high speed IB network to other Storage Nodes. - NFS Client nodes (=TSM Server) have

Re: nfstimeout on server ISILON storage

2018-09-06 Thread Zoltan Forray
>>> Are the timeouts repeatable enough that you can get a packet capture in there before and while they're happening? They happen often/sometimes all-the-time if there is any kind of storagepool activity. Looking through /var/log/messages - it happened almost every 5-minutes starting before from

Re: [EXTERNAL] Re: nfstimeout on server ISILON storage

2018-09-06 Thread Rick Adamson
- From: ADSM: Dist Stor Manager On Behalf Of Zoltan Forray Sent: Wednesday, September 5, 2018 3:07 PM To: ADSM-L@VM.MARIST.EDU Subject: Re: [ADSM-L] [EXTERNAL] Re: nfstimeout on server ISILON storage * This email originated outside of the organization. Use caution when opening attachments or

Re: [EXTERNAL] Re: nfstimeout on server ISILON storage

2018-09-06 Thread Rhodes, Richard L.
M: Dist Stor Manager On Behalf Of Zoltan > Forray > Sent: Wednesday, September 5, 2018 9:03 AM > To: ADSM-L@VM.MARIST.EDU > Subject: [EXTERNAL] Re: nfstimeout on server ISILON storage > > A pair of 10G links bonded - CISCO switches. > > On Tue, Sep 4, 2018 at 7:54 PM Skylar Thompso

Re: nfstimeout on server ISILON storage

2018-09-05 Thread Skylar Thompson
Yep, you're right, I misread that (shouldn't send email pre-coffee). Are the timeouts repeatable enough that you can get a packet capture in there before and while they're happening? On Wed, Sep 05, 2018 at 07:09:09PM -0400, Zoltan Forray wrote: > Skylar, > > I sent your comment about UDP vs TDP

Re: nfstimeout on server ISILON storage

2018-09-05 Thread Zoltan Forray
Skylar, I sent your comment about UDP vs TDP to my OS tech (beyond my ken) - got this feedback: I assume what they are talking about is this: hhisilonnfs23.rams.adp.vcu.edu:/ifs/NFS/TSM on /tsmnfs type nfs (rw,relatime,sync,vers=3,rsize=131072,wsize=524288,namlen=255,hard,proto=tcp,timeo=600,ret

Re: [EXTERNAL] Re: nfstimeout on server ISILON storage

2018-09-05 Thread Zoltan Forray
half Of Zoltan > Forray > Sent: Wednesday, September 5, 2018 9:03 AM > To: ADSM-L@VM.MARIST.EDU > Subject: [EXTERNAL] Re: nfstimeout on server ISILON storage > > A pair of 10G links bonded - CISCO switches. > > On Tue, Sep 4, 2018 at 7:54 PM Skylar Thompson wrote: > >

Re: [EXTERNAL] Re: nfstimeout on server ISILON storage

2018-09-05 Thread Rhodes, Richard L.
ill a bug in LACP support. Rick -Original Message- From: ADSM: Dist Stor Manager On Behalf Of Zoltan Forray Sent: Wednesday, September 5, 2018 9:03 AM To: ADSM-L@VM.MARIST.EDU Subject: [EXTERNAL] Re: nfstimeout on server ISILON storage A pair of 10G links bonded - CISCO switches. O

Re: nfstimeout on server ISILON storage

2018-09-05 Thread Skylar Thompson
It looks like you're using UDP as a transport - have you tried switching to TCP? Especially with large NFS payload sizes, you're going to get lots of fragmentation with UDP's 512-byte packet limit. On Wed, Sep 05, 2018 at 09:03:25AM -0400, Zoltan Forray wrote: > A pair of 10G links bonded - CISCO

Re: nfstimeout on server ISILON storage

2018-09-05 Thread Zoltan Forray
A pair of 10G links bonded - CISCO switches. On Tue, Sep 4, 2018 at 7:54 PM Skylar Thompson wrote: > Quick question - what's the data link protocol (Ethernet, IB, etc.) and > link rate > that you're using? > > On Tue, Sep 04, 2018 at 02:05:33PM -0400, Zoltan Forray wrote: > > We are still fighti

Re: nfstimeout on server ISILON storage

2018-09-04 Thread Skylar Thompson
Quick question - what's the data link protocol (Ethernet, IB, etc.) and link rate that you're using? On Tue, Sep 04, 2018 at 02:05:33PM -0400, Zoltan Forray wrote: > We are still fighting issues with ISILON storage. Our current issue is with > NFS timeouts for the storage a server is using. We s

nfstimeout on server ISILON storage

2018-09-04 Thread Zoltan Forray
We are still fighting issues with ISILON storage. Our current issue is with NFS timeouts for the storage a server is using. We see message like these in the server /var/log Sep 4 13:21:49 earth kernel: nfs: server hhisilonnfs23.rams.adp.vcu.edu not responding, still trying Sep 4 13:21:49 earth