Re: SAML with Okta is very slow

2020-09-30 Thread Mark Schroering
It ended up not being an IO issue. We use https://www.jenkins.io/projects/jcasc/ and the official Jenkins docker image to deploy it to AWS ECS. We have a startup script that does some cleanup in the mounted jenkins home directory to make sure that updated plugins are installed properly. We n

Re: SAML with Okta is very slow

2020-09-25 Thread kuisathaverat
as I said your problem is the IO, if you enter en the Instance by ssh and check the iostats you will see more than 5-10% of your operations waiting for IO. NFS, EFS, and in general network filesystems works well with big files but with small files and write concurrence is where the problems start

Re: SAML with Okta is very slow

2020-09-25 Thread Mark Schroering
We are using AWS EFS for the Jenkins Home mount. It was configured for burst throughput, and after reading https://aws.amazon.com/blogs/storage/best-practices-for-using-amazon-efs-for-container-storage/ we just changed it to provisioned throughput of 150 MiB/s. The change did not help wit

Re: SAML with Okta is very slow

2020-09-24 Thread Ivan Fernandez Calvo
Is your Jenkins home in a NFS or other network storage? I think so for the mount point, when a user enter a few files are written, because your IO is slow the IO operations are blocked waiting to finish that make the login slower than expected. You probably has more performance issues, I usually

Re: SAML with Okta is very slow

2020-09-24 Thread Mark Schroering
Here are the logs in a better format. Sep 24, 2020 7:52:17 AM FINE org.pac4j.saml.client.SAML2Client retrieveUserProfileAdding attribute value mark.schroering@*.com for attribute null Sep 24, 2020 7:52:17 AM FINE org.pac4j.core.profile.UserProfile addAttributeno conversion => key: email

SAML with Okta is very slow

2020-09-24 Thread Mark Schroering
We have noticed it taking a very long time (up to 60s) to complete the SAML auth flow. Here are some logs showing the bigger time gaps. We are on version 1.1.7 of the SAML plugin and running Jenkins version 2.257. Sep 24, 2020 7:52:17 AM FINE org.pac4j.saml.client.SAML2Client retrieveUserPr