Ability to set max TTL added to PR, please recheck.

T

On Mon, Mar 27, 2023 at 1:22 PM Michael Vitz
<vitz.mich...@googlemail.com.invalid> wrote:

> From my perspective, disabling the pooling (like the PR does) OR having the
> ability to set a time to live (TTL) for the pooled connections, should be
> good enough for tackling the current problem.
>
> Michael
>
>
> Am Mo., 27. März 2023 um 12:41 Uhr schrieb Tamás Cservenák <
> ta...@cservenak.net>:
>
> > Created https://github.com/apache/maven-resolver/pull/274
> >
> > T
> >
> > On Mon, Mar 27, 2023 at 11:04 AM Tamás Cservenák <ta...@cservenak.net>
> > wrote:
> >
> > > In this case, is "configuration of pool" (I guess you mean like sizes
> > etc)
> > > really needed?
> > > Would "use pool" (on/off) enough instead?
> > >
> > > T
> > >
> > > On Mon, Mar 27, 2023 at 10:54 AM Michael Vitz
> > > <vitz.mich...@googlemail.com.invalid> wrote:
> > >
> > >> Yes, it seems the isStale-Implementation of BHttpConnectionBase
> > >>
> > >> ```
> > >>     @Override
> > >>     public boolean isStale() throws IOException {
> > >>         if (!isOpen()) {
> > >>             return true;
> > >>         }
> > >>         try {
> > >>             final int bytesRead =
> > >> fillInputBuffer(Timeout.ofMilliseconds(1));
> > >>             return bytesRead < 0;
> > >>         } catch (final SocketTimeoutException ex) {
> > >>             return false;
> > >>         } catch (final SocketException ex) {
> > >>             return true;
> > >>         }
> > >>     }
> > >> ```
> > >>
> > >> does not work correctly with the way Azure intercepts these
> connections.
> > >>
> > >> What's the way to improve the situation? Opening a bug there and
> waiting
> > >> for it to be fixed? Or is the option to allow configuration of the
> pool,
> > >> TTL and or Disabling the pool, like wagon did feasible?
> > >>
> > >> Am Mo., 27. März 2023 um 09:58 Uhr schrieb Konrad Windszus <
> > >> konra...@gmx.de
> > >> >:
> > >>
> > >> > Thanks for the reproducer and sorry for not reading thoroughly
> before.
> > >> > Indeed the connection pool is not configurable yet with the native
> > >> > connector. According to
> > >> >
> > >>
> >
> https://hc.apache.org/httpcomponents-client-4.5.x/current/httpclient/apidocs/org/apache/http/impl/conn/PoolingHttpClientConnectionManager.html
> > >> > stale connections are checked after 2 seconds before being reused
> but
> > >> > probably the check does not work for some reason with Azure….
> > >> >
> > >> > > Am 27.03.2023 um 09:49 schrieb Michael Vitz <
> > >> vitz.mich...@googlemail.com
> > >> > .invalid>:
> > >> > >
> > >> > > Hi Konrad,
> > >> > >
> > >> > > The only two timeout related properties are
> > >> > aether.connector.requestTimeout
> > >> > > and aether.connector.connectTimeout.
> > >> > > Both do not solve my problem.
> > >> > >
> > >> > > I created a GitHub project (
> > >> > > https://github.com/mvitz/maven-39-gh-actions-timeouts) which
> > >> reproduces
> > >> > the
> > >> > > problem.
> > >> > > It declares many dependencies/plugins to make sure the complete
> HTTP
> > >> pool
> > >> > > is used before the tests, contains a test that runs in total
> about 6
> > >> > > minutes to take longer than the four-minute Azure timeout and uses
> > >> Maven
> > >> > > 3.9.1.
> > >> > >
> > >> > > The first build (
> > >> > >
> > >> >
> > >>
> >
> https://github.com/mvitz/maven-39-gh-actions-timeouts/actions/runs/4524745997/jobs/7968775993
> > >> > )
> > >> > > uses -Daether.connector.requestTimeout=270000 which is longer than
> > >> four
> > >> > > minutes and fails.
> > >> > > The second one (
> > >> > >
> > >> >
> > >>
> >
> https://github.com/mvitz/maven-39-gh-actions-timeouts/actions/runs/4524925829/jobs/7969075725
> > >> > )
> > >> > > uses -Daether.connector.connectTimeout=60000
> > >> > > -Daether.connector.requestTimeout=180000 which is shorter but
> fails,
> > >> too.
> > >> > >
> > >> > > Both builds fail with:
> > >> > > Error:  Failed to execute goal
> > >> > > org.apache.maven.plugins:maven-jar-plugin:3.3.0:jar (default-jar)
> on
> > >> > > project maven-39-gh-actions-timeouts: Execution default-jar of
> goal
> > >> > > org.apache.maven.plugins:maven-jar-plugin:3.3.0:jar failed: Plugin
> > >> > > org.apache.maven.plugins:maven-jar-plugin:3.3.0 or one of its
> > >> > dependencies
> > >> > > could not be resolved: Failed to collect dependencies at
> > >> > > org.apache.maven.plugins:maven-jar-plugin:jar:3.3.0 ->
> > >> > > org.apache.maven.shared:file-management:jar:3.1.0: Failed to read
> > >> > artifact
> > >> > > descriptor for org.apache.maven.shared:file-management:jar:3.1.0:
> > The
> > >> > > following artifacts could not be resolved:
> > >> > > org.apache.maven.shared:file-management:pom:3.1.0 (absent): Could
> > not
> > >> > > transfer artifact
> org.apache.maven.shared:file-management:pom:3.1.0
> > >> > from/to
> > >> > > central (https://repo.maven.apache.org/maven2): Read timed out ->
> > >> [Help
> > >> > 1]
> > >> > >
> > >> > > I suspect that the HTTP pool is fully used before the tests, all
> > >> > > connections are silently interrupted during the test run that
> takes
> > >> > longer
> > >> > > than the Azure limit of four minutes, and the pool unable to
> recover
> > >> any
> > >> > > connection from that.
> > >> > >
> > >> > > As told before when using wagon with
> > -Dmaven.resolver.transport=wagon
> > >> and
> > >> > > configuring a pool time to love of 3 minutes via
> > >> > > -Dmaven.wagon.httpconnectionManager.ttlSeconds=180 everything
> works
> > as
> > >> > > expected (
> > >> > >
> > >> >
> > >>
> >
> https://github.com/mvitz/maven-39-gh-actions-timeouts/actions/runs/4529868249/jobs/7978093099
> > >> > > ).
> > >> > >
> > >> > > Regards,
> > >> > > Michael
> > >> > >
> > >> > >
> > >> > >> Am So., 26. März 2023 um 12:54 Uhr schrieb Konrad Windszus <
> > >> > konra...@gmx.de
> > >> > >>> :
> > >> > >>
> > >> > >> The timeouts are configurable even with new native connector:
> > >> > >> https://maven.apache.org/resolver/configuration.html.
> > >> > >> Please try if those settings help.
> > >> > >> Konrad
> > >> > >>
> > >> > >>> Am 25.03.2023 um 15:08 schrieb Michael Vitz <
> > >> > vitz.mich...@googlemail.com
> > >> > >> .invalid>:
> > >> > >>>
> > >> > >>> Hi all,
> > >> > >>>
> > >> > >>> We recently switched from Maven 3.8.x to 3.9.x and all of a
> > sudden,
> > >> we
> > >> > >> ran
> > >> > >>> into connection timeouts during downloading the maven-jar-plugin
> > >> after
> > >> > >> all
> > >> > >>> tests passed.
> > >> > >>>
> > >> > >>> After some digging, I suspect that it is a combination of GitHub
> > >> > Actions
> > >> > >>> running on Azure which silently drops open connections after
> > around
> > >> > four
> > >> > >>> minutes (see discussion in
> > >> > >>> https://github.com/actions/runner-images/issues/1499) and the
> > >> switch
> > >> > to
> > >> > >> the
> > >> > >>> native HTTP transport in Maven 3.9.x.
> > >> > >>> Our tests take quite some time and because the maven-jar-plugin
> is
> > >> > >> freshly
> > >> > >>> downloaded after these, the used connection was opened before
> the
> > >> tests
> > >> > >>> were started and is dropped by Azure meanwhile.
> > >> > >>>
> > >> > >>> Our current workaround is switching to the old Wagon HTTP
> provider
> > >> > (with
> > >> > >>> -Dmaven.resolver.transport=wagon) and setting a TTL for the used
> > >> HTTP
> > >> > >> pool
> > >> > >>> (-Dmaven.wagon.httpconnectionManager.ttlSeconds=180) or
> disabling
> > >> the
> > >> > >> pool
> > >> > >>> and keep alive (-Dhttp.keepAlive=false
> > >> -Dmaven.wagon.http.pool=false).
> > >> > >>>
> > >> > >>> Unfortunately, the new HTTP transport does not allow changing
> the
> > >> TTL
> > >> > >>> (which by default is -1 which means forever) or disabling it
> > >> > altogether.
> > >> > >> It
> > >> > >>> would be nice if such settings would be added in one of the next
> > >> > >> releases.
> > >> > >>>
> > >> > >>> I would be happy to help/try to provide a patch on my own if
> this
> > >> > helps.
> > >> > >>>
> > >> > >>> Regards,
> > >> > >>> Michael
> > >> > >>
> > >> >
> > >>
> > >
> >
>

Reply via email to