Re: [Math] Review of "genetic algorithm" module

[Gilles Sadowski]

Hello.

Le ven. 18 févr. 2022 à 11:32, Avijit Basak  a écrit :
>
> Hi All
>
> Please see my comments below.
>
>> [...]
>
> --I have modified the exception class name to
> GeneticIllegalArgumentException.

The build fails because of CheckStyle errors:
https://app.travis-ci.com/github/apache/commons-math/builds/246683712

>>> [...]
> >> >
> >> >I did not suggest to remove any Javadoc, only to rephrase it as:
> >> >---CUT---
> >> >/** "Message template". */
> >> >---CUT---
> >> >
> >> -- Just for confirmation. Do you need the comment to be changed from
> >> ---CUT---
> >> /** Error message for "out of range" condition. */
> >> ---CUT---
> >> *to*
> >> ---CUT---
> >> /** "out of range" */
> >> ---CUT---
> >
> >No, sorry, I just meant that e.g.
> >---CUT---
> >/** Message template. */
> >private static final String OUT_OF_RANGE = "Out of range: min={0} max={1}";
> >---CUT---
> >is sufficient. Since the field is private, the comment will mostly ever
> >be read while looking at the source code.
>
> --I think we have a misunderstanding here. The field is public, not
> private.

Sorry, my mistake.

> Will a hard coded "Message Template" comment be sufficient?

You are right, better be somewhat more specific.
However, it would even better if we can manage to keep
the internal classes private. [This can be explored further
when the design has stabilized.]

>
> >>   JDK(*ProviderBuilder.RandomSourceInternal.JDK*),
> >> ...
> >> }
> >> ---CUT---
> >> I am not sure how to achieve this for our algorithm classes.
> >
> >I'm a bit lost here; I don't get the relationship of this with my
> >remark above (constant vs adaptive)...  Maybe you could file
> >JIRA report to clarify and further discussion?
>
> --Let me try to clarify my original understanding. I think the proposal is
> to introduce an Enum and declare each type of algorithm classes as
> instances of enum. So it might follow this template:
> --CUT--
> enum GeneticAlgorithms {
> SGA([args...]),//Simple genetic algorithm equivalent to
> GeneticAlgorithm class
> AGA([args...]) //Adaptive genetic algorithm.
> }
> --CUT--
> Where [args...] are selection, crossover, mutation operators and
> probalities.

I didn't mean anything like that; AFAICT there may have
been some mixing between two comments. ;-)

>
> As the AdaptiveGeneticAlgorithm is a generalization of GeneticAlgorithm it
> would be possible to represent the later by former using constant crossover
> and mutation rate generators.
> But the problem lies in a different area.
> In enumeration the required arguments ([args...]) need to be specified
> during the declaration using constant values.
> But for our implementation those arguments need to be provided by the
> client programs. I don't know how to achieve that using enum.
> Let me know if you have any different views regarding this.

We can revisit this later.

> >> [...]
> >
> >As hinted by my comment is the previous message, I've still to
> >clarify my own expectations; but I vaguely sense some lost
> >opportunity for simpler usage simpler and increased performance
> >through the caller just needs to specify the number of "worker
> >threads".
> -- We should have both options. Users can execute the algorithm by
> specifying only the number of worker threads with a single population as
> well as optimize multiple populations in a parallel fashion.

Another misunderstanding (probably); we must figure out where
the parallelism will be implemented.
IIUC the current state of the code, optimizing multiple populations
in parallel would be the same as launching multiple JVMs; I'd want
to explore low-level parallelism (i.e. at the "Chromosome" level).

> For the multi-population option the common thread pool would be reused for
> all populations.
> >
> >Do we at least agree that
> >1. Adding/retrieving a "Chromosome" to/from a "Population"
> >must be thread-safe (and is not trivial)
> >2. Fitness computation is where most time is usually spent
> >(so that multi-threading must be achieved at that granularity)
> >?
> --The way I am thinking of designing a task is that it should accept the
> current population and return an instance of ChromosomePair.
> The chromosomes within this pair would be added to the population by the
> caller thread. Population won't be updated by multiple threads.

Then, it would not be a multi-threaded library.
This kind of parallelism does not need support from the library,
and can be implemented at the application level.

> The code snippet below shows the body of the method which will be executed
> inside the task.
> --CUT--
>
> //selection
> ChromosomePair pair = getSelectionPolicy().select(current);
>
> // crossover
> if (randGen.nextDouble() < getCrossoverRate()) {
> // apply crossover policy to create two offspringoport
> pair = getCrossoverPolicy().crossover(pair.getFirst(), pair.getSecond());
> }
>
> // mutation
> if (randGen.nextDouble() < getMutationRate()) {
> // apply mutation policy to the chromosomes
> pair = new C

Fwd: Build failed in Jenkins: Commons » commons-codec #138

[Gary Gregory]

Is this a Jenkins issue?

[INFO] --- maven-compiler-plugin:3.8.1:compile (default-compile) @
commons-codec ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 73 source files to
/home/jenkins/workspace/Commons/commons-codec/target/classes
[WARNING] Attempt to (de-)serialize anonymous class
org.jfrog.hudson.maven2.MavenDependenciesRecorder$1; see:
https://jenkins.io/redirect/serialization-of-anonymous-classes/

-- Forwarded message -
From: Apache Jenkins Server 
Date: Sat, Feb 19, 2022, 20:12
Subject: Build failed in Jenkins: Commons » commons-codec #138
To: , , <
boa...@gmail.com>


See <
https://ci-builds.apache.org/job/Commons/job/commons-codec/138/display/redirect?page=changes
>

Changes:

[github] CODEC-285 replace JUnit v4 test expected with assertThrows (#112)

[github] Avoid C-style array declaration and use Java instead. (#108)

[github] Java 8 improvements: (#106)


--
Started by an SCM change
Running as SYSTEM
[EnvInject] - Loading node environment variables.
Building remotely on builds31 (ubuntu) in workspace <
https://ci-builds.apache.org/job/Commons/job/commons-codec/ws/>
The recommended git tool is: NONE
No credentials specified
 > git rev-parse --resolve-git-dir <
https://ci-builds.apache.org/job/Commons/job/commons-codec/ws/.git> #
timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/apache/commons-codec #
timeout=10
Fetching upstream changes from https://github.com/apache/commons-codec
 > git --version # timeout=10
 > git --version # 'git version 2.17.1'
 > git fetch --tags --progress -- https://github.com/apache/commons-codec
+refs/heads/*:refs/remotes/origin/* # timeout=10
 > git rev-parse refs/remotes/origin/master^{commit} # timeout=10
Checking out Revision fb84f6d9732d44cda0edf449f7a5cba0b340731a
(refs/remotes/origin/master)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f fb84f6d9732d44cda0edf449f7a5cba0b340731a # timeout=10
Commit message: "Java 8 improvements: (#106)"
 > git rev-list --no-walk 438b54b2075652215a75a834ee64ec5b0b29a6f8 #
timeout=10
[EnvInject] - Injecting environment variables from a build step.
Setting JDK_1_8_LATEST_HOME=/home/jenkins/tools/java/latest1.8
[EnvInject] - Injecting as environment variables the properties content
JAVA_1_7_HOME=${JDK_1_7_LATEST_HOME}

[EnvInject] - Variables injected successfully.
Setting JDK_1_8_LATEST_HOME=/home/jenkins/tools/java/latest1.8
Parsing POMs
Established TCP socket on 44177
maven35-agent.jar already up to date
maven35-interceptor.jar already up to date
maven3-interceptor-commons.jar already up to date
[commons-codec] $ /home/jenkins/tools/java/latest1.8/bin/java -cp
/home/jenkins/maven35-agent.jar:/home/jenkins/tools/maven/latest3/boot/plexus-classworlds-2.6.0.jar:/home/jenkins/tools/maven/latest3/conf/logging
jenkins.maven3.agent.Maven35Main /home/jenkins/tools/maven/latest3/
/home/jenkins/agent.jar /home/jenkins/maven35-interceptor.jar
/home/jenkins/maven3-interceptor-commons.jar 44177
<===[JENKINS REMOTING CAPACITY]===>   channel started
Executing Maven:  -B -f <
https://ci-builds.apache.org/job/Commons/job/commons-codec/ws/pom.xml>
clean package deploy -Dgpg.skip -Pjava-1.8
[INFO] Scanning for projects...
[HUDSON] Collecting dependencies info
[INFO]
[INFO] < commons-codec:commons-codec
>-
[INFO] Building Apache Commons Codec 1.17-SNAPSHOT
[INFO] [ jar
]-
[INFO]
[INFO] --- maven-clean-plugin:3.1.0:clean (default-clean) @ commons-codec
---
[INFO] Deleting <
https://ci-builds.apache.org/job/Commons/job/commons-codec/ws/target>
[INFO]
[INFO] --- maven-enforcer-plugin:3.0.0-M3:enforce (enforce-maven-version) @
commons-codec ---
[INFO]
[INFO] --- maven-enforcer-plugin:3.0.0-M3:enforce (enforce-maven-3) @
commons-codec ---
[INFO]
[INFO] --- apache-rat-plugin:0.13:check (rat-check) @ commons-codec ---
[INFO] Enabled default license matchers.
[INFO] Will parse SCM ignores for exclusions...
[INFO] Parsing exclusions from <
https://ci-builds.apache.org/job/Commons/job/commons-codec/ws/.gitignore>
[INFO] Finished adding exclusions from SCM ignore files.
[INFO] 73 implicit excludes (use -debug for more details).
[INFO] 12 explicit excludes (use -debug for more details).
[INFO] 314 resources included (use -debug for more details)
[INFO] Rat check: Summary over all files. Unapproved: 0, unknown: 0,
generated: 0, approved: 308 licenses.
[INFO]
[INFO] --- build-helper-maven-plugin:3.2.0:parse-version (parse-version) @
commons-codec ---
[INFO]
[INFO] --- maven-antrun-plugin:1.8:run (javadoc.resources) @ commons-codec
---
[INFO] Executing tasks

main:
 [copy] Copying 2 files to <
https://ci-builds.apache.org/job/Commons/job/commons-codec/ws/target/apidocs/META-INF
>
[INFO] Executed tasks
[INFO]
[INFO] --- maven-remote-resources-plugin:1.7.0:process
(process-resource-bundles) @ commons-c

Re: Build failed in Jenkins: Commons » commons-codec #138

[Bruno Kinoshita]

Not sure, but the console output is similar to this build from 2 days ago,
that also failed:
https://ci-builds.apache.org/job/Commons/job/commons-codec/commons-codec$commons-codec/134/console

Maybe it's due to Maven version, or to some Maven plugin? Travis/GH Actions
appear to be running OK.

-Bruno

On Sun, 20 Feb 2022 at 14:15, Gary Gregory  wrote:

> Is this a Jenkins issue?
>
> [INFO] --- maven-compiler-plugin:3.8.1:compile (default-compile) @
> commons-codec ---
> [INFO] Changes detected - recompiling the module!
> [INFO] Compiling 73 source files to
> /home/jenkins/workspace/Commons/commons-codec/target/classes
> [WARNING] Attempt to (de-)serialize anonymous class
> org.jfrog.hudson.maven2.MavenDependenciesRecorder$1; see:
> https://jenkins.io/redirect/serialization-of-anonymous-classes/
>
> -- Forwarded message -
> From: Apache Jenkins Server 
> Date: Sat, Feb 19, 2022, 20:12
> Subject: Build failed in Jenkins: Commons » commons-codec #138
> To: , , <
> boa...@gmail.com>
>
>
> See <
>
> https://ci-builds.apache.org/job/Commons/job/commons-codec/138/display/redirect?page=changes
> >
>
> Changes:
>
> [github] CODEC-285 replace JUnit v4 test expected with assertThrows (#112)
>
> [github] Avoid C-style array declaration and use Java instead. (#108)
>
> [github] Java 8 improvements: (#106)
>
>
> --
> Started by an SCM change
> Running as SYSTEM
> [EnvInject] - Loading node environment variables.
> Building remotely on builds31 (ubuntu) in workspace <
> https://ci-builds.apache.org/job/Commons/job/commons-codec/ws/>
> The recommended git tool is: NONE
> No credentials specified
>  > git rev-parse --resolve-git-dir <
> https://ci-builds.apache.org/job/Commons/job/commons-codec/ws/.git> #
> timeout=10
> Fetching changes from the remote Git repository
>  > git config remote.origin.url https://github.com/apache/commons-codec #
> timeout=10
> Fetching upstream changes from https://github.com/apache/commons-codec
>  > git --version # timeout=10
>  > git --version # 'git version 2.17.1'
>  > git fetch --tags --progress -- https://github.com/apache/commons-codec
> +refs/heads/*:refs/remotes/origin/*
> 
> # timeout=10
>  > git rev-parse refs/remotes/origin/master^{commit} # timeout=10
> Checking out Revision fb84f6d9732d44cda0edf449f7a5cba0b340731a
> (refs/remotes/origin/master)
>  > git config core.sparsecheckout # timeout=10
>  > git checkout -f fb84f6d9732d44cda0edf449f7a5cba0b340731a # timeout=10
> Commit message: "Java 8 improvements: (#106)"
>  > git rev-list --no-walk 438b54b2075652215a75a834ee64ec5b0b29a6f8 #
> timeout=10
> [EnvInject] - Injecting environment variables from a build step.
> Setting JDK_1_8_LATEST_HOME=/home/jenkins/tools/java/latest1.8
> [EnvInject] - Injecting as environment variables the properties content
> JAVA_1_7_HOME=${JDK_1_7_LATEST_HOME}
>
> [EnvInject] - Variables injected successfully.
> Setting JDK_1_8_LATEST_HOME=/home/jenkins/tools/java/latest1.8
> Parsing POMs
> Established TCP socket on 44177
> maven35-agent.jar already up to date
> maven35-interceptor.jar already up to date
> maven3-interceptor-commons.jar already up to date
> [commons-codec] $ /home/jenkins/tools/java/latest1.8/bin/java -cp
>
> /home/jenkins/maven35-agent.jar:/home/jenkins/tools/maven/latest3/boot/plexus-classworlds-2.6.0.jar:/home/jenkins/tools/maven/latest3/conf/logging
> jenkins.maven3.agent.Maven35Main /home/jenkins/tools/maven/latest3/
> /home/jenkins/agent.jar /home/jenkins/maven35-interceptor.jar
> /home/jenkins/maven3-interceptor-commons.jar 44177
> <===[JENKINS REMOTING CAPACITY]===>   channel started
> Executing Maven:  -B -f <
> https://ci-builds.apache.org/job/Commons/job/commons-codec/ws/pom.xml>
> clean package deploy -Dgpg.skip -Pjava-1.8
> [INFO] Scanning for projects...
> [HUDSON] Collecting dependencies info
> [INFO]
> [INFO] < commons-codec:commons-codec
> >-
> [INFO] Building Apache Commons Codec 1.17-SNAPSHOT
> [INFO] [ jar
> ]-
> [INFO]
> [INFO] --- maven-clean-plugin:3.1.0:clean (default-clean) @ commons-codec
> ---
> [INFO] Deleting <
> https://ci-builds.apache.org/job/Commons/job/commons-codec/ws/target>
> [INFO]
> [INFO] --- maven-enforcer-plugin:3.0.0-M3:enforce (enforce-maven-version) @
> commons-codec ---
> [INFO]
> [INFO] --- maven-enforcer-plugin:3.0.0-M3:enforce (enforce-maven-3) @
> commons-codec ---
> [INFO]
> [INFO] --- apache-rat-plugin:0.13:check (rat-check) @ commons-codec ---
> [INFO] Enabled default license matchers.
> [INFO] Will parse SCM ignores for exclusions...
> [INFO] Parsing exclusions from <
> https://ci-builds.apache.org/job/Commons/job/commons-codec/ws/.gitignore>
> [INFO] Finished adding exclusions from SCM ignore files.
> [INFO] 73 implicit excludes (use -debug for more details).
> [INFO] 12 explicit excludes (use -debug 

CODED-285 - JUnit5

[Itamar C]

Just finished changing all tests to JUnit5 and dropped junit-vintage-engine
from pom.xml.

It's on PR #113.

I saw that there are another 8 PRs in the issue, I hope there are not too
many conflicts in there for the reviewer.

Any doubt about my changes, I'm here to answer.

(By the way, I just sent my ICLA to the Apache Foundation Secretary.)

Regards,

Itamar Carvalho


On Thu, Feb 17, 2022 at 11:09 PM Matt Sicker  wrote:

> The same applies to changing access modifiers. JUnit 5 encourages use of
> package private everything as it’s the least typing and now supported (as
> in v5 will reflectively allow access to your test code if it’s not public).
>
> —
> Matt Sicker
>
> > On Feb 17, 2022, at 19:59, Gary Gregory  wrote:
> >
> > On Thu, Feb 17, 2022 at 8:47 PM Itamar C  wrote:
> >
> >>> On Thu, Feb 17, 2022 at 8:16 PM Gilles Sadowski 
> >>> wrote:
> >>>
> >>> Which discussion (since this thread covered more than one subject)?
> >>> If you mean the "migration to Junit 5" task for [Codec], it's already
> >>> there.[1]
> >>> If you mean the method rename (to remove the "test" prefix), then
> >>> the "dev" ML is where to continue the discussion (and/or start a vote
> >>> if there is no clear agreement).
> >>>
> >>> Regards,
> >>> Gilles
> >>>
> >>>
> >> Hi.
> >>
> >> I was talking about the method rename (to remove the "test" prefix).
> >> If this ML is the right place to continue the discussion or make a
> vote, ok
> >> then.
> >>
> >
> > I am opposed to changing hundreds if not thousands of methods names just
> > for cosmetic reasons, it is certainly not required to use JUnit 5.
> >
> > Gary
> >
> >
> >> Regards,
> >> Itamar
> >>
>