uschindler commented on code in PR #1320: URL: https://github.com/apache/solr/pull/1320#discussion_r1094153859
########## gradle/template.gradle.properties: ########## @@ -0,0 +1,106 @@ +############################# +# Local developer settings # +############################# +# +# The first invocation of any task in Solr gradlew will generate and save this project-local 'gradle.properties' file. +# This file contains the defaults you may (but don't have to) tweak for your particular hardware (or taste). Note there +# are certain settings in that file that may be _required_ at runtime for certain plugins (an example is the spotless/ +# google java format plugin, which requires adding custom exports to JVM modules). gradlew only generates this file +# if it's not already present (it never overwrites the defaults) -- occasionally you may have to manually delete (or move) +# this file and regenerate from scratch. +# +# This is an overview of some of these settings. +# +############### +# Parallelism # +############### +# +# Gradle build can run tasks in parallel but by default it consumes all CPU cores which +# is too optimistic a default for Solr tests. You can disable the parallelism +# entirely or assign it a 'low' priority with these properties: +# +# org.gradle.parallel=[true, false] +# org.gradle.priority=[normal, low] +# +# The default level of parallelism is computed based on the number of cores on +# your machine (on the first run of gradle build). By default these are fairly conservative +# settings (half the number of cores for workers, for example): +# +# org.gradle.workers.max=[X] +# tests.jvms=[N <= X] +# +# The number of test JVMs can be lower than the number of workers: this just means +# that two projects can run tests in parallel to saturate all the workers. The I/O and memory +# bandwidth limits will kick in quickly so even if you have a very beefy machine bumping +# it too high may not help. +# +# You can always override these settings locally using command line as well: +# gradlew -Ptests.jvms=N --max-workers=X +# +############# +# Test JVMS # +############# +# +# Test JVMs have their own set of arguments which can be customized. These are configured +# separately from the gradle workers, for example: +# +# tests.jvms=3 +# tests.heapsize=512m +# tests.minheapsize=512m +# tests.jvmargs=-XX:+UseParallelGC -XX:TieredStopAtLevel=1 -XX:ActiveProcessorCount=1 +# +################# +# Gradle Daemon # +################# +# +# The gradle daemon is a background process that keeps an evaluated copy of the project +# structure, some caches, etc. It speeds up repeated builds quite a bit but if you don't +# like the idea of having a (sizeable) background process running in the background, +# disable it. +# +# org.gradle.daemon=[true, false] +# org.gradle.jvmargs=... +############################################################################################# + +# UTF-8 as standard file encoding +systemProp.file.encoding=UTF-8 + +# Set up gradle JVM defaults. +# +# We also open up internal compiler modules for spotless/ google java format. +org.gradle.jvmargs=-Xmx1g -XX:TieredStopAtLevel=1 -XX:+UseParallelGC -XX:ActiveProcessorCount=1 \ + --add-exports jdk.compiler/com.sun.tools.javac.api=ALL-UNNAMED \ + --add-exports jdk.compiler/com.sun.tools.javac.file=ALL-UNNAMED \ + --add-exports jdk.compiler/com.sun.tools.javac.parser=ALL-UNNAMED \ + --add-exports jdk.compiler/com.sun.tools.javac.tree=ALL-UNNAMED \ + --add-exports jdk.compiler/com.sun.tools.javac.util=ALL-UNNAMED + +# Run at normal priority, in parallel +org.gradle.parallel=true +org.gradle.priority=normal + +# This setting enables local task output caches. This will speed up +# your local builds in most cases but will also consume disk space in your +# gradle home. See SOLR-15603 for details. +# org.gradle.caching=true + +# Silence gradle warnings. We'll deal with them when we upgrade the wrapper. +org.gradle.warning.mode=none + +# You may disable the background daemon if it consumes too much memory. +org.gradle.daemon=true +# timeout after 15 mins of inactivity. +org.gradle.daemon.idletimeout=900000 + +# Maximum number of parallel gradle workers. +org.gradle.workers.max=2 + +# Maximum number of test JVMs forked per test task. +tests.jvms=2 + +# Disable auto JVM provisioning (we don't use toolchains yet but want no surprises). +org.gradle.java.installations.auto-download=false + +# Set these to enable automatic JVM location discovery. +org.gradle.java.installations.fromEnv=JDK11,JDK12,JDK13,JDK14,JDK15,JDK16,JDK17 +org.gradle.java.installations.paths=(custom paths) Review Comment: Yes this needs to go away, see Lucene build. If this is enabled Gradle prits a warning everytime as soon as you access any toolchain API. ########## buildSrc/src/main/java/org/apache/lucene/gradle/GradlePropertiesGenerator.java: ########## @@ -0,0 +1,86 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.lucene.gradle; + +import java.io.IOException; +import java.nio.file.Files; +import java.nio.file.Path; +import java.nio.file.Paths; +import java.nio.file.StandardOpenOption; +import java.util.Map; +import java.util.stream.Collectors; + +/** + * Standalone class that generates a populated gradle.properties from a template. + * + * <p>Has no dependencies outside of standard java libraries + */ +public class GradlePropertiesGenerator { + public static void main(String[] args) { + if (args.length != 2) { + System.err.println("Usage: java GradlePropertiesGenerator.java <source> <destination>"); + System.exit(2); + } + + try { + new GradlePropertiesGenerator().run(Paths.get(args[0]), Paths.get(args[1])); + } catch (Exception e) { + System.err.println("ERROR: " + e.getMessage()); + System.exit(3); + } + } + + public void run(Path source, Path destination) throws IOException { + if (!Files.exists(source)) { + throw new IOException("template file not found: " + source); + } + if (Files.exists(destination)) { + System.out.println(destination + " already exists, skipping generation."); + } + + // Approximate a common-sense default for running gradle/tests with parallel + // workers: half the count of available cpus but not more than 12. + var cpus = Runtime.getRuntime().availableProcessors(); + var maxWorkers = (int) Math.max(1d, Math.min(cpus * 0.5d, 12)); + var testsJvms = (int) Math.max(1d, Math.min(cpus * 0.5d, 12)); + + var replacements = Map.of("org.gradle.workers.max", maxWorkers, "tests.jvms", testsJvms); + + // Java properties doesn't preserve comments, so going line by line instead + // The properties file isn't big, nor is the map of replacements, so not worrying about speed Review Comment: I am also not really happy with this parsing. My idea would be: As the template is always applied (no problem with windows or any othe roperating system), we can make the template a real template and add two search/replace markers in the file. We should then load the file into a String (UTF-8 encoding!!!) and then do String#replace('PLACEHOLDER', value). Those replaces can be done using `replacements.entries().forEach(e -> fileContent = fileContent.replace(e.getKey(), e.getValue())` (don't use replaceAll, as it uses regex and we have no forbiddenapis here to check this!!!!). So I would add `@MAX_WORKERS@` and `@TEST_JVMS@` as map keys and replace those tags. This is like Ant did it for long time :-) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
