Hello Everyone,

Despite how really simple this PR is, I thought I would take this time to
relay some helpful tips I learned to hopefully help those learning Beam and
wanting to contribute new modules.

*For those who know Beam*:

https://github.com/apache/beam/pull/25951 adds the Java IO modules
sdks/java/io/csv
<https://github.com/apache/beam/tree/master/sdks/java/io/csv> and
sdks/java/io/file-schema-transform
<https://github.com/apache/beam/tree/master/sdks/java/io/file-schema-transform>
to PreCommit_Java_IO checks.

*For those on the learning path*:

The takeaway of the following is that to add a new Java SDK module:
*1. Configure settings.gradle.kts
<https://github.com/apache/beam/blob/master/settings.gradle.kts>*
*2. Configure build.gradle.kts
<https://github.com/apache/beam/blob/master/build.gradle.kts>*
*2. Reference the module in one of .test-infra/jenkins groovy files
<https://github.com/apache/beam/tree/master/.test-infra/jenkins>.*

Beam uses two technologies to automate developer operations.  One is
called gradle and the other is Jenkins.  Don't worry.  You don't need to
know a lot about these technologies to contribute to Beam.

*1. Configure settings.gradle.kts
<https://github.com/apache/beam/blob/master/settings.gradle.kts>*

If you are familiar with gradle, you know that to add a module, you would
configure using the root project settings.gradle or settings.gradle.kts.
(See: gradle documentation
<https://docs.gradle.org/current/userguide/structuring_software_products.html#defining_an_inner_structure_for_components>).
If you are not familiar, simply know that you just need to add the
following in Beam's settings.gradle.kts
<https://github.com/apache/beam/blob/master/settings.gradle.kts>.  Just
look where the other includes already exist in the file and add yours,
ideally alphabetically sorted.

include(":path:to:my:amazing:module")

For example, include(":sdks:java:io:csv") tells gradle to navigate to the
directory path from the root sdks/java/io/csv
<https://github.com/apache/beam/tree/master/sdks/java/io/csv>.

What does this do?  If you open up your terminal and navigate to the root
directory of the Beam project and run:

./gradlew :sdks:java:io:csv:test

This tells gradle to expect a gradle.build file in the folder path
sdks/java/io/csv
<https://github.com/apache/beam/tree/master/sdks/java/io/csv> and run its
test task in the sdks/java/io/csv
<https://github.com/apache/beam/tree/master/sdks/java/io/csv>'s module.

*2. Configure build.gradle.kts
<https://github.com/apache/beam/blob/master/build.gradle.kts>*

Next add your module to one of build.gradle.kts
<https://github.com/apache/beam/blob/master/build.gradle.kts>'s gradle
tasks.  A gradle task is "A task represents some atomic piece of work which
a build performs." (See gradle documentation
<https://docs.gradle.org/current/userguide/tutorial_using_tasks.html>).
Don't worry about fully understanding the details.  In this PR
<https://github.com/apache/beam/pull/25951>'s context, it adds to
the javaioPreCommit task as these are IO related modules.  Work with the
person reviewing your PR to determine which makes sense for your case.

*3. Reference the module in one of .test-infra/jenkins groovy files
<https://github.com/apache/beam/tree/master/.test-infra/jenkins>.*

Now let's talk about Jenkins or the absolute bare minimum you need to know
about it.  Beam uses Jenkins to automate various checks before and after
code is submitted to the project's GitHub repository.  What you do need to
know is that if you add a new module, you might want to reference it in one
of the various .test-infra/jenkins groovy files
<https://github.com/apache/beam/tree/master/.test-infra/jenkins>.

In this PR <https://github.com/apache/beam/pull/25951>'s case, it simply
adds csv and file-schema-transform so that gradle executes these module's
tests within the Jenkins process.

Best,

Damon

Reply via email to