This is an automated email from the ASF dual-hosted git repository.
skrawcz pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/hamilton.git
The following commit(s) were added to refs/heads/main by this push:
new a2217ff6 Add missing headers and update files to help with release
(#1378)
a2217ff6 is described below
commit a2217ff695820d693437d8cde590708a917c7d6a
Author: Stefan Krawczyk <[email protected]>
AuthorDate: Mon Sep 15 21:11:01 2025 -0700
Add missing headers and update files to help with release (#1378)
* Adds release helper script
This script will help us release code.
* Adds more scripts to help
* Adds missing headers
Removes MD files from getting into the source release.
* Updates scripts to help with release
More to be done here. This is WIP but good enough to commit.
* Adds missing header
---
CONTRIBUTING.md | 17 ++
MANIFEST.in | 18 +-
contrib/LICENSE | 228 ++++++++++++++++++++++++
pyproject.toml | 17 ++
scripts/apache_release_helper.py | 370 +++++++++++++++++++++++++++++++++++++++
scripts/qualify.sh | 124 +++++++++++++
scripts/setup_keys.sh | 95 ++++++++++
7 files changed, 868 insertions(+), 1 deletion(-)
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
index 800a4470..11af3833 100644
--- a/CONTRIBUTING.md
+++ b/CONTRIBUTING.md
@@ -1,3 +1,20 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
# Guidance on how to contribute
> All contributions to this project will be released under the [Apache 2.0
> License](LICENSE). .
diff --git a/MANIFEST.in b/MANIFEST.in
index 50cad726..d34828ff 100644
--- a/MANIFEST.in
+++ b/MANIFEST.in
@@ -1,4 +1,20 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
include LICENSE
-include *.md
include NOTICE
include DISCLAIMER
diff --git a/contrib/LICENSE b/contrib/LICENSE
new file mode 100644
index 00000000..7193c0c1
--- /dev/null
+++ b/contrib/LICENSE
@@ -0,0 +1,228 @@
+ Apache License
+ Version 2.0, January 2004
+ http://www.apache.org/licenses/
+
+ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+ 1. Definitions.
+
+ "License" shall mean the terms and conditions for use, reproduction,
+ and distribution as defined by Sections 1 through 9 of this document.
+
+ "Licensor" shall mean the copyright owner or entity authorized by
+ the copyright owner that is granting the License.
+
+ "Legal Entity" shall mean the union of the acting entity and all
+ other entities that control, are controlled by, or are under common
+ control with that entity. For the purposes of this definition,
+ "control" means (i) the power, direct or indirect, to cause the
+ direction or management of such entity, whether by contract or
+ otherwise, or (ii) ownership of fifty percent (50%) or more of the
+ outstanding shares, or (iii) beneficial ownership of such entity.
+
+ "You" (or "Your") shall mean an individual or Legal Entity
+ exercising permissions granted by this License.
+
+ "Source" form shall mean the preferred form for making modifications,
+ including but not limited to software source code, documentation
+ source, and configuration files.
+
+ "Object" form shall mean any form resulting from mechanical
+ transformation or translation of a Source form, including but
+ not limited to compiled object code, generated documentation,
+ and conversions to other media types.
+
+ "Work" shall mean the work of authorship, whether in Source or
+ Object form, made available under the License, as indicated by a
+ copyright notice that is included in or attached to the work
+ (an example is provided in the Appendix below).
+
+ "Derivative Works" shall mean any work, whether in Source or Object
+ form, that is based on (or derived from) the Work and for which the
+ editorial revisions, annotations, elaborations, or other modifications
+ represent, as a whole, an original work of authorship. For the purposes
+ of this License, Derivative Works shall not include works that remain
+ separable from, or merely link (or bind by name) to the interfaces of,
+ the Work and Derivative Works thereof.
+
+ "Contribution" shall mean any work of authorship, including
+ the original version of the Work and any modifications or additions
+ to that Work or Derivative Works thereof, that is intentionally
+ submitted to Licensor for inclusion in the Work by the copyright owner
+ or by an individual or Legal Entity authorized to submit on behalf of
+ the copyright owner. For the purposes of this definition, "submitted"
+ means any form of electronic, verbal, or written communication sent
+ to the Licensor or its representatives, including but not limited to
+ communication on electronic mailing lists, source code control systems,
+ and issue tracking systems that are managed by, or on behalf of, the
+ Licensor for the purpose of discussing and improving the Work, but
+ excluding communication that is conspicuously marked or otherwise
+ designated in writing by the copyright owner as "Not a Contribution."
+
+ "Contributor" shall mean Licensor and any individual or Legal Entity
+ on behalf of whom a Contribution has been received by Licensor and
+ subsequently incorporated within the Work.
+
+ 2. Grant of Copyright License. Subject to the terms and conditions of
+ this License, each Contributor hereby grants to You a perpetual,
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+ copyright license to reproduce, prepare Derivative Works of,
+ publicly display, publicly perform, sublicense, and distribute the
+ Work and such Derivative Works in Source or Object form.
+
+ 3. Grant of Patent License. Subject to the terms and conditions of
+ this License, each Contributor hereby grants to You a perpetual,
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+ (except as stated in this section) patent license to make, have made,
+ use, offer to sell, sell, import, and otherwise transfer the Work,
+ where such license applies only to those patent claims licensable
+ by such Contributor that are necessarily infringed by their
+ Contribution(s) alone or by combination of their Contribution(s)
+ with the Work to which such Contribution(s) was submitted. If You
+ institute patent litigation against any entity (including a
+ cross-claim or counterclaim in a lawsuit) alleging that the Work
+ or a Contribution incorporated within the Work constitutes direct
+ or contributory patent infringement, then any patent licenses
+ granted to You under this License for that Work shall terminate
+ as of the date such litigation is filed.
+
+ 4. Redistribution. You may reproduce and distribute copies of the
+ Work or Derivative Works thereof in any medium, with or without
+ modifications, and in Source or Object form, provided that You
+ meet the following conditions:
+
+ (a) You must give any other recipients of the Work or
+ Derivative Works a copy of this License; and
+
+ (b) You must cause any modified files to carry prominent notices
+ stating that You changed the files; and
+
+ (c) You must retain, in the Source form of any Derivative Works
+ that You distribute, all copyright, patent, trademark, and
+ attribution notices from the Source form of the Work,
+ excluding those notices that do not pertain to any part of
+ the Derivative Works; and
+
+ (d) If the Work includes a "NOTICE" text file as part of its
+ distribution, then any Derivative Works that You distribute must
+ include a readable copy of the attribution notices contained
+ within such NOTICE file, excluding those notices that do not
+ pertain to any part of the Derivative Works, in at least one
+ of the following places: within a NOTICE text file distributed
+ as part of the Derivative Works; within the Source form or
+ documentation, if provided along with the Derivative Works; or,
+ within a display generated by the Derivative Works, if and
+ wherever such third-party notices normally appear. The contents
+ of the NOTICE file are for informational purposes only and
+ do not modify the License. You may add Your own attribution
+ notices within Derivative Works that You distribute, alongside
+ or as an addendum to the NOTICE text from the Work, provided
+ that such additional attribution notices cannot be construed
+ as modifying the License.
+
+ You may add Your own copyright statement to Your modifications and
+ may provide additional or different license terms and conditions
+ for use, reproduction, or distribution of Your modifications, or
+ for any such Derivative Works as a whole, provided Your use,
+ reproduction, and distribution of the Work otherwise complies with
+ the conditions stated in this License.
+
+ 5. Submission of Contributions. Unless You explicitly state otherwise,
+ any Contribution intentionally submitted for inclusion in the Work
+ by You to the Licensor shall be under the terms and conditions of
+ this License, without any additional terms or conditions.
+ Notwithstanding the above, nothing herein shall supersede or modify
+ the terms of any separate license agreement you may have executed
+ with Licensor regarding such Contributions.
+
+ 6. Trademarks. This License does not grant permission to use the trade
+ names, trademarks, service marks, or product names of the Licensor,
+ except as required for reasonable and customary use in describing the
+ origin of the Work and reproducing the content of the NOTICE file.
+
+ 7. Disclaimer of Warranty. Unless required by applicable law or
+ agreed to in writing, Licensor provides the Work (and each
+ Contributor provides its Contributions) on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+ implied, including, without limitation, any warranties or conditions
+ of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+ PARTICULAR PURPOSE. You are solely responsible for determining the
+ appropriateness of using or redistributing the Work and assume any
+ risks associated with Your exercise of permissions under this License.
+
+ 8. Limitation of Liability. In no event and under no legal theory,
+ whether in tort (including negligence), contract, or otherwise,
+ unless required by applicable law (such as deliberate and grossly
+ negligent acts) or agreed to in writing, shall any Contributor be
+ liable to You for damages, including any direct, indirect, special,
+ incidental, or consequential damages of any character arising as a
+ result of this License or out of the use or inability to use the
+ Work (including but not limited to damages for loss of goodwill,
+ work stoppage, computer failure or malfunction, or any and all
+ other commercial damages or losses), even if such Contributor
+ has been advised of the possibility of such damages.
+
+ 9. Accepting Warranty or Additional Liability. While redistributing
+ the Work or Derivative Works thereof, You may choose to offer,
+ and charge a fee for, acceptance of support, warranty, indemnity,
+ or other liability obligations and/or rights consistent with this
+ License. However, in accepting such obligations, You may act only
+ on Your own behalf and on Your sole responsibility, not on behalf
+ of any other Contributor, and only if You agree to indemnify,
+ defend, and hold each Contributor harmless for any liability
+ incurred by, or claims asserted against, such Contributor by reason
+ of your accepting any such warranty or additional liability.
+
+ END OF TERMS AND CONDITIONS
+
+ APPENDIX: How to apply the Apache License to your work.
+
+ To apply the Apache License to your work, attach the following
+ boilerplate notice, with the fields enclosed by brackets "[]"
+ replaced with your own identifying information. (Don't include
+ the brackets!) The text should be enclosed in the appropriate
+ comment syntax for the file format. We also recommend that a
+ file or class name and description of purpose be included on the
+ same "printed page" as the copyright notice for easier
+ identification within third-party archives.
+
+ Copyright [yyyy] [name of copyright owner]
+
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+
+-------------------------------
+
+contrib/hamilton/contrib/user/skrawcz/customize_embeddings/__init__.py is
copied
+from https://github.com/openai/openai-cookbook and is licensed under the MIT
License.
+
+MIT License
+
+Copyright (c) 2025 OpenAI
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.
diff --git a/pyproject.toml b/pyproject.toml
index 932c8b04..172f9634 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -1,3 +1,20 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
[build-system]
requires = ["setuptools>=72.1"]
build-backend = "setuptools.build_meta"
diff --git a/scripts/apache_release_helper.py b/scripts/apache_release_helper.py
new file mode 100644
index 00000000..55255389
--- /dev/null
+++ b/scripts/apache_release_helper.py
@@ -0,0 +1,370 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+import argparse
+import glob
+import hashlib
+import os
+import shutil
+import subprocess
+import sys
+from typing import Optional
+
+# --- Configuration ---
+# You need to fill these in for your project.
+# The name of your project's short name (e.g., 'myproject').
+PROJECT_SHORT_NAME = "hamilton"
+# The file where you want to update the version number.
+# Common options are setup.py, __init__.py, or a dedicated VERSION file.
+# For example: "src/main/python/myproject/__init__.py"
+VERSION_FILE = "hamilton/version.py"
+# A regular expression pattern to find the version string in the VERSION_FILE.
+# For example: r"__version__ = \"(\d+\.\d+\.\d+)\""
+# The capture group (parentheses) should capture the version number.
+VERSION_PATTERN = r"VERSION = \((\d+), (\d+), (\d+)(, \"(\w+)\")?\)"
+
+
+def get_version_from_file(file_path: str) -> str:
+ """Get the version from a file."""
+ import re
+
+ with open(file_path) as f:
+ content = f.read()
+ match = re.search(VERSION_PATTERN, content)
+ if match:
+ major, minor, patch, rc_group, rc = match.groups()
+ version = f"{major}.{minor}.{patch}"
+ if rc:
+ raise ValueError("Do not commit RC to the version file.")
+ return version
+ raise ValueError(f"Could not find version in {file_path}")
+
+
+def check_prerequisites():
+ """Checks for necessary command-line tools and Python modules."""
+ print("Checking for required tools...")
+ required_tools = ["git", "gpg", "svn"]
+ for tool in required_tools:
+ if shutil.which(tool) is None:
+ print(f"Error: '{tool}' not found. Please install it and ensure
it's in your PATH.")
+ sys.exit(1)
+
+ try:
+ import build # noqa:F401
+
+ print("Python 'build' module found.")
+ except ImportError:
+ print(
+ "Error: The 'build' module is not installed. Please install it
with 'pip install build'."
+ )
+ sys.exit(1)
+
+ print("All required tools found.")
+
+
+def update_version(version, rc_num):
+ """Updates the version number in the specified file."""
+ import re
+
+ print(f"Updating version in {VERSION_FILE} to {version} RC{rc_num}...")
+ try:
+ with open(VERSION_FILE, "r") as f:
+ content = f.read()
+ major, minor, patch = version.split(".")
+ if int(rc_num) >= 0:
+ new_version_tuple = f'VERSION = ({major}, {minor}, {patch},
"RC{rc_num}")'
+ else:
+ new_version_tuple = f"VERSION = ({major}, {minor}, {patch})"
+ new_content = re.sub(VERSION_PATTERN, new_version_tuple, content)
+ if new_content == content:
+ print("Error: Could not find or replace version string. Check your
VERSION_PATTERN.")
+ return False
+
+ with open(VERSION_FILE, "w") as f:
+ f.write(new_content)
+
+ print("Version updated successfully.")
+ return True
+
+ except FileNotFoundError:
+ print(f"Error: {VERSION_FILE} not found.")
+ return False
+ except Exception as e:
+ print(f"An error occurred while updating the version: {e}")
+ return False
+
+
+def sign_artifacts(archive_name: str) -> Optional[list[str]]:
+ """Creates signed files for the designated artifact."""
+ files = []
+ # Sign the tarball with GPG. The user must have a key configured.
+ try:
+ subprocess.run(
+ ["gpg", "--armor", "--output", f"{archive_name}.asc",
"--detach-sig", archive_name],
+ check=True,
+ )
+ files.append(f"{archive_name}.asc")
+ print(f"Created GPG signature: {archive_name}.asc")
+ except subprocess.CalledProcessError as e:
+ print(f"Error signing tarball: {e}")
+ return None
+
+ # Generate SHA512 checksum.
+ sha512_hash = hashlib.sha512()
+ with open(archive_name, "rb") as f:
+ while True:
+ data = f.read(65536)
+ if not data:
+ break
+ sha512_hash.update(data)
+
+ with open(f"{archive_name}.sha512", "w") as f:
+ f.write(f"{sha512_hash.hexdigest()}\n")
+ print(f"Created SHA512 checksum: {archive_name}.sha512")
+ files.append(f"{archive_name}.sha512")
+ return files
+
+
+def create_release_artifacts(version) -> tuple[list[str], list[str]]:
+ """Creates the source tarball, GPG signature, and checksums using `python
-m build`."""
+ print("Creating release artifacts with 'python -m build'...")
+
+ # Clean the dist directory before building.
+ if os.path.exists("dist"):
+ shutil.rmtree("dist")
+
+ # Use python -m build to create the source distribution.
+ try:
+ subprocess.run(["python", "-m", "build", "--sdist", "."], check=True)
+ print("Source distribution created successfully.")
+ except subprocess.CalledProcessError as e:
+ print(f"Error creating source distribution: {e}")
+ return None
+
+ # Find the created tarball in the dist directory.
+ expected_tar_ball = f"dist/sf_hamilton-{version.lower()}.tar.gz"
+ tarball_path = glob.glob(expected_tar_ball)
+
+ if not tarball_path:
+ print(
+ f"Error: Could not find {expected_tar_ball} the generated source
tarball in the 'dist' directory."
+ )
+ if os.path.exists("dist"):
+ print("Contents of 'dist' directory:")
+ for item in os.listdir("dist"):
+ print(f"- {item}")
+ else:
+ print("'dist' directory not found.")
+ raise ValueError("Could not find the generated source tarball in the
'dist' directory.")
+
+ # copy the tarball to be
apache-hamilton-{version.lower()}-incubating.tar.gz
+ new_tar_ball = f"dist/apache-hamilton-{version.lower()}-incubating.tar.gz"
+ shutil.copy(tarball_path[0], new_tar_ball)
+ archive_name = new_tar_ball
+ print(f"Found source tarball: {archive_name}")
+ main_signed_files = sign_artifacts(archive_name)
+ if main_signed_files is None:
+ raise ValueError("Could not sign the main release artifacts.")
+ # create sf-hamilton release artifacts
+ sf_hamilton_signed_files = sign_artifacts(expected_tar_ball)
+ return [new_tar_ball] + main_signed_files, [expected_tar_ball] +
sf_hamilton_signed_files
+
+
+def svn_upload(version, rc_num, archive_files, sf_hamilton_archive_files,
apache_id):
+ """Uploads the artifacts to the ASF dev distribution repository."""
+ print("Uploading artifacts to ASF SVN...")
+ svn_path =
f"https://dist.apache.org/repos/dist/dev/incubator/{PROJECT_SHORT_NAME}/apache-hamilton/{version}-incubating-RC{rc_num}";
+
+ try:
+ # Create a new directory for the release candidate.
+ subprocess.run(
+ [
+ "svn",
+ "mkdir",
+ "-m",
+ f"Creating directory for {version}-incubating-RC{rc_num}",
+ svn_path,
+ ],
+ check=True,
+ )
+
+ # Get the files to import (tarball, asc, sha512).
+ files_to_import = archive_files + sf_hamilton_archive_files
+
+ # Use svn import for the new directory.
+ for file_path in files_to_import:
+ subprocess.run(
+ [
+ "svn",
+ "import",
+ file_path,
+ f"{svn_path}/{os.path.basename(file_path)}",
+ "-m",
+ f"Adding {os.path.basename(file_path)}",
+ "--username",
+ apache_id,
+ ],
+ check=True,
+ )
+
+ print(f"Artifacts successfully uploaded to: {svn_path}")
+ return svn_path
+
+ except subprocess.CalledProcessError as e:
+ print(f"Error during SVN upload: {e}")
+ print("Make sure you have svn access configured for your Apache ID.")
+ return None
+
+
+def generate_email_template(version, rc_num, svn_url):
+ """Generates the content for the [VOTE] email."""
+ print("Generating email template...")
+ version_with_incubating = f"{version}-incubating"
+ tag = f"v{version}"
+
+ email_content = f"""[VOTE] Release Apache {PROJECT_SHORT_NAME}
{version_with_incubating} (release candidate {rc_num})
+
+Hi all,
+
+This is a call for a vote on releasing Apache {PROJECT_SHORT_NAME}
{version_with_incubating},
+release candidate {rc_num}.
+
+This release includes the following changes (see CHANGELOG for details):
+- [List key changes here]
+
+The artifacts for this release candidate can be found at:
+{svn_url}
+
+The Git tag to be voted upon is:
+{tag}
+
+The release hash is:
+[Insert git commit hash here]
+
+
+Release artifacts are signed with the following key:
+[Insert your GPG key ID here]
+The KEYS file is available at:
+https://downloads.apache.org/incubator/{PROJECT_SHORT_NAME}/KEYS
+
+Please download, verify, and test the release candidate.
+
+For testing, please run some of the examples, scripts/qualify.sh has
+a sampling of them to run.
+
+The vote will run for a minimum of 72 hours.
+Please vote:
+
+[ ] +1 Release this package as Apache {PROJECT_SHORT_NAME}
{version_with_incubating}
+[ ] +0 No opinion
+[ ] -1 Do not release this package because... (Please provide a reason)
+
+Checklist for reference:
+[ ] Download links are valid.
+[ ] Checksums and signatures.
+[ ] LICENSE/NOTICE files exist
+[ ] No unexpected binary files
+[ ] All source files have ASF headers
+[ ] Can compile from source
+
+On behalf of the Apache {PROJECT_SHORT_NAME} PPMC,
+[Your Name]
+"""
+ print("\n" + "=" * 80)
+ print("EMAIL TEMPLATE (COPY AND PASTE TO YOUR MAILING LIST)")
+ print("=" * 80)
+ print(email_content)
+ print("=" * 80)
+
+
+def main():
+ """
+ ### How to Use the Updated Script
+
+ 1. **Install the `build` module**:
+ ```bash
+ pip install build
+ ```
+ 2. **Configure the Script**: Open `apache_release_helper.py` in a text
editor and update the three variables at the top of the file with your
project's details:
+ * `PROJECT_SHORT_NAME`
+ * `VERSION_FILE` and `VERSION_PATTERN`
+ 3. **Prerequisites**:
+ * You must have `git`, `gpg`, `svn`, and the `build` Python module
installed.
+ * Your GPG key and SVN access must be configured for your Apache ID.
+ 4. **Run the Script**:
+ Open your terminal, navigate to the root of your project directory,
and run the script with the desired version, release candidate number, and
Apache ID.
+
+
+ python apache_release_helper.py 1.2.3 0 your_apache_id
+ """
+ parser = argparse.ArgumentParser(description="Automates parts of the
Apache release process.")
+ parser.add_argument("version", help="The new release version (e.g.,
'1.0.0').")
+ parser.add_argument("rc_num", help="The release candidate number (e.g.,
'0' for RC0).")
+ parser.add_argument("apache_id", help="Your apache user ID.")
+ args = parser.parse_args()
+
+ version = args.version
+ rc_num = args.rc_num
+ apache_id = args.apache_id
+
+ check_prerequisites()
+
+ current_version = get_version_from_file(VERSION_FILE)
+ print(current_version)
+ if current_version != version:
+ print("Update the version in the version file to match the expected
version.")
+ sys.exit(1)
+
+ tag_name = f"v{version}-incubating-RC{rc_num}"
+ print(f"\nChecking for git tag '{tag_name}'...")
+ try:
+ # Check if the tag already exists
+ existing_tag = subprocess.check_output(["git", "tag", "-l",
tag_name]).decode().strip()
+ if existing_tag == tag_name:
+ print(f"Git tag '{tag_name}' already exists.")
+ response = input("Do you want to continue without creating a new
tag? (y/n): ").lower()
+ if response != "y":
+ print("Aborting.")
+ sys.exit(1)
+ else:
+ # Tag does not exist, create it
+ print(f"Creating git tag '{tag_name}'...")
+ subprocess.run(["git", "tag", tag_name], check=True)
+ print(f"Git tag {tag_name} created.")
+ except subprocess.CalledProcessError as e:
+ print(f"Error checking or creating Git tag: {e}")
+ sys.exit(1)
+
+ # Create artifacts
+ main_archive_files, sf_hamilton_archive_files =
create_release_artifacts(version)
+ if not main_archive_files:
+ sys.exit(1)
+
+ # Upload artifacts
+ # NOTE: You MUST have your SVN client configured to use your Apache ID and
have permissions.
+ svn_url = svn_upload(version, rc_num, main_archive_files,
sf_hamilton_archive_files, apache_id)
+ if not svn_url:
+ sys.exit(1)
+
+ # Generate email
+ generate_email_template(version, rc_num, svn_url)
+
+ print("\nProcess complete. Please copy the email template to your mailing
list.")
+
+
+if __name__ == "__main__":
+ main()
diff --git a/scripts/qualify.sh b/scripts/qualify.sh
new file mode 100755
index 00000000..60d1d9fc
--- /dev/null
+++ b/scripts/qualify.sh
@@ -0,0 +1,124 @@
+#!/bin/zsh
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+cd examples
+pushd hello_world
+pwd
+echo "-----hello world----"
+python my_script.py
+popd
+pushd numpy/air-quality-analysis
+pwd
+echo "-----Numpy----"
+python run_analysis.py
+popd
+pushd model_examples/scikit-learn
+pwd
+echo "---- ML example ---"
+python run.py iris svm
+popd
+pushd ray/hello_world
+pwd
+echo "-----ray-------"
+python run.py
+echo "----- ray workflow -----"
+python run_rayworkflow.py
+popd
+pushd dask/hello_world
+pwd
+echo "---- dask 1 ----"
+python run.py
+echo "----- dask with delayed ----"
+python run_with_delayed.py
+echo "---- dask delayed with objects ----"
+python run_with_delayed_and_dask_objects.py
+popd
+pushd data_quality/simple/
+echo "---- data quality simple----"
+python run.py
+echo "--- data quality ray ----"
+python run_ray.py
+echo "----- data quality dask ----"
+python run_dask.py
+popd
+pushd data_quality/pandera/
+echo "---- pandera simple ---"
+python run.py
+echo "---- pandera ray ---"
+python run_ray.py
+echo "---- pandera dask ---"
+python run_dask.py
+popd
+pushd lineage
+echo "---- lineage script ----"
+python lineage_script.py
+python lineage_commands.py PII
+python lineage_commands.py visualize upstream training_set_v1 tsv1
+python lineage_commands.py path age fit_random_forest
+popd
+pushd spark/pandas_on_spark
+echo "---- pandas on spark ----"
+python run.py
+popd
+pushd pandas/materialization
+echo "---- pandas materializers ----"
+python my_script.py
+rm -rf df.* # removes files created.
+popd
+echo "---- parallel examples -----"
+pushd parallelism/file_processing
+python run.py --mode dask
+python run.py --mode ray
+python run.py --mode multithreading
+python run.py --mode local
+popd
+echo "----- styling viz -----"
+pushd styling_visualization
+python run.py
+popd
+echo "---- ibis -------"
+pushd ibis/feature_engineering
+python run.py --level column # --model linear
+popd
+echo "---- scraping ----"
+pushd LLM_Workflows/scraping_and_chunking
+python run.py > scraping.out
+popd
+echo "---- experiment tracker ----"
+pushd experiment_management
+python run.py
+popd
+echo "---- polars test cases ----"
+pushd polars
+python my_script.py
+pushd lazyframe
+python my_script.py
+popd
+popd
+echo "---- schema test cases ---"
+pushd schema
+python dataflow.py
+popd
+echo "----- ui SDK test cases -- you need to have the UI running ---"
+pushd hamilton_ui
+python run.py --username [email protected] --project-id=1
+popd
+echo "----- Ray Tracker + SDK test cases -- you need to have the UI running
---"
+pushd ray/ray_Hamilton_UI_tracking
+python run.py
+popd
diff --git a/scripts/setup_keys.sh b/scripts/setup_keys.sh
new file mode 100755
index 00000000..15514bab
--- /dev/null
+++ b/scripts/setup_keys.sh
@@ -0,0 +1,95 @@
+#!/bin/bash
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# This script helps new Apache committers set up their GPG keys for releases.
+# It guides you through creating a new key, exports the public key, and
+# provides instructions on how to add it to your project's KEYS file.
+
+echo "========================================================"
+echo " Apache GPG Key Setup Script"
+echo "========================================================"
+echo " "
+echo "Step 1: Generating a new GPG key."
+echo " "
+echo "Please be aware of Apache's best practices for GPG keys:"
+echo "- **Key Type:** Select **(1) RSA and RSA**."
+echo "- **Key Size:** Enter **4096**."
+echo "- **Email Address:** Use your official **@apache.org** email address."
+echo "- **Passphrase:** Use a strong, secure passphrase."
+echo " "
+read -p "Press [Enter] to start the GPG key generation..."
+
+# Generate a new GPG key
+# The --batch and --passphrase-fd 0 options are used for automation,
+# but the script will still require interactive input.
+gpg --full-gen-key
+
+if [ $? -ne 0 ]; then
+ echo "Error: GPG key generation failed. Please check your GPG installation."
+ exit 1
+fi
+
+echo " "
+echo "Step 2: Listing your GPG keys to find the new key ID."
+echo "Your new key is listed under 'pub' with a string of 8 or 16 characters
after the '/'."
+
+# List all GPG keys
+gpg --list-keys
+
+echo " "
+read -p "Please copy and paste your new key ID here (e.g., A1B2C3D4 or
1234ABCD5678EF01): " KEY_ID
+
+if [ -z "$KEY_ID" ]; then
+ echo "Error: Key ID cannot be empty. Exiting."
+ exit 1
+fi
+
+echo " "
+echo "Step 3: Exporting your public key to a file."
+
+# Export the public key in ASCII armored format
+gpg --armor --export "$KEY_ID" > "$KEY_ID.asc"
+
+if [ $? -ne 0 ]; then
+ echo "Error: Public key export failed. Please ensure the Key ID is correct."
+ rm -f "$KEY_ID.asc"
+ exit 1
+fi
+
+echo "Checking out dist repository to update KEYS file"
+svn checkout --depth immediates https://dist.apache.org/repos/dist dist
+cd dist/release
+svn checkout https://dist.apache.org/repos/dist/release/incubator/hamilton
incubator/hamilton
+
+cd ../../
+gpg --list-keys "$KEY_ID" >> dis/release/incubator/hamilton/KEYS
+cat "$KEY_ID.asc" >> dis/release/incubator/hamilton/KEYS
+cd dist/release/incubator/hamilton
+
+echo " "
+echo "========================================================"
+echo " Setup Complete!"
+echo "========================================================"
+echo "Your public key has been saved to: $KEY_ID.asc"
+echo " "
+echo "NEXT STEPS (VERY IMPORTANT):"
+echo "1. Please inspect the KEYS file to ensure the new key is added
correctly. It should be in the current directory."
+echo "2. If all good run: svn update KEYS && svn commit -m \"Adds new key
$KEY_ID for YOUR NAME\""
+echo "3. Inform the mailing list that you've updated the KEYS file."
+echo " The updated KEYS file is essential for others to verify your release
signatures."
+echo " "
