Hi everyone, As my Google Summer of Code 2025 project with Apache Beam comes to a
close, I'm excited to share a summary of the work I've done. My project focused on
building a new set of tools to automate our infrastructure, improve security, and
make managing our GCP resources much easier. I developed a full suite of tools that
work together to create a more secure, efficient, and cost-effective infrastructure
for the project. What I Delivered Automated Resource Cleaner: I built a tool that
automatically finds and removes old, unused GCP Pub/Sub topics and subscriptions.
This will help reduce clutter and save on costs without manual intervention.
Git-Based Access Control: I implemented a more transparent way to manage GCP
permissions. All access control is now handled through a central users.yml file
using Terraform, which means all changes are managed via pull requests, creating a
clear audit trail. Automatic Key Rotation & Compliance: To boost security, I
built a framework that automatically rotates service account keys on a schedule. I
also created an Infrastructure Enforcer that runs regular checks to ensure our
configurations comply with these new standards. It's currently in a dry-run mode,
but the plan is for it to send email notifications to this list for any issues it
finds. Security Monitoring: Finally, I set up a security log analyzer to monitor
GCP audit logs for suspicious activity related to access control and service
account keys. It's designed to send weekly reports and is currently in its final
testing stage How This Affects You To Request GCP Access: If you need to request or
change GCP permissions, the process is now managed entirely through a pull request.
Simply submit your changes to the infra/iam/users.yml configuration file. To
Request a Service Account Key: Similarly, if you need a service account managed by
this new system, please create a pull request with your change in
infra/keys/keys.yaml This has been an incredible learning experience, and I want to
extend a huge thank you to my mentor @pabloem and the entire Apache Beam community
for your support and guidance throughout the summer. If you’re interested in the
technical details, you can find them in the links below. I've also added a summary
of this work to the latest community draft report. GSoC Project Page:
https://summerofcode.withgoogle.com/programs/2025/projects/QRKMhW67 Final Work
Report: https://gist.github.com/ksobrenat32/b028b8303393afbe73a8fc5e17daff90
Community Draft Report Contribution: https://s.apache.org/beam-draft-report-2025-09
Best regards, Enrique Calderon
