satwikmishra11 opened a new pull request, #15624:
URL: https://github.com/apache/datafusion/pull/15624
### **Description**
This PR introduces an end-to-end performance monitoring system for Apache
DataFusion, automating benchmark execution on cloud infrastructure (AWS/GCP),
integrating regression alerts with GitHub, and providing a React-based
dashboard for performance analysis.
**Key Changes**:
1. **Benchmark Automation**:
- GitHub Actions workflows to trigger benchmarks on PRs/releases.
- Terraform scripts for provisioning EC2 Spot Instances (AWS) or
Preemptible VMs (GCP).
2. **Regression Detection**:
- GitHub Checks API integration to flag performance regressions in PRs.
- Results stored in PostgreSQL + S3 for historical tracking.
3. **Dashboard**:
- React frontend with Plotly visualizations for comparing query
performance across versions.
**Dependencies**:
- Terraform >= 1.5.0
- React >= 18.0
- `@datafusion-dev/client` (new SDK for querying benchmark results)
---
### **Code Snippets**
#### Terraform Script (AWS)
```hcl
# infra/aws/benchmark.tf
resource "aws_instance" "benchmark_runner" {
ami = "ami-0c55b159cbfafe1f0"
instance_type = "c5.4xlarge"
spot_price = "0.15"
wait_for_fulfillment = true
tags = {
Name = "datafusion-benchmark-runner"
}
}
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For additional commands, e-mail: github-h...@datafusion.apache.org
