Re: [I] [FeatureRequest] Inquiry Regarding Hudi Exporter with SQL Transformer for Data Filtering [hudi]

Fri, 26 Jul 2024 08:44:56 -0700 


soumilshah1995 commented on issue #10644:
URL: https://github.com/apache/hudi/issues/10644#issuecomment-2253027370

   tested with Master Branch 
   Test passed 
   
   ```
   
   spark-submit \
       --class org.apache.hudi.utilities.HoodieSnapshotExporter \
       --packages org.apache.hudi:hudi-spark3.4-bundle_2.12:1.0.0-beta2 \
       --master 'local[*]' \
       --executor-memory 1g \
       
/Users/soumilshah/IdeaProjects/SparkProject/apache-hudi-delta-streamer-labs/E1/jar/hudi-utilities-slim-bundle_2.12-1.0.0-beta2.jar
 \
       --source-base-path 
'/Users/soumilshah/IdeaProjects/SparkProject/apache-hudi-delta-streamer-labs/E1/silver/'
 \
       --target-output-path 
'/Users/soumilshah/IdeaProjects/SparkProject/apache-hudi-delta-streamer-labs/E1/dump/parquet/'
 \
       --transformer-class 
"org.apache.hudi.utilities.transform.SqlQueryBasedTransformer" \
       --transformer-sql "SELECT * FROM <SRC> WHERE destinationstate='NY'" \
       --output-format 'parquet'
   ```
   
   
![image](https://github.com/user-attachments/assets/e786d0eb-9b25-4b35-896d-ef0ba64fbf67)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to