Re: [I] Plan to replace `SchemaAdapter` with `PhysicalExprAdapter` [datafusion]

[via GitHub]

adriangb commented on issue #16800:
URL: https://github.com/apache/datafusion/issues/16800#issuecomment-3080077103

   Here's an example from Comet: 
https://github.com/vaibhawvipul/datafusion-comet/blob/main/native/core/src/parquet/schema_adapter.rs.
 As you can see it's _a lot_ of code with [a lot of 
duplication](https://github.com/vaibhawvipul/datafusion-comet/blob/main/native/core/src/parquet/schema_adapter.rs#L150-L152).
 The change will essentially be to re-implement that in terms of 
`PhysicalExprAdapter` which I think should be a lot less code and duplication. 
Maybe something like this (very much pseudocode):
   
   ```rust
   /// A cast kernel that respect's Spark's casting rules.
   pub struct SparkCast {
        /// The expression to cast
       pub expr: Arc<dyn PhysicalExpr>,
       /// The data type to cast to
       cast_type: DataType,
       /// Cast options
       cast_options: CastOptions<'static>,
   }
   
   impl PhysicalExpr for CastExpr {
       ...
       fn evaluate(&self, batch: &RecordBatch) -> Result<ColumnarValue> {
           let value = self.expr.evaluate(batch)?;
           spark_cast(value, &self.cast_type, Some(&self.cast_options))
       }
       ...
   }
   
   struct SparkPhysicalExprAdapter {
      inner: DefaultPhysicalExprAdapter,
   }
   
   impl PhysicalExprAdapter for SparkPhysicalExprAdapter {
     fn rewrite(&self, mut expr: Arc<dyn PhysicalExpr>) -> Result<Arc<dyn 
PhysicalExpr>> {
       expr = self.inner.rewrite(expr);
       expr.transform(|expr| {
           if let Some(cast) = expr.as_any().downcast_ref::<CastExpr>() {
               return Ok(Transformed::yes(SparkCastExpr::from(cast))
           }
           Ok(Transformed::no(expr))
      }).data()
    }
   }
   ```
              


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For additional commands, e-mail: github-h...@datafusion.apache.org