jayzhan211 commented on issue #14123: URL: https://github.com/apache/datafusion/issues/14123#issuecomment-2688082518
> Would this imply that LogicalPlan is closer to a "resolved" representation of the query yes, this is our goal. > but such capability cannot be leveraged for projects that builds LogicalPlan from non-SQL inputs (e.g. Sail). So I'm wondering if it's possible to improve the modularity of logical analysis and benefit downstream projects. Do you have thoughts around this ``` In SQL, the transformation follows this path: SQL → Expr → LogicalPlan For DataFrame operations, the transformation is: Expr → LogicalPlan Your case: Other SQL -> Expr -> LogicalPlan ``` Even if we introduce an additional rewrite rule between Expr -> LogicalPlan, it may not provide much benefit. For the count wildcard rule, the count_all() API is explicitly expected to be called, so there isn’t a strong use case for rewriting other expressions into count_all(). Instead, introducing a more user-friendly DataFrame API seems like a better approach. With resolved Expr (`Expr` what we have after SQL -> Expr in Datafusion), the need for a rewrite is eliminated, reducing unnecessary processing overhead. > It seems that SqlToRel is acquiring more capability for logical analysis I wouldn’t describe this as logical analysis—it’s more about extensibility with custom rewrite rules rather than performing analysis. Since this applies to SQL -> Expr, it’s not relevant for those who already know exactly which `Expr` they need. Also, we have logical optimizer, which is possible for extensibility by adding your own rewrite rule after the LogicalPlan is built. If you need something similar to count wildcard rewrite rule in analyzer before, it is in theory possible to have it in optimizer rule too. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
