wpf375516041 commented on issue #1392: URL: https://github.com/apache/datafusion-sqlparser-rs/issues/1392#issuecomment-3424484676
> > I am not quite sure what you mean by "partial parse of a sql string" -- maybe you mean the parser would be able to produce an AST even for a partial SQL query? > > The goal would be to be error tolerant yes. Instead of bailing out eagerly when an unexpected token is encountered, an invalid AST node can be produced, and the parsing can continue. > > Taking the following example: > > SELECT col_name FROM > The parser expects a [table factor](https://github.com/apache/datafusion-sqlparser-rs/blob/7461d8bd02701de55960851c531c1b5326a737c9/src/parser/mod.rs#L13548), but will return an error early. > > One option could be to still produce a `TableFactor::Table` node, with an empty name. Some additional metadata can also be added to indicate that it is an invalid node. > > By successfully parsing the given SQL query, editors can more easily provide completions and other LSP features, without having to rely solely on tokens. > > For reference, this is how the Ruff's Python parser do it. With the following code: > > a = > It produces a `StmtAssign` node, with the assigned value being an `Name` node, with an empty string, and `Invalid` set as `ctx` ([playground)](https://play.ruff.rs/c127b331-e213-48e6-813a-45e4b955ba52?secondary=AST). > > Introducing such an error tolerant mechanism is a lot of work, although can be done incrementally. However, this may require some breaking changes, in particular in the returned value type of the parsing functions. It currently returns the AST, but would ideally return a parsed result struct, containing the AST and a list of encountered errors. Any feedback on whether there is interest in having this introduced in the parser is welcomed! The question seems to have deviated somewhat. The original goal of this issue was to provide two built-in capabilities: 1. Extracting keywords, such as returning an array of ['select', 'from', ...] 2. SQL formatting Based on my observations at the time, we only needed to provide an enhanced visitor similar to the one used in the Alibaba/Druid projects. For SQL engine developers, these two capabilities can be achieved using a basic visitor, but for general developers, it seems a bit difficult. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
