andygrove opened a new issue, #3182:
URL: https://github.com/apache/datafusion-comet/issues/3182
## What is the problem the feature request solves?
> **Note:** This issue was generated with AI assistance. The specification
details have been extracted from Spark documentation and may need verification.
Comet does not currently support the Spark `right` function, causing queries
using this function to fall back to Spark's JVM execution instead of running
natively on DataFusion.
The `Right` expression extracts a specified number of characters from the
right side of a string. It is implemented as a `RuntimeReplaceable` expression
that transforms into conditional logic using `Substring` operations during
query planning.
Supporting this expression would allow more Spark workloads to benefit from
Comet's native acceleration.
## Describe the potential solution
### Spark Specification
**Syntax:**
```sql
RIGHT(str, len)
```
```scala
// DataFrame API
import org.apache.spark.sql.functions._
df.select(expr("RIGHT(column_name, length)"))
```
**Arguments:**
| Argument | Type | Description |
|----------|------|-------------|
| str | String | The input string from which to extract characters |
| len | Integer | The number of characters to extract from the right side |
**Return Type:** Returns a `StringType` with the same collation as the input
string.
**Supported Data Types:**
- **Input string**: `StringTypeWithCollation` (supports trim collation)
- **Length parameter**: `IntegerType`
**Edge Cases:**
- **Null input**: Returns null when the input string is null
- **Zero or negative length**: Returns empty string
(`UTF8String.EMPTY_UTF8`) when len ≤ 0
- **Length greater than string length**: Returns the entire string when
requested length exceeds string length
- **Empty string input**: Returns empty string regardless of length parameter
- **Type coercion**: Automatically casts compatible types through
`ImplicitCastInputTypes`
**Examples:**
```sql
-- Extract last 3 characters
SELECT RIGHT('Hello World', 3); -- Returns 'rld'
-- Handle edge cases
SELECT RIGHT('abc', 0); -- Returns ''
SELECT RIGHT('abc', -1); -- Returns ''
SELECT RIGHT('abc', 10); -- Returns 'abc'
SELECT RIGHT(NULL, 3); -- Returns NULL
```
```scala
// DataFrame API usage
import org.apache.spark.sql.functions._
df.select(expr("RIGHT(name, 3)"))
df.select(expr("RIGHT(description, 5)").as("suffix"))
// With column references
df.select(expr("RIGHT(text_column, length_column)"))
```
### Implementation Approach
See the [Comet guide on adding new
expressions](https://datafusion.apache.org/comet/contributor-guide/adding_a_new_expression.html)
for detailed instructions.
1. **Scala Serde**: Add expression handler in
`spark/src/main/scala/org/apache/comet/serde/`
2. **Register**: Add to appropriate map in `QueryPlanSerde.scala`
3. **Protobuf**: Add message type in `native/proto/src/proto/expr.proto` if
needed
4. **Rust**: Implement in `native/spark-expr/src/` (check if DataFusion has
built-in support first)
## Additional context
**Difficulty:** Small
**Spark Expression Class:** `org.apache.spark.sql.catalyst.expressions.Right`
**Related:**
- `Left` - Extract characters from the left side of a string
- `Substring` - Extract characters from any position in a string
- `Length` - Get the length of a string
- `Trim` - Remove whitespace from strings
---
*This issue was auto-generated from Spark reference documentation.*
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
