[Feature][SQL function] Enhance SQL Transform COALESCE functions to support type cast #9299

ocean-zhc · 2025-05-09T10:19:51Z

…upport type cast

Purpose of this pull request

Does this PR introduce any user-facing change?

How was this patch tested?

Check list

If any new Jar binary package adding in your PR, please add License Notice according
New License Guide
If necessary, please update the documentation to describe the new feature. https://github.com/apache/seatunnel/tree/dev/docs
If you are contributing the connector code, please check that the following files are updated:
1. Update plugin-mapping.properties and add new connector information in it
2. Update the pom file of seatunnel-dist
3. Add ci label in label-scope-conf
4. Add e2e testcase in seatunnel-e2e
5. Update connector plugin_config
Update the release-note.

…upport type cast

Copilot

Pull Request Overview

This pull request enhances the SQL Transform functions (COALESCE and IFNULL) to support type casting, ensuring that when arguments differ in type the correct type conversion is applied. Key changes include:

Adding test cases to verify type conversion behavior in COALESCE and IFNULL.
Extending SystemFunction with new overloads that accept a target type, along with a type-mapper table for casting.
Updating ZetaSQLFunction to determine and apply the target type for COALESCE/IFNULL calls and updating documentation accordingly.

Reviewed Changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
seatunnel-transforms-v2/src/test/java/org/apache/seatunnel/transform/sql/SQLTransformTest.java	Added tests for verifying COALESCE/IFNULL type conversion behavior.
seatunnel-transforms-v2/src/main/java/org/apache/seatunnel/transform/sql/zeta/functions/SystemFunction.java	Implements enhanced coalesce functions with explicit type conversion and introduces a type mapper.
seatunnel-transforms-v2/src/main/java/org/apache/seatunnel/transform/sql/zeta/ZetaSQLFunction.java	Modifies function evaluation to use enhanced system functions with target type conversion.
docs/zh/transform-v2/sql-functions.md	Updates documentation to reflect the new type conversion behavior.
docs/en/transform-v2/sql-functions.md	Updates documentation to reflect the new type conversion behavior.

Copilot · 2025-05-11T14:26:59Z

...nel-transforms-v2/src/main/java/org/apache/seatunnel/transform/sql/zeta/ZetaSQLFunction.java

@@ -202,6 +202,9 @@ public class ZetaSQLFunction {

    private final List<ZetaUDF> udfList;

+    // Add instance variable to track the current function being processed
+    private Function currentFunction = null;


[nitpick] Consider parameterizing the raw 'Function' type with appropriate generic types (e.g., Function<Expression, SeaTunnelDataType<?>>) to improve type safety and clarity.

Suggested change

private Function currentFunction = null;

private Function<Expression, SeaTunnelDataType<?>> currentFunction = null;

I can't change it like this, because I need the Function class of JSqlParser to be changed, and if I change it, it will become the Functional Interface of Java, and the two types are completely different.

Hisoka-X · 2025-05-12T03:25:48Z

...nel-transforms-v2/src/main/java/org/apache/seatunnel/transform/sql/zeta/ZetaSQLFunction.java

+                        SeaTunnelDataType<?> targetType =
+                                zetaSQLType.getExpressionType(currentFunction);
+                        return SystemFunction.coalesce(args, targetType);


why we should get targetType from currentFunction not args[0]?

If the first argument is NULL (which is common in COALESCE and IFNULL functions), getting the type information from args[0] is unreliable because NULL itself has no specific type.

Hisoka-X · 2025-05-12T03:40:28Z

...forms-v2/src/main/java/org/apache/seatunnel/transform/sql/zeta/functions/SystemFunction.java

+
+        if (firstArg != null) {
+            Function<Object, SeaTunnelDataType<?>> typeMapper =
+                    TYPE_MAPPERS.get(firstArg.getClass());


we can reuse ZetaSQLType::getExpressionType

If it is not null, it can be obtained directly from the parameter. If you want to maintain a high degree of consistency in the code, it is better to change it to ZetaSQLType::getExpressionType

we can reuse ZetaSQLType::getExpressionType

+1

Hisoka-X · 2025-05-15T12:10:05Z

...nel-transforms-v2/src/main/java/org/apache/seatunnel/transform/sql/zeta/ZetaSQLFunction.java

@@ -202,6 +202,9 @@ public class ZetaSQLFunction {

    private final List<ZetaUDF> udfList;

+    // Add instance variable to track the current function being processed
+    private Function currentFunction = null;


I think we don't need currentFunction any more.

seatunnel-transforms-v2/src/main/java/org/apache/seatunnel/transform/sql/zeta/ZetaSQLType.java

Hisoka-X · 2025-05-15T12:12:46Z

seatunnel-transforms-v2/src/main/java/org/apache/seatunnel/transform/sql/zeta/ZetaSQLType.java

+        if (parameters == null) {
+            throw new TransformException(
+                    CommonErrorCodeDeprecated.UNSUPPORTED_OPERATION,
+                    "COALESCE function requires at least one parameter");


Suggested change

"COALESCE function requires at least one parameter");

function.getName() + " function requires at least one parameter");

Hisoka-X · 2025-05-15T12:14:06Z

...forms-v2/src/main/java/org/apache/seatunnel/transform/sql/zeta/functions/SystemFunction.java

+                } else {
+                    throw new TransformException(
+                            CommonErrorCodeDeprecated.UNSUPPORTED_OPERATION,
+                            String.format("Unsupported CAST to INTEGER: %s", v1));


Suggested change

String.format("Unsupported CAST to INTEGER: %s", v1));

String.format("Unsupported CAST %s to INTEGER", v1));

Hisoka-X · 2025-05-15T12:14:15Z

...forms-v2/src/main/java/org/apache/seatunnel/transform/sql/zeta/functions/SystemFunction.java

+                } else {
+                    throw new TransformException(
+                            CommonErrorCodeDeprecated.UNSUPPORTED_OPERATION,
+                            String.format("Unsupported CAST to LONG: %s", v1));


Hisoka-X · 2025-05-15T12:14:27Z

...forms-v2/src/main/java/org/apache/seatunnel/transform/sql/zeta/functions/SystemFunction.java

@@ -203,4 +235,29 @@ public static Object castAs(List<Object> args) {
                CommonErrorCodeDeprecated.UNSUPPORTED_OPERATION,
                String.format("Unsupported CAST AS type: %s", v2));
    }
+
+    // TYPE-MAP-TABLE
+    private static final Map<Class<?>, Function<Object, SeaTunnelDataType<?>>> TYPE_MAPPERS =


we can delete it now?

Referring to the implementation as shown, but I've modified it as suggested

Hisoka-X

LGTM if ci passes.

Copilot

Pull Request Overview

This PR enhances the SQL Transform COALESCE function to support type casting and updates related functions and documentation.

Adds new test cases for COALESCE behavior with different data types and null parameters.
Updates SystemFunction, ZetaSQLType, and ZetaSQLFunction to incorporate a target type for conversions.
Revises the documentation (both Chinese and English) to clarify the behavior of COALESCE with type conversions.

Reviewed Changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
seatunnel-transforms-v2/src/test/java/org/apache/seatunnel/transform/sql/SQLTransformTest.java	Adds comprehensive tests for COALESCE behavior under various scenarios.
seatunnel-transforms-v2/src/main/java/org/apache/seatunnel/transform/sql/zeta/functions/SystemFunction.java	Updates coalesce and ifnull functions to support type casting logic.
seatunnel-transforms-v2/src/main/java/org/apache/seatunnel/transform/sql/zeta/ZetaSQLType.java	Revises function type determination for COALESCE with a helper for expression extraction.
seatunnel-transforms-v2/src/main/java/org/apache/seatunnel/transform/sql/zeta/ZetaSQLFunction.java	Adjusts function invocations to pass the expression to determine target type.
docs/zh/transform-v2/sql-functions.md	Enhances documentation to explain type conversion behavior in COALESCE.
docs/en/transform-v2/sql-functions.md	Updates documentation to include type conversion examples for the COALESCE function.

Copilot · 2025-05-26T03:43:00Z

seatunnel-transforms-v2/src/test/java/org/apache/seatunnel/transform/sql/SQLTransformTest.java

+        Exception exception =
+                Assertions.assertThrows(
+                        Exception.class,


Consider using a more specific exception type (e.g., TransformException) instead of the generic Exception when testing non-existent columns so that the test precisely captures the expected failure condition.

Suggested change

Exception exception =

Assertions.assertThrows(

Exception.class,

TransformException exception =

Assertions.assertThrows(

TransformException.class,

Copilot · 2025-05-26T03:43:01Z

...forms-v2/src/main/java/org/apache/seatunnel/transform/sql/zeta/functions/SystemFunction.java

            case "INTEGER":
-                return Integer.parseInt(v1.toString());
+                if (v1 instanceof String) {
+                    return Integer.parseInt(v1.toString());
+                } else if (v1 instanceof Number) {
+                    return ((Number) v1).intValue();


[nitpick] Consider adding comments to clarify the conversion logic in this section for 'INT' and 'LONG' types to improve code readability and facilitate future maintenance.

Suggested change

case "INTEGER":

return Integer.parseInt(v1.toString());

if (v1 instanceof String) {

return Integer.parseInt(v1.toString());

} else if (v1 instanceof Number) {

return ((Number) v1).intValue();

case "INTEGER":

// Handle conversion to INT/INTEGER type

// If the input is a String, attempt to parse it as an Integer

if (v1 instanceof String) {

return Integer.parseInt(v1.toString());

// If the input is a Number, extract its integer value

} else if (v1 instanceof Number) {

return ((Number) v1).intValue();

// Throw an exception for unsupported types

corgy-w · 2025-05-28T14:17:02Z

@ocean-zhc Try again and resolve conflicts

[Feature][SQL function] Enhance SQL Transform COALESCE functions to s…

55d38a8

…upport type cast

github-actions bot added document Transform-v2 e2e labels May 9, 2025

hailin0 linked an issue May 9, 2025 that may be closed by this pull request

[Feature][SQL function] Enhance SQL Transform COALESCE functions to support type cast #9295

Open

9 tasks

Optimization Code

aedb07d

github-actions bot removed the e2e label May 9, 2025

hailin0 requested a review from Copilot May 11, 2025 14:26

Copilot AI reviewed May 11, 2025

View reviewed changes

ocean-zhc added 2 commits May 12, 2025 09:18

Merge branch 'apache:dev' into COALESCE

3049b48

Merge branch 'apache:dev' into COALESCE

9929d53

Hisoka-X reviewed May 12, 2025

View reviewed changes

ocean-zhc and others added 6 commits May 12, 2025 15:45

Optimization Code

d7a7e04

Optimization Code

7480933

Optimization Code

056711e

Optimization Code

a2f7fdf

Merge branch 'apache:dev' into COALESCE

58ebde7

Merge branch 'apache:dev' into COALESCE

24cd8d5

Hisoka-X reviewed May 15, 2025

View reviewed changes

ocean-zhc and others added 6 commits May 19, 2025 09:17

Merge branch 'apache:dev' into COALESCE

c676437

Optimization Code

a4ecc4b

Merge branch 'apache:dev' into COALESCE

dc4e1d2

Merge branch 'apache:dev' into COALESCE

1cbb7a4

Merge branch 'apache:dev' into COALESCE

b2b7ce4

Optimization Code

9ff4f52

Hisoka-X approved these changes May 22, 2025

View reviewed changes

github-actions bot added approved reviewed labels May 22, 2025

nielifeng requested a review from Copilot May 26, 2025 03:42

Copilot AI reviewed May 26, 2025

View reviewed changes

Merge branch 'apache:dev' into COALESCE

6389e59

	private Function currentFunction = null;
	private Function<Expression, SeaTunnelDataType<?>> currentFunction = null;

	"COALESCE function requires at least one parameter");
	function.getName() + " function requires at least one parameter");

	String.format("Unsupported CAST to INTEGER: %s", v1));
	String.format("Unsupported CAST %s to INTEGER", v1));

[Feature][SQL function] Enhance SQL Transform COALESCE functions to support type cast #9299

Are you sure you want to change the base?

[Feature][SQL function] Enhance SQL Transform COALESCE functions to support type cast #9299

Uh oh!

Conversation

ocean-zhc commented May 9, 2025

Purpose of this pull request

Does this PR introduce any user-facing change?

How was this patch tested?

Check list

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Copilot AI May 11, 2025

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ocean-zhc May 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Hisoka-X left a comment

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Copilot AI May 26, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI May 26, 2025

Choose a reason for hiding this comment

Uh oh!

corgy-w commented May 28, 2025

Uh oh!

Uh oh!

ocean-zhc May 12, 2025 •

edited

Loading