Skip to content

Re-enable tests for FIRST/LAST #1646

@andygrove

Description

@andygrove

What is the problem the feature request solves?

During the upgrade to DataFusion 47.0.0 () it was necessary to disable some tests that use FIRST and LAST because the behavior of these functions changed. These functions are non-determistic in Spark but previous versions of DataFusion had the same behavior as Spark for the single partition case used in the unit tests.

This issue is for re-enabling the tests somehow, or for implementing a Comet-specific implementation of FIRST/LAST to match Spark behavior (just for the single partition case used in tests).

These are the tests (all in CometAggregateSuite):

  ignore("single group-by column + aggregate column, multiple batches, no null") {
  ignore("multiple group-by columns + single aggregate column (first/last), with nulls") {
  ignore("multiple group-by columns + multiple aggregate column (first/last), with nulls") {
  ignore("all types first/last, with nulls") {
  ignore("first/last") {

Describe the potential solution

No response

Additional context

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions