Skip to content

failed to filter with compatible type #605

@Sunyue

Description

@Sunyue

Describe the bug, including details regarding any error messages, version, and platform.

When we are trying to upgrade to the latest release, we saw some regression in our unit test. I created a simple example below:

import (
	"testing"

	"github.com/apache/arrow-go/v18/arrow"
	"github.com/apache/arrow-go/v18/arrow/array"
	"github.com/apache/arrow-go/v18/arrow/compute"
	"github.com/apache/arrow-go/v18/arrow/compute/exprs"
	"github.com/apache/arrow-go/v18/arrow/memory"
	"github.com/stretchr/testify/require"
	"github.com/substrait-io/substrait-go/v7/expr"
)

func Test_FilterOnFloat(t *testing.T) {
	t.Parallel()

	tt := []struct {
		name string
	}{
		{
			name: "this was ok",
		},
	}

	for _, tc := range tt {
		t.Run(tc.name, func(t *testing.T) {
			t.Parallel()

			ctx := t.Context()
			rq := require.New(t)

			typ := arrow.PrimitiveTypes.Float32
			field := arrow.Field{
				Name:     "col",
				Type:     typ,
				Nullable: true,
			}
			schema := arrow.NewSchema([]arrow.Field{field}, nil)

			db := array.NewFloat32Builder(memory.DefaultAllocator)
			defer db.Release()

			db.Append(float32(1.1))
			db.Append(float32(5.5))
			db.Append(float32(10.1))

			rec := array.NewRecordBatch(schema, []arrow.Array{db.NewArray()}, int64(3))

			extSet := exprs.GetExtensionIDSet(ctx)
			builder := exprs.NewExprBuilder(extSet)

			err := builder.SetInputSchema(schema)
			rq.NoError(err, "Failed to set input schema")

			b, err := builder.CallScalar("less", nil,
				builder.FieldRef("col"),
				builder.Literal(expr.NewPrimitiveLiteral(float64(5.5), true)),
			)

			rq.NoError(err, "Failed to call scalar")

			e, err := b.BuildExpr()
			rq.NoError(err, "Failed to build expression")

			ctx = exprs.WithExtensionIDSet(ctx, extSet)

			dr := compute.NewDatum(rec)
			defer dr.Release()

			_, err = exprs.ExecuteScalarExpression(ctx, schema, e, dr)
			rq.NoError(err, "Failed to execute scalar expression")
		})
	}
}

Basically in previous releases, we were able to filter an arrow Float32 value with float64 value, or arrow Int32 value with int64 value. This now gives error "type parameter any1 cannot be both fp32 and fp64" with the latest release. Could you help to have a look? Thanks

Component(s)

Release, Other

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type: bugSomething isn't working

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions