fix: Bug fix (#321)

---------

Co-authored-by: Krishna mohan Pulipati <kpulipati@expediagroup.com>

* feat: Redis range query materialization (#315)

* feat: Redis range query materialization

* fex: fix requirements

* feat: Add batching and collocation

* fix: fix formatting

* fix: Add comment

* fix: remove collocation

* fix: add requirements files

* fix: fix formatting

* Update redis.py

* Update test_redis.py

* Update test_redis.py

---------

Co-authored-by: Krishna mohan Pulipati <kpulipati@expediagroup.com>

* feat: Add Valkey changes (#317)

* feat: Redis range query materialization

* fex: fix requirements

* feat: Add batching and collocation

* fix: fix formatting

* fix: Add comment

* fix: remove collocation

* fix: add requirements files

* fix: fix formatting

* Update redis.py

* Update test_redis.py

* Update test_redis.py

* feat: Added valkey changes

* fix: fix formatting

---------

Co-authored-by: Krishna mohan Pulipati <kpulipati@expediagroup.com>

* fix: Exclude docker tests from macos (#319)

* feat: Redis range query materialization

* fex: fix requirements

* feat: Add batching and collocation

* fix: fix formatting

* fix: Add comment

* fix: remove collocation

* fix: add requirements files

* fix: fix formatting

* Update redis.py

* Update test_redis.py

* Update test_redis.py

* feat: Added valkey changes

* fix: fix formatting

* fix: exclude docker tests from macos

* fix : fix formatting

* fix : separate repo config with and without connection string

* fix : separate repo config with and without connection string

---------

Co-authored-by: Krishna mohan Pulipati <kpulipati@expediagroup.com>

* fix: Rename repo config (#320)

* feat: Redis range query materialization

* fex: fix requirements

* feat: Add batching and collocation

* fix: fix formatting

* fix: Add comment

* fix: remove collocation

* fix: add requirements files

* fix: fix formatting

* Update redis.py

* Update test_redis.py

* Update test_redis.py

* feat: Added valkey changes

* fix: fix formatting

* fix: exclude docker tests from macos

* fix : fix formatting

* fix : separate repo config with and without connection string

* fix : separate repo config with and without connection string

* fix : rename repoconfig

* fix : formatting

* fix : fix tests

---------

Co-authored-by: Krishna mohan Pulipati <kpulipati@expediagroup.com>

* feat: Added Online Read Range functionality for Redis (#318)

* added redis online read range functionality

* using reverse instead of ZREVRANGE

* adding reverse logic and valkey implementation

* adding comments

* moved util functionality into utils and addressed PR comments

* reverting sample local changes

---------

Co-authored-by: Manisha Sudhir <msudhir@expediagroup.com>

* feat: Added Integration Tests for Valkey OnlineReadRange testing (#323)

* added integration tests

* Remove accidentally committed feast binary

* adding ignore build integration

---------

Co-authored-by: Manisha Sudhir <msudhir@expediagroup.com>

* feat: Ttl changes (#311)

* ttl changes

* fix linting error

* fixing the tests

* updated the threading logic

* fixing a typo

* fix linting error

* cleanup

* cleanup

* reformatting

* pull max_evnts from feature view tags

* fix import

* Refactoring and added more unit tests

* fix: formatting

* formatting

* fix tests

* formatting

* formatting

* updated based on the materilaization changes

* formatting

* formatting

* added valkey changes

* bug fix

* Address review comments

* type

* fix typo

* formatting

* fix failing unit tests

* update tests

* fix: Count ttl command

* bug fix

* Update sdk/python/feast/infra/online_stores/redis.py

Co-authored-by: kpulipati29 <kpulipati29@gmail.com>

* address review comments

* lint fixes

* fix: Enable event time ttl

* fix: Use event timestamp to set ttl in redis hash

* fix: linting

* fix: imports

* fix: imports

* fix: Small fix

* fix: Small fix

* fix: exception handling and some cleanup

* fix: formatting fix

* fix: lint fix

* fix: Reduce batch size

* fix: reduce batch size

* fix: dont expire if ttl is zerp

---------

Co-authored-by: vbhagwat <vbhagwat@expediagroup.com>
Co-authored-by: Krishna mohan Pulipati <kpulipati@expediagroup.com>
Co-authored-by: kpulipati29 <kpulipati29@gmail.com>

* fix: Fix minor bug (#324)

* ttl changes

* fix linting error

* fixing the tests

* updated the threading logic

* fixing a typo

* fix linting error

* cleanup

* cleanup

* reformatting

* pull max_evnts from feature view tags

* fix import

* Refactoring and added more unit tests

* fix: formatting

* formatting

* fix tests

* formatting

* formatting

* updated based on the materilaization changes

* formatting

* formatting

* added valkey changes

* bug fix

* Address review comments

* type

* fix typo

* formatting

* fix failing unit tests

* update tests

* fix: Count ttl command

* bug fix

* Update sdk/python/feast/infra/online_stores/redis.py

Co-authored-by: kpulipati29 <kpulipati29@gmail.com>

* address review comments

* lint fixes

* fix: Enable event time ttl

* fix: Use event timestamp to set ttl in redis hash

* fix: linting

* fix: imports

* fix: imports

* fix: Small fix

* fix: Small fix

* fix: exception handling and some cleanup

* fix: formatting fix

* fix: lint fix

* fix: Reduce batch size

* fix: reduce batch size

* fix: dont expire if ttl is zerp

* fix: add results for sfv

---------

Co-authored-by: vbhagwat <vbhagwat@expediagroup.com>
Co-authored-by: vanitabhagwat <92561664+vanitabhagwat@users.noreply.github.com>
Co-authored-by: Krishna mohan Pulipati <kpulipati@expediagroup.com>

* removing limit on filter and adding it on retrieval (#326)

Co-authored-by: Manisha Sudhir <msudhir@expediagroup.com>

* modifying batch size behaviour (#327)

Co-authored-by: Manisha Sudhir <msudhir@expediagroup.com>

* fix: Adding Parallelization Logic (#328)

* adding parallelization logic

* modifying comment

* adding logs for debugging

* removing logs and cleaning up

* removing logs and cleaning up

* Add comments for better understandability

---------

Co-authored-by: Manisha4 <Manisha4@github.com>

* fix: Some bug fixes (#325)

* ttl changes

* fix linting error

* fixing the tests

* updated the threading logic

* fixing a typo

* fix linting error

* cleanup

* cleanup

* reformatting

* pull max_evnts from feature view tags

* fix import

* Refactoring and added more unit tests

* fix: formatting

* formatting

* fix tests

* formatting

* formatting

* updated based on the materilaization changes

* formatting

* formatting

* added valkey changes

* bug fix

* Address review comments

* type

* fix typo

* formatting

* fix failing unit tests

* update tests

* fix: Count ttl command

* bug fix

* Update sdk/python/feast/infra/online_stores/redis.py

Co-authored-by: kpulipati29 <kpulipati29@gmail.com>

* address review comments

* lint fixes

* fix: Enable event time ttl

* fix: Use event timestamp to set ttl in redis hash

* fix: linting

* fix: imports

* fix: imports

* fix: Small fix

* fix: Small fix

* fix: exception handling and some cleanup

* fix: formatting fix

* fix: lint fix

* fix: Reduce batch size

* fix: reduce batch size

* fix: dont expire if ttl is zerp

* fix: add results for sfv

* fix: increase batch size

* fix: fix a ttl bug

* fix: Remove ttl logic to see if it improves latency

* fix: some simplifications to ttl logic

* fix: remove count based ttl

* fix: add log

* fix: formatting

* fix: remove debug logs

* fix: typo in test case

* fix: linting

* fix: increase ttl to see if that fixes tests

* fix: lint

* fix: improve tests

---------

Co-authored-by: vbhagwat <vbhagwat@expediagroup.com>
Co-authored-by: vanitabhagwat <92561664+vanitabhagwat@users.noreply.github.com>
Co-authored-by: Krishna mohan Pulipati <kpulipati@expediagroup.com>

* fix: Update Redis Implementation to Match Valkey (#329)

* Port over valkey implementation to Redis

* Fix bug and update tests

* formatting fixes

* formatting fixes

---------

Co-authored-by: Manisha4 <Manisha4@github.com>

* Update featurestore_test.go

remove types2 import

---------

Author: kpulipati29

.github/workflows/unit_tests.yml

@@ -47,7 +47,7 @@ jobs:
         run: make test-python-unit
       - name: Test Python for Mac
         if: matrix.os != 'ubuntu-latest'
-        run: python -m pytest -n 8 --color=yes sdk/python/tests --ignore=sdk/python/tests/expediagroup
+        run: python -m pytest -n 8 --color=yes sdk/python/tests --ignore=sdk/python/tests/expediagroup -m "not docker"
       - name: Minimize uv cache
         run: uv cache prune --ci
 

go.mod

@@ -11,6 +11,7 @@ require (
 	github.com/DataDog/dd-trace-go/contrib/redis/go-redis.v9/v2 v2.2.3
 	github.com/DataDog/dd-trace-go/contrib/valkey-io/valkey-go/v2 v2.2.3
 	github.com/DataDog/dd-trace-go/v2 v2.2.3
+	github.com/alicebob/miniredis/v2 v2.35.0
 	github.com/apache/arrow/go/v17 v17.0.0
 	github.com/aws/aws-sdk-go-v2 v1.36.4
 	github.com/aws/aws-sdk-go-v2/config v1.29.14
@@ -126,6 +127,7 @@ require (
 	github.com/stretchr/objx v0.5.2 // indirect
 	github.com/tklauser/go-sysconf v0.3.15 // indirect
 	github.com/tklauser/numcpus v0.10.0 // indirect
+	github.com/yuin/gopher-lua v1.1.1 // indirect
 	github.com/yusufpapurcu/wmi v1.2.4 // indirect
 	github.com/zeebo/xxh3 v1.0.2 // indirect
 	go.opentelemetry.io/auto/sdk v1.1.0 // indirect

go.sum

@@ -53,6 +53,8 @@ github.com/Masterminds/semver/v3 v3.3.1/go.mod h1:4V+yj/TJE1HU9XfppCwVMZq3I84lpr
 github.com/Microsoft/go-winio v0.5.0/go.mod h1:JPGBdM1cNvN/6ISo+n8V5iA4v8pBzdOpzfwIujj1a84=
 github.com/Microsoft/go-winio v0.6.2 h1:F2VQgta7ecxGYO8k3ZZz3RS8fVIXVxONVUPlNERoyfY=
 github.com/Microsoft/go-winio v0.6.2/go.mod h1:yd8OoFMLzJbo9gZq8j5qaps8bJ9aShtEA8Ipt1oGCvU=
+github.com/alicebob/miniredis/v2 v2.35.0 h1:QwLphYqCEAo1eu1TqPRN2jgVMPBweeQcR21jeqDCONI=
+github.com/alicebob/miniredis/v2 v2.35.0/go.mod h1:TcL7YfarKPGDAthEtl5NBeHZfeUQj6OXMm/+iu5cLMM=
 github.com/andybalholm/brotli v1.1.0 h1:eLKJA0d02Lf0mVpIDgYnqXcUn0GqVmEFny3VuID1U3M=
 github.com/andybalholm/brotli v1.1.0/go.mod h1:sms7XGricyQI9K10gOSf56VKKWS4oLer58Q+mhRPtnY=
 github.com/apache/arrow/go/v17 v17.0.0 h1:RRR2bdqKcdbss9Gxy2NS/hK8i4LDMh23L6BbkN5+F54=
@@ -299,6 +301,8 @@ github.com/vmihailenco/tagparser v0.1.2/go.mod h1:OeAg3pn3UbLjkWt+rN9oFYB6u/cQgq
 github.com/yuin/goldmark v1.1.27/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
 github.com/yuin/goldmark v1.2.1/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
 github.com/yuin/goldmark v1.3.5/go.mod h1:mwnBkeHKe2W/ZEtQ+71ViKU8L12m81fl3OWwC1Zlc8k=
+github.com/yuin/gopher-lua v1.1.1 h1:kYKnWBjvbNP4XLT3+bPEwAXJx262OhaHDWDVOPjL46M=
+github.com/yuin/gopher-lua v1.1.1/go.mod h1:GBR0iDaNXjAgGg9zfCvksxSRnQx76gclCIb7kdAd1Pw=
 github.com/yusufpapurcu/wmi v1.2.4 h1:zFUKzehAFReQwLys1b/iSMl+JQGSCSjtVqQn9bBrPo0=
 github.com/yusufpapurcu/wmi v1.2.4/go.mod h1:SBZ9tNy3G9/m5Oi98Zks0QjeHVDvuK0qfxQmPyzfmi0=
 github.com/zeebo/assert v1.3.0 h1:g7C04CbJuIDKNPFHmsk4hwZDO5O+kntRxzaUoNXj+IQ=

go/internal/feast/integration_tests/valkey/feature_repo/example_repo.py

@@ -2,7 +2,9 @@
 
 from datetime import timedelta
 
-from feast import Entity, FeatureView, Field, FileSource, Project, FeatureService
+from feast import Entity, FeatureView, Field, FileSource, Project, FeatureService, SortedFeatureView
+from feast.protos.feast.core.SortedFeatureView_pb2 import SortOrder
+from feast.sort_key import SortKey
 from feast.types import (
     Array,
     Bool,
@@ -89,3 +91,56 @@
     features=[mlpfs_test_all_datatypes_view],
 )
 
+mlpfs_test_all_datatypes_sorted_view: SortedFeatureView = SortedFeatureView(
+    name="all_dtypes_sorted",
+    entities=[index_entity],
+    ttl=timedelta(),
+    source=mlpfs_test_all_datatypes_source,
+    tags=tags,
+    description="Feature View with all supported feast datatypes",
+    owner=owner,
+    online=True,
+    sort_keys=[
+        SortKey(
+            name="event_timestamp",
+            value_type=ValueType.UNIX_TIMESTAMP,
+            default_sort_order=SortOrder.DESC,
+        )
+    ],
+    schema=[
+        Field(name="index_id", dtype=Int64),
+        Field(name="int_val", dtype=Int32),
+        Field(name="long_val", dtype=Int64),
+        Field(name="float_val", dtype=Float32),
+        Field(name="double_val", dtype=Float64),
+        Field(name="byte_val", dtype=Bytes),
+        Field(name="string_val", dtype=String),
+        Field(name="timestamp_val", dtype=UnixTimestamp),
+        Field(name="boolean_val", dtype=Bool),
+        Field(name="array_int_val", dtype=Array(Int32)),
+        Field(name="array_long_val", dtype=Array(Int64)),
+        Field(name="array_float_val", dtype=Array(Float32)),
+        Field(name="array_double_val", dtype=Array(Float64)),
+        Field(name="array_byte_val", dtype=Array(Bytes)),
+        Field(name="array_string_val", dtype=Array(String)),
+        Field(name="array_timestamp_val", dtype=Array(UnixTimestamp)),
+        Field(name="array_boolean_val", dtype=Array(Bool)),
+        Field(name="null_int_val", dtype=Int32),
+        Field(name="null_long_val", dtype=Int64),
+        Field(name="null_float_val", dtype=Float32),
+        Field(name="null_double_val", dtype=Float64),
+        Field(name="null_byte_val", dtype=Bytes),
+        Field(name="null_string_val", dtype=String),
+        Field(name="null_timestamp_val", dtype=UnixTimestamp),
+        Field(name="null_boolean_val", dtype=Bool),
+        Field(name="null_array_int_val", dtype=Array(Int32)),
+        Field(name="null_array_long_val", dtype=Array(Int64)),
+        Field(name="null_array_float_val", dtype=Array(Float32)),
+        Field(name="null_array_double_val", dtype=Array(Float64)),
+        Field(name="null_array_byte_val", dtype=Array(Bytes)),
+        Field(name="null_array_string_val", dtype=Array(String)),
+        Field(name="null_array_timestamp_val", dtype=Array(UnixTimestamp)),
+        Field(name="null_array_boolean_val", dtype=Array(Bool)),
+        Field(name="event_timestamp", dtype=UnixTimestamp),
+    ],
+)