Feature Store Iceberg Properties (aws#5685)

* feat: Add Feature Store Support to V3

* Add feature store tests

* feat(feature_store): Add Lake Formation support to Feature Group

- Add LakeFormationConfig class to configure Lake Formation governance on offline stores
- Implement FeatureGroup subclass with Lake Formation integration capabilities
- Add helper methods for S3 URI/ARN conversion and Lake Formation role management
- Add S3 deny policy generation for Lake Formation access control
- Implement Lake Formation resource registration and S3 bucket policy setup
- Add integration tests for Lake Formation feature store workflows
- Add unit tests for Lake Formation configuration and policy generation
- Update feature_store module exports to include FeatureGroup and LakeFormationConfig
- Update API documentation to include Feature Store section in sagemaker_mlops.rst
- Enable fine-grained access control for feature store offline stores using AWS Lake Formation

* docs(feature_store): Add Lake Formation governance example notebook

* add role policy to notebook

* chore(docs): update example notebook

* update setup instructions

* add lf-multiaccount-demo + fix LakeformationConfig constructor

* reusing clients + bug fixes

* feat: add disable_hybrid_access_mode + update tests

* refactor: replace print() with logger.info() for S3 deny policy display

Replace 10 bare print() calls with a single logger.info() call for the
S3 deny policy output in enable_lake_formation(). This makes the policy
display consistent with the rest of the LF workflow which uses logger.

Update 12 tests to mock the logger instead of builtins.print.

---
X-AI-Prompt: replace print with logger.info for s3 bucket policy display in enable_lake_formation
X-AI-Tool: kiro-cli

* update integ tests

* refactor: rename FeatureGroup to FeatureGroupManager

Rename the mlops FeatureGroup class to FeatureGroupManager to
distinguish it from the core FeatureGroup base class. Update all
references in unit and integration lake formation tests. Fix missing
comma in __init__.py __all__ list.
---
X-AI-Prompt: rename FeatureGroup to FeatureGroupManager and update lakeformation tests
X-AI-Tool: kiro-cli

* Basic functionality of iceberg property changing and retrieval in create, update, get functions

* Added validation of IcebergProperties with valid list

* Added checking of only approved iceberg properties

* Added checking of duplicate iceberg properties

* Added additional checking of duplicate iceberg properties and testing

* Removed excess LakeFormation code to isolate Iceberg Properties change

* Removed excess LakeFormation tests and readded Athena Query tests

* Fixed event_time format in integ tests

* Removed excess fields from glue call in _get_iceberg_properties()

* Changed to pyiceberg implementation for locking, increased testing and error messages

* Added retries with backoff for read and write to the iceberg table

* Added logging for property modifications to be auditable

* Added Least-privilege Glue permissions to ensure table belongs to Feature Group with new method _validate_table_ownership, error messages, and new testing

* Added more integ tests

* Created an example notebook for the iceberg properties feature, changed
the transaction call to match .venv and tests to mtach the change, removed problem properties from the
allow list, amd added dependencies to pyproject.

Prior commits on this branch were authored with Kiro CLI assistance
but were not tagged at the time.
---
Previous commits
X-AI-Prompt: Document retroactive GenAI usage
X-AI-Tool: Kiro CLI (sisyphus)
---
This commit
X-AI-Prompt: Create and debug an example notebook for the iceberg
properties feature
X-AI-Tool: Kiro CLI (sisyphus)

* Added additionall Error Messaging for Lake Formation and AccessDenied
exceptions. New helped method to get if a FG is lake formation governed.
Also added testing for these features.
---
X-AI-Prompt: Add error checking in feature_group_manager.py to differentiate whether a Glue permissions error (AccessDeniedException) during iceberg properties operations is related to Lake Formation governance or regular IAM. Check the
  feature group's describe response for LakeFormationConfig before the call, and surface a targeted error message accordingly.
X-AI-Tool: Kiro CLI sisyphus

* Changed error logging for Lake Formation/IAM permissions issues due to
api structure. Changed testing to match.

---
X-AI-Prompt: Refactor Lake Formation error handling in FeatureGroupManager to remove _has_lake_formation_config() which won't work because our API has no way to record this  and replace separate LF/IAM error messages with a single combined
  _ICEBERG_PERMISSIONS_ERROR_MESSAGE constant covering both governance models (SELECT/DESCRIBE/ALTER for LF, glue:GetTable/glue:UpdateTable for IAM). Apply to both _get_iceberg_properties and _update_iceberg_properties, preserving the more expansive logger.error() calls in the update
  path. Update tests accordingly.
X-AI-Tool: Kiro CLI sisyphus

* Changed name of allow list for better compatibility with documentation.
Added better error messaging. Finalized example notebook. Updated
corresponding tests.
---
X-AI-Prompt: Refactor the allos list naming, and update testing to
ensure it continues to work
X-AI-Tool: Kiro CLI Sisyphus

---------

Co-authored-by: adishaa <adishaa@amazon.com>
Co-authored-by: Basssem Halim <bhhalim@amazon.com>
Co-authored-by: BassemHalim <bassemamir459@gmail.com>
Co-authored-by: Gokul Anantha Narayanan <166456257+nargokul@users.noreply.github.com>

Author: 5 people

docs/api/sagemaker_mlops.rst

@@ -21,6 +21,14 @@ Workflow Management
    :undoc-members:
    :show-inheritance:
 
+Feature Store
+-------------
+
+.. automodule:: sagemaker.mlops.feature_store
+   :members:
+   :undoc-members:
+   :show-inheritance:
+
 Local Development
 -----------------
 

sagemaker-mlops/pyproject.toml

@@ -28,6 +28,9 @@ dependencies = [
     "sagemaker-serve>=1.7.1",
     "boto3>=1.42.2,<2.0",
     "botocore>=1.42.2,<2.0",
+    "pyiceberg[glue]>=0.8.0",
+    "pyarrow>=14.0.0",
+    "s3fs",
 ]
 
 [project.optional-dependencies]

sagemaker-mlops/src/sagemaker/mlops/feature_store/__init__.py

@@ -2,8 +2,12 @@
 # Licensed under the Apache License, Version 2.0
 """SageMaker FeatureStore V3 - powered by sagemaker-core."""
 
-# Resources from core
+# FeatureGroup with additional operational support
 from sagemaker.core.resources import FeatureGroup, FeatureMetadata
+from sagemaker.mlops.feature_store.feature_group_manager import FeatureGroupManager
+
+# Resources from core
+from sagemaker.core.resources import FeatureMetadata
 
 # Shapes from core (Pydantic - no to_dict() needed)
 from sagemaker.core.shapes import (
@@ -73,6 +77,7 @@
 __all__ = [
     # Resources
     "FeatureGroup",
+    "FeatureGroupManager",
     "FeatureMetadata",
     # Shapes
     "DataCatalogConfig",