Best-effort breaking change fixes, update changelog

Author: andystaples

CHANGELOG.md

@@ -52,65 +52,71 @@ ADDED
   uses a deterministic instance ID (`export-job-{job_id}`, exposed via
   `orchestrator_instance_id_for(...)`) so callers can correlate a job ID
   with its orchestrator for logging, monitoring, and restart.
-- Added a pluggable `DataConverter` abstraction (`durabletask.serialization`)
-  consumed by both the worker and the client. `TaskHubGrpcWorker`,
-  `TaskHubGrpcClient`, and `AsyncTaskHubGrpcClient` accept a `data_converter`
-  argument; every payload serialization boundary (inputs, outputs, events,
+- Added a pluggable `DataConverter` (`durabletask.serialization`) accepted by
+  `TaskHubGrpcWorker`, `TaskHubGrpcClient`, and `AsyncTaskHubGrpcClient` via a
+  `data_converter` argument. Every payload boundary (inputs, outputs, events,
   custom status, entity state) routes through it. The default
-  `JsonDataConverter` preserves existing behavior, so supplying a custom
-  converter (for example, one backed by pydantic) is purely opt-in. Custom
-  objects opt in via a `to_json()` hook -- invoked as `type(obj).to_json(obj)`
-  so both instance methods and `@staticmethod` hooks work -- and a
-  `from_json(value)` classmethod, matching the `azure-functions-durable`
-  convention.
-- Type-aware deserialization of payloads. `OrchestrationContext.call_activity`,
-  `call_sub_orchestrator`, and `call_entity` accept an optional `return_type`,
-  and `wait_for_external_event` accepts an optional `data_type`. When provided,
-  the result/event payload is coerced to that type: dataclasses are
-  reconstructed from their dict payloads (including nested dataclass, `Optional`,
-  and `list` fields), and types exposing a `from_json()` classmethod are rebuilt
-  via that hook. When omitted, the raw deserialized JSON is returned as before.
-  The `return_type` / `data_type` argument also refines the static type of the
-  returned task (e.g. `call_activity(..., return_type=Foo)` is typed as
-  `CompletableTask[Foo]`).
+  `JsonDataConverter` preserves existing behavior, so a custom converter (for
+  example one backed by pydantic) is opt-in. Custom objects can opt in via a
+  `to_json()` hook and a `from_json(value)` classmethod.
+- `OrchestrationContext.call_activity`, `call_sub_orchestrator`, and
+  `call_entity` accept an optional `return_type`, and `wait_for_external_event`
+  accepts an optional `data_type`. When provided, the result/event payload is
+  reconstructed as that type (dataclasses — including nested dataclass,
+  `Optional`, and `list` fields — and `from_json()`-capable types) and the
+  returned task is typed accordingly (e.g. `call_activity(..., return_type=Foo)`
+  yields `CompletableTask[Foo]`). When omitted, the raw deserialized JSON is
+  returned as before.
 - Inbound payloads are reconstructed from function type annotations. When an
-  orchestrator, activity, or entity operation annotates its input parameter with
-  a dataclass or a `from_json()`-capable type, the incoming payload is
-  automatically coerced to that type. Coercion is best-effort: builtins and
-  unannotated/unknown types are passed through unchanged, and a payload that
-  cannot be coerced to the requested type falls back to the raw deserialized
-  value (logged at debug level) rather than raising. This best-effort policy is
-  owned by the default `JsonDataConverter`; a stricter converter can change it.
-- `call_activity` results are reconstructed from the activity's return
-  annotation. When an activity function reference is passed (not a string name)
-  and its return type is annotated with a dataclass or `from_json()`-capable
-  type, the result is automatically coerced to that type. An explicit
-  `return_type` argument takes precedence over the discovered annotation.
+  orchestrator, activity, or entity operation annotates its input parameter (or
+  an activity its return value) with a dataclass or `from_json()`-capable type,
+  the payload is reconstructed as that type. Builtins and unannotated/unknown
+  types are passed through unchanged. An explicit `return_type` takes precedence
+  over a discovered annotation.
 - Added typed accessors to `client.OrchestrationState`: `get_input()`,
   `get_output()`, and `get_custom_status()` each accept an optional
-  `expected_type` and deserialize the corresponding `serialized_*` payload,
-  reconstructing dataclasses and `from_json()`-capable types. The raw
-  `serialized_input` / `serialized_output` / `serialized_custom_status` string
-  fields are retained.
+  `expected_type` and deserialize the corresponding payload, reconstructing
+  dataclasses and `from_json()`-capable types. The raw `serialized_*` fields are
+  retained.
 - Objects exposing a `to_json()` method are now JSON-serializable when passed as
   activity/orchestrator inputs or outputs.
-- Entity state retrieval (`get_state(intended_type=...)`) now reconstructs
-  dataclasses from their stored dict payloads and supports types exposing a
-  `from_json()` classmethod, in addition to the existing constructor-based
+- Added `EntityMetadata.get_typed_state(intended_type=...)`, which deserializes
+  the entity's persisted state and reconstructs dataclasses and
+  `from_json()`-capable types. The existing `get_state()` is unchanged: with no
+  argument it returns the raw serialized JSON payload, and `get_state(some_type)`
+  applies constructor-based coercion (`some_type(raw)`).
+- Entity runtime state retrieval (`EntityContext.get_state(intended_type=...)` /
+  `DurableEntity.get_state(...)`) now also reconstructs dataclasses and
+  `from_json()`-capable types, in addition to the existing constructor-based
   coercion.
 
 CHANGED
 
-- Custom objects (dataclasses, `SimpleNamespace`) are now serialized as plain
-  JSON without an internal type marker. Decoding without a `return_type` /
-  `data_type` therefore yields a plain `dict` (previously a `SimpleNamespace`
-  for marked payloads). Pass the new type arguments to reconstruct the original
-  type. Payloads produced by older SDK versions (carrying the legacy marker)
-  continue to deserialize, including into a `SimpleNamespace` when no type is
-  supplied.
+- Custom objects (dataclasses, `SimpleNamespace`, namedtuples) are now
+  serialized as plain JSON. Decoding such a payload *without* a type hint now
+  yields a plain `dict` (previously a `SimpleNamespace`; a namedtuple now
+  round-trips as a JSON array). To get the original type back, pass the new
+  `return_type` / `data_type` arguments, annotate the consuming function's
+  parameter or return type, or use the typed client accessors. Payloads produced
+  by older SDK versions still deserialize — including into a `SimpleNamespace`
+  when no type is supplied — so in-flight orchestrations continue to replay
+  across an upgrade.
 - JSON serialization failures now raise a `TypeError` that chains the original
-  error (`__cause__`) and names the offending type, making serialization issues
-  easier to diagnose.
+  error (`__cause__`) and names the offending type.
+
+BREAKING CHANGES (type-level only — no runtime impact for typical users)
+
+These changes do not alter runtime behavior, but because the package ships
+`py.typed`, consumers running strict type checkers (pyright/mypy) — or
+subclassing the public abstract types — may need to update their code:
+
+- `OrchestrationContext.call_activity`, `call_sub_orchestrator`, `call_entity`,
+  and `wait_for_external_event` gained new keyword-only parameters
+  (`return_type` / `data_type`). Subclasses overriding these methods should add
+  the parameter to match the base signature.
+- `client.OrchestrationState` gained a non-public `_data_converter` field
+  (excluded from equality and `repr`). Code constructing `OrchestrationState`
+  positionally should pass it via the new field or rely on its default.
 
 ## v1.5.0
 

durabletask/entities/entity_metadata.py

@@ -85,7 +85,40 @@ def get_state(self, intended_type: None = None) -> Any:
         ...
 
     def get_state(self, intended_type: type[TState] | None = None) -> TState | Any | None:
-        """Get the current state of the entity, optionally converting it to a specified type.
+        """Get the entity's raw persisted state, optionally constructor-coerced.
+
+        The state is held as the raw serialized JSON payload (a ``str``). With no
+        argument the raw payload is returned unchanged; passing ``intended_type``
+        applies the legacy constructor-based coercion (``intended_type(raw)``)
+        and raises ``TypeError`` if that fails.
+
+        This preserves the pre-existing contract. To deserialize the payload and
+        reconstruct dataclasses or ``from_json()``-capable types, use
+        :meth:`get_typed_state` instead.
+        """
+        if intended_type is None or self._state is None:
+            return self._state
+
+        if isinstance(self._state, intended_type):
+            return self._state
+
+        try:
+            return intended_type(self._state)  # type: ignore[call-arg]
+        except Exception as ex:
+            raise TypeError(
+                f"Could not convert state of type '{type(self._state).__name__}' to '{intended_type.__name__}'"
+            ) from ex
+
+    @overload
+    def get_typed_state(self, intended_type: type[TState]) -> TState | None:
+        ...
+
+    @overload
+    def get_typed_state(self, intended_type: None = None) -> Any:
+        ...
+
+    def get_typed_state(self, intended_type: type[TState] | None = None) -> TState | Any | None:
+        """Deserialize the entity's persisted state, optionally reconstructing a type.
 
         The state is stored as its raw serialized JSON payload and deserialized
         here. When ``intended_type`` is provided the payload is reconstructed as

durabletask/extensions/history_export/client.py

@@ -220,7 +220,7 @@ def get_job(self, job_id: str) -> ExportJobDescription | None:
         meta = self._client.get_entity(entity_id, include_state=True)
         if meta is None:
             return None
-        state = meta.get_state()
+        state = meta.get_typed_state()
         if not state:
             return None
         if not isinstance(state, dict):
@@ -260,7 +260,7 @@ def list_jobs(
             # explicit entity-name check.
             if meta.id.entity != ENTITY_NAME.lower():
                 continue
-            raw = meta.get_state()
+            raw = meta.get_typed_state()
             if not raw:
                 logger.warning(
                     "list_jobs: skipping export-job entity %r with no "

durabletask/internal/entity_state_shim.py

@@ -38,7 +38,22 @@ def get_state(self, intended_type: type[TState] | None = None, default: TState |
         if intended_type is None:
             return self._current_state
 
-        return self._data_converter.coerce(self._current_state, intended_type)
+        coerced = self._data_converter.coerce(self._current_state, intended_type)
+
+        # An explicit ``intended_type`` is a request to receive that type. The
+        # default converter is best-effort and would silently return the raw
+        # value on a failed coercion; restore the stricter contract here by
+        # raising when a non-None state could not be coerced to a concrete type.
+        # ``intended_type`` may be a typing generic (e.g. ``list[int]``) at
+        # runtime, which is not a ``type`` instance, so the guard is required.
+        if (self._current_state is not None
+                and isinstance(intended_type, type)  # pyright: ignore[reportUnnecessaryIsInstance]
+                and not isinstance(coerced, intended_type)):
+            raise TypeError(
+                f"Could not convert state of type '{type(self._current_state).__name__}' to '{intended_type.__name__}'"
+            )
+
+        return coerced
 
     def set_state(self, state: Any) -> None:
         self._current_state = state

durabletask/internal/shared.py

@@ -7,6 +7,15 @@
 
 import grpc
 import grpc.aio
+
+# Backwards-compatibility re-exports. The JSON codec moved to
+# ``durabletask.internal.json_codec``; these aliases keep older imports from
+# ``durabletask.internal.shared`` working.
+from durabletask.internal.json_codec import (  # noqa: F401
+    AUTO_SERIALIZED as AUTO_SERIALIZED,
+    from_json as from_json,
+    to_json as to_json,
+)
 from durabletask.grpc_options import GrpcChannelOptions
 
 ClientInterceptor: TypeAlias = (

tests/durabletask/extensions/history_export/test_entity.py

@@ -87,7 +87,7 @@ def _create_payload() -> dict:
 
 
 def _state_dict(metadata) -> dict:
-    state = metadata.get_state()
+    state = metadata.get_typed_state()
     assert isinstance(state, dict)
     return state
 
@@ -105,7 +105,7 @@ def _check() -> Optional[dict]:
         meta = c.get_entity(entity_id, include_state=True)
         if meta is None:
             return None
-        state = meta.get_state()
+        state = meta.get_typed_state()
         if not isinstance(state, dict):
             return None
         return state if predicate(state) else None

tests/durabletask/test_entity_executor.py

@@ -177,8 +177,11 @@ def from_json(cls, data):
         assert isinstance(result, Wrapped)
         assert result.n == 3
 
-    def test_get_state_invalid_coercion_falls_back_to_raw(self):
-        # The default converter is best-effort: a coercion failure returns the
-        # raw value rather than raising.
+    def test_get_state_invalid_coercion_raises(self):
+        # An explicit intended_type that the state cannot be coerced to raises,
+        # restoring the pre-existing strict contract for entity state access.
+        import pytest
+
         state = StateShim("not-an-int")
-        assert state.get_state(int) == "not-an-int"
+        with pytest.raises(TypeError):
+            state.get_state(int)