feat: add viaGateway mode for shared per-gateway service (#777)

* refactor: move roles + isGateway from Service to Device/DeviceConfig

Preparation for the viaGateway shared-service mode (next commit): when one
ViCareServiceViaGateway instance serves all devices on a gateway, the service
cannot answer per-device role questions because its roles list would be
whichever device was processed first. Move role-based decisions onto the
device-identity side of the architecture, where they belong.

- DeviceConfig.hasRoles(roles) + DeviceConfig.isGateway() use self.roles
  instead of self.service.hasRoles / self.service._isGateway.
- Device base class now takes optional roles list; all DeviceConfig.as*
  factories pass it through. Device.isLegacyDevice / isE3Device use
  self.roles.
- HeatingDevice.get_heat_curve_formular uses self.roles for the
  heatpump/E3 check.
- Service-side hasRoles / _isGateway are kept untouched for backwards
  compatibility (still used internally by the per-device service for URL
  shape selection).

No public API removals. Test mocks updated to pass roles via the
PyViCareDeviceConfig constructor instead of patching service.hasRoles.

* feat: add viaGateway mode for bulk per-gateway feature fetching

Adds an opt-in service mode where ONE ViCareCachedServiceViaGateway
instance serves all devices on a gateway from a single bulk API call,
instead of N per-device calls per refresh.

Endpoint used (already verified byte-identical for overlapping features):
  GET /features/installations/{id}/gateways/{serial}/features?includeDevicesFeatures=true

Public API:
  vicare = PyViCare()
  vicare.loadViaGateway(True)   # default False; must precede initWith*
  vicare.initWithCredentials(...)

Per-device entry points (device.getProperty, DeviceConfig.as*) are
unchanged. The shared service is wired in PyViCare.__extract_all_devices:
one service per gateway in viaGateway mode, one per device otherwise.

Architecture:
- ViCareServiceViaGateway: transport, hits the bulk URL for fetch_all_features
  and getProperty; setProperty still uses the per-device URL (writes target
  one specific feature on one specific device).
- ViCareCachedServiceViaGateway: caches both getProperty and
  fetch_all_features against the same cached payload. The fetch_all_features
  cache is the key optimization for HA Core's DataUpdateCoordinator pattern,
  where each per-device coordinator calls service.clear_cache +
  service.fetch_all_features per refresh -- without it, N coordinators on a
  shared service would still trigger N bulk fetches per cycle.
- Defensive error handling (PACKAGE_NOT_PAID_FOR, DeviceCommunicationError,
  InternalServerError) mirrors ViCareCachedService: serve stale cache on
  transient failure, raise on first-fetch failure.

Carries the design intent of CFenner's stalled #626 forward. The bulk
response test fixtures (heatbox1/heatbox2/tcu1) are lifted from that draft.
Implementation is fresh on current master because #774 made the previous
sketch incompatible (accessor lives on Device now, service is stateless),
and the original draft did not actually share the service instance across
devices.

Tests: 20 new ones across three files covering URL building, per-device
filtering of the bulk response, shared-cache behavior across devices,
TTL respect, setProperty cache invalidation, stale-cache fallback on
transient errors, and end-to-end PyViCare wiring with mocked OAuth.

Refs: #626 (CFenner draft)
Refs: home-assistant/core#173776 (raised the question)

* test(via-gateway): move ViCareCachedService import to module top

CI pylint failed with C0415 (import-outside-toplevel).

* refactor: dedupe cached-service caching and gateway role check

- extract ViCareCachedServiceBase (TTL, locking, stale-cache fallback,
  write invalidation) shared by ViCareCachedService and
  ViCareCachedServiceViaGateway via _fetch_uncached/_extract_entities hooks
- replace ViCareService._isGateway with module-level GATEWAY_ROLES /
  is_gateway_role, also used by PyViCareDeviceConfig.isGateway
- move PyViCare.viaGateway to a class attribute
- annotate filter_features_for_device

Addresses review feedback on #777.

* fix: make cached base a ViCareService so setProperty resolves cleanly

Avoids the mixin super() lint/type noise; the concrete gateway __init__
keeps its explicit parent init (pylint super-init-not-called silenced).

Author: lackas

PyViCare/PyViCare.py

@@ -4,9 +4,11 @@
 from PyViCare.PyViCareAbstractOAuthManager import AbstractViCareOAuthManager
 from PyViCare.PyViCareBrowserOAuthManager import ViCareBrowserOAuthManager
 from PyViCare.PyViCareCachedService import ViCareCachedService
+from PyViCare.PyViCareCachedServiceViaGateway import ViCareCachedServiceViaGateway
 from PyViCare.PyViCareDeviceConfig import PyViCareDeviceConfig
 from PyViCare.PyViCareOAuthManager import ViCareOAuthManager
 from PyViCare.PyViCareService import ViCareDeviceAccessor, ViCareService
+from PyViCare.PyViCareServiceViaGateway import ViCareServiceViaGateway
 from PyViCare.PyViCareUtils import PyViCareInvalidDataError
 
 logger = logging.getLogger(__name__)
@@ -15,12 +17,25 @@
 
 class PyViCare:
     """"Viessmann ViCare API Python tools"""
+    viaGateway = False
+
     def __init__(self) -> None:
         self.cacheDuration = 60
 
     def setCacheDuration(self, cache_duration):
         self.cacheDuration = int(cache_duration)
 
+    def loadViaGateway(self, via_gateway: bool = True) -> None:
+        """Opt in to per-gateway bulk fetching.
+
+        When enabled, ONE service instance per gateway serves all devices on
+        that gateway via the bulk endpoint
+        `/features/installations/{id}/gateways/{serial}/features?includeDevicesFeatures=true`,
+        instead of one HTTP call per device per refresh. Must be called
+        before initWith* (services are wired during __loadInstallations).
+        """
+        self.viaGateway = via_gateway
+
     def initWithCredentials(self, username: str, password: str, client_id: str, token_file: str):
         self.initWithExternalOAuth(ViCareOAuthManager(
             username, password, client_id, token_file))
@@ -37,6 +52,11 @@ def __buildService(self, roles):
             return ViCareCachedService(self.oauth_manager, roles, self.cacheDuration)
         return ViCareService(self.oauth_manager, roles)
 
+    def __buildGatewayService(self):
+        if self.cacheDuration > 0:
+            return ViCareCachedServiceViaGateway(self.oauth_manager, self.cacheDuration)
+        return ViCareServiceViaGateway(self.oauth_manager)
+
     def __loadInstallations(self):
         installations = self.oauth_manager.get(
             "/equipment/installations?includeGateways=true")
@@ -58,10 +78,11 @@ def __loadInstallations(self):
     def __extract_all_devices(self):
         for installation in self.installations:
             for gateway in installation.gateways:
+                shared_service = self.__buildGatewayService() if self.viaGateway else None
                 for device in gateway.devices:
                     accessor = ViCareDeviceAccessor(
                         installation.id, gateway.serial, device.id)
-                    service = self.__buildService(device.roles)
+                    service = shared_service if shared_service is not None else self.__buildService(device.roles)
 
                     logger.info("Device found: %s (type=%s)", device.modelId, device.deviceType)
 

PyViCare/PyViCareCachedService.py

@@ -1,73 +1,23 @@
 import logging
-import threading
-from datetime import datetime
-from typing import Any, List, Optional
+from typing import Any, List
 
 from PyViCare.PyViCareAbstractOAuthManager import AbstractViCareOAuthManager
-from PyViCare.PyViCareService import (ViCareDeviceAccessor, ViCareService,
-                                      readFeature)
-from PyViCare.PyViCareUtils import (PyViCareDeviceCommunicationError,
-                                    PyViCareInvalidDataError,
-                                    PyViCareInternalServerError,
-                                    PyViCareNotPaidForError,
-                                    PyViCareNotSupportedFeatureError,
-                                    ViCareTimer)
+from PyViCare.PyViCareCachedServiceBase import ViCareCachedServiceBase
+from PyViCare.PyViCareService import ViCareDeviceAccessor, ViCareService
 
 logger = logging.getLogger(__name__)
 logger.addHandler(logging.NullHandler())
 
 
-class ViCareCachedService(ViCareService):
+class ViCareCachedService(ViCareCachedServiceBase, ViCareService):
 
     def __init__(self, oauth_manager: AbstractViCareOAuthManager, roles: List[str], cacheDuration: int) -> None:
         ViCareService.__init__(self, oauth_manager, roles)
-        self.__cacheDuration = cacheDuration
-        self.__cache: Optional[dict] = None
-        self.__cacheTime: Optional[datetime] = None
-        self.__lock = threading.Lock()
+        self._init_cache(cacheDuration)
 
-    def getProperty(self, accessor: ViCareDeviceAccessor, property_name: str) -> Any:
-        data = self.__get_or_update_cache(accessor)
-        entities = data["data"]
-        return readFeature(entities, property_name)
+    def _fetch_uncached(self, accessor: ViCareDeviceAccessor) -> Any:
+        return ViCareService.fetch_all_features(self, accessor)
 
-    def setProperty(self, accessor: ViCareDeviceAccessor, property_name: str, action: str, data: Any) -> Any:
-        response = super().setProperty(accessor, property_name, action, data)
-        self.clear_cache()
-        return response
-
-    def __get_or_update_cache(self, accessor: ViCareDeviceAccessor):
-        with self.__lock:
-            if self.is_cache_invalid():
-                # we always set the cache time before we fetch the data
-                # to avoid consuming all the api calls if the api is down
-                # see https://github.com/home-assistant/core/issues/67052
-                # we simply return the old cache in this case
-                self.__cacheTime = ViCareTimer().now()
-
-                try:
-                    data = self.fetch_all_features(accessor)
-                except PyViCareNotPaidForError as e:
-                    logger.error("Viessmann API denied access (PACKAGE_NOT_PAID_FOR). Features unavailable: %s", e)
-                    if self.__cache is not None:
-                        return self.__cache
-                    raise PyViCareNotSupportedFeatureError("PACKAGE_NOT_PAID_FOR")
-                except (PyViCareDeviceCommunicationError, PyViCareInternalServerError) as e:
-                    if self.__cache is not None:
-                        logger.warning("API error, returning stale cache: %s", e)
-                        return self.__cache
-                    raise
-
-                if "data" not in data:
-                    logger.error("Missing 'data' property when fetching data.")
-                    raise PyViCareInvalidDataError(data)
-                self.__cache = data
-            return self.__cache
-
-    def is_cache_invalid(self) -> bool:
-        return self.__cache is None or self.__cacheTime is None or (ViCareTimer().now() - self.__cacheTime).seconds > self.__cacheDuration
-
-    def clear_cache(self):
-        with self.__lock:
-            self.__cache = None
-            self.__cacheTime = None
+    def _extract_entities(self, data: dict, accessor: ViCareDeviceAccessor) -> list[dict[str, Any]]:
+        entities: list[dict[str, Any]] = data["data"]
+        return entities

PyViCare/PyViCareCachedServiceBase.py

@@ -0,0 +1,82 @@
+import logging
+import threading
+from datetime import datetime
+from typing import Any, Optional
+
+from PyViCare.PyViCareService import (ViCareDeviceAccessor, ViCareService,
+                                      readFeature)
+from PyViCare.PyViCareUtils import (PyViCareDeviceCommunicationError,
+                                    PyViCareInternalServerError,
+                                    PyViCareInvalidDataError,
+                                    PyViCareNotPaidForError,
+                                    PyViCareNotSupportedFeatureError,
+                                    ViCareTimer)
+
+logger = logging.getLogger(__name__)
+logger.addHandler(logging.NullHandler())
+
+
+class ViCareCachedServiceBase(ViCareService):
+    """Time-based caching shared by the per-device and per-gateway services.
+
+    Subclasses provide `_fetch_uncached` (the HTTP fetch) and `_extract_entities`
+    (pick this device's features out of the cached payload).
+    """
+
+    def _init_cache(self, cacheDuration: int) -> None:
+        self._cacheDuration = cacheDuration
+        self._cache: Optional[dict] = None
+        self._cacheTime: Optional[datetime] = None
+        self._cacheLock = threading.Lock()
+
+    def getProperty(self, accessor: ViCareDeviceAccessor, property_name: str) -> Any:
+        data = self._get_or_update_cache(accessor)
+        entities = self._extract_entities(data, accessor)
+        return readFeature(entities, property_name)
+
+    def setProperty(self, accessor: ViCareDeviceAccessor, property_name: str, action: str, data: Any) -> Any:
+        response = super().setProperty(accessor, property_name, action, data)
+        self.clear_cache()
+        return response
+
+    def _get_or_update_cache(self, accessor: ViCareDeviceAccessor):
+        with self._cacheLock:
+            if self.is_cache_invalid():
+                # we always set the cache time before we fetch the data
+                # to avoid consuming all the api calls if the api is down
+                # see https://github.com/home-assistant/core/issues/67052
+                # we simply return the old cache in this case
+                self._cacheTime = ViCareTimer().now()
+
+                try:
+                    data = self._fetch_uncached(accessor)
+                except PyViCareNotPaidForError as e:
+                    logger.error("Viessmann API denied access (PACKAGE_NOT_PAID_FOR). Features unavailable: %s", e)
+                    if self._cache is not None:
+                        return self._cache
+                    raise PyViCareNotSupportedFeatureError("PACKAGE_NOT_PAID_FOR")
+                except (PyViCareDeviceCommunicationError, PyViCareInternalServerError) as e:
+                    if self._cache is not None:
+                        logger.warning("API error, returning stale cache: %s", e)
+                        return self._cache
+                    raise
+
+                if "data" not in data:
+                    logger.error("Missing 'data' property when fetching data.")
+                    raise PyViCareInvalidDataError(data)
+                self._cache = data
+            return self._cache
+
+    def is_cache_invalid(self) -> bool:
+        return self._cache is None or self._cacheTime is None or (ViCareTimer().now() - self._cacheTime).seconds > self._cacheDuration
+
+    def clear_cache(self):
+        with self._cacheLock:
+            self._cache = None
+            self._cacheTime = None
+
+    def _fetch_uncached(self, accessor: ViCareDeviceAccessor) -> Any:
+        raise NotImplementedError
+
+    def _extract_entities(self, data: dict, accessor: ViCareDeviceAccessor) -> list[dict[str, Any]]:
+        raise NotImplementedError

PyViCare/PyViCareCachedServiceViaGateway.py

@@ -0,0 +1,34 @@
+import logging
+from typing import Any
+
+from PyViCare.PyViCareAbstractOAuthManager import AbstractViCareOAuthManager
+from PyViCare.PyViCareCachedServiceBase import ViCareCachedServiceBase
+from PyViCare.PyViCareService import ViCareDeviceAccessor
+from PyViCare.PyViCareServiceViaGateway import (
+    ViCareServiceViaGateway, filter_features_for_device)
+
+logger = logging.getLogger(__name__)
+logger.addHandler(logging.NullHandler())
+
+
+class ViCareCachedServiceViaGateway(ViCareCachedServiceBase, ViCareServiceViaGateway):
+    """Cached variant of the gateway bulk-fetch service.
+
+    One instance is shared across all devices on the gateway; every device's
+    getProperty reads from the same cached bulk payload. setProperty clears the
+    whole cache.
+    """
+
+    def __init__(self, oauth_manager: AbstractViCareOAuthManager, cacheDuration: int) -> None:  # pylint: disable=super-init-not-called
+        ViCareServiceViaGateway.__init__(self, oauth_manager)
+        self._init_cache(cacheDuration)
+
+    def _fetch_uncached(self, accessor: ViCareDeviceAccessor) -> Any:
+        return ViCareServiceViaGateway.fetch_all_features(self, accessor)
+
+    def _extract_entities(self, data: dict, accessor: ViCareDeviceAccessor) -> list[dict[str, Any]]:
+        return filter_features_for_device(data["data"], accessor.device_id)
+
+    def fetch_all_features(self, accessor: ViCareDeviceAccessor) -> Any:
+        # cached too, so per-device coordinators share one bulk fetch
+        return self._get_or_update_cache(accessor)

PyViCare/PyViCareDevice.py

@@ -1,6 +1,6 @@
 from typing import Any
 
-from PyViCare.PyViCareService import ViCareDeviceAccessor, ViCareService
+from PyViCare.PyViCareService import ViCareDeviceAccessor, ViCareService, hasRoles
 from PyViCare.PyViCareUtils import PyViCareNotSupportedFeatureError, handleAPICommandErrors, handleNotSupported
 
 
@@ -11,9 +11,10 @@ class Device:
     Note that currently, a new token is generated for each run.
     """
 
-    def __init__(self, accessor: ViCareDeviceAccessor, service: ViCareService) -> None:
+    def __init__(self, accessor: ViCareDeviceAccessor, service: ViCareService, roles: list[str] | None = None) -> None:
         self.accessor = accessor
         self.service = service
+        self.roles = roles if roles is not None else []
 
     def getProperty(self, property_name: str) -> Any:
         return self.service.getProperty(self.accessor, property_name)
@@ -30,10 +31,10 @@ def getDeviceErrors(self) -> list[Any]:
         return list[Any](self.getProperty("device.messages.errors.raw")["properties"]["entries"]["value"])
 
     def isLegacyDevice(self) -> bool:
-        return self.service.hasRoles(["type:legacy"])
+        return hasRoles(["type:legacy"], self.roles)
 
     def isE3Device(self) -> bool:
-        return self.service.hasRoles(["type:E3"])
+        return hasRoles(["type:E3"], self.roles)
 
     def isDomesticHotWaterDevice(self):
         return self._isTypeDevice("heating.dhw")

PyViCare/PyViCareDeviceConfig.py

@@ -16,7 +16,8 @@
 from PyViCare.PyViCareRepeater import Repeater
 from PyViCare.PyViCareElectricalEnergySystem import ElectricalEnergySystem
 from PyViCare.PyViCareGateway import Gateway
-from PyViCare.PyViCareService import ViCareDeviceAccessor, ViCareService
+from PyViCare.PyViCareService import (ViCareDeviceAccessor, ViCareService,
+                                      hasRoles, is_gateway_role)
 from PyViCare.PyViCareUtils import PyViCareNotPaidForError
 from PyViCare.PyViCareVentilationDevice import VentilationDevice
 
@@ -36,52 +37,52 @@ def __init__(self, accessor: ViCareDeviceAccessor, service: ViCareService, devic
         self.roles = roles if roles is not None else []
 
     def asGeneric(self):
-        return HeatingDevice(self.accessor, self.service)
+        return HeatingDevice(self.accessor, self.service, self.roles)
 
     def asGazBoiler(self):
-        return GazBoiler(self.accessor, self.service)
+        return GazBoiler(self.accessor, self.service, self.roles)
 
     def asFuelCell(self):
-        return FuelCell(self.accessor, self.service)
+        return FuelCell(self.accessor, self.service, self.roles)
 
     def asHeatPump(self):
-        return HeatPump(self.accessor, self.service)
+        return HeatPump(self.accessor, self.service, self.roles)
 
     def asOilBoiler(self):
-        return OilBoiler(self.accessor, self.service)
+        return OilBoiler(self.accessor, self.service, self.roles)
 
     def asPelletsBoiler(self):
-        return PelletsBoiler(self.accessor, self.service)
+        return PelletsBoiler(self.accessor, self.service, self.roles)
 
     def asHybridDevice(self):
-        return Hybrid(self.accessor, self.service)
+        return Hybrid(self.accessor, self.service, self.roles)
 
     def asRadiatorActuator(self):
-        return RadiatorActuator(self.accessor, self.service)
+        return RadiatorActuator(self.accessor, self.service, self.roles)
 
     def asFloorHeating(self):
-        return FloorHeating(self.accessor, self.service)
+        return FloorHeating(self.accessor, self.service, self.roles)
 
     def asFloorHeatingChannel(self):
-        return FloorHeatingChannel(self.accessor, self.service)
+        return FloorHeatingChannel(self.accessor, self.service, self.roles)
 
     def asRoomSensor(self):
-        return RoomSensor(self.accessor, self.service)
+        return RoomSensor(self.accessor, self.service, self.roles)
 
     def asRoomControl(self):
-        return RoomControl(self.accessor, self.service)
+        return RoomControl(self.accessor, self.service, self.roles)
 
     def asRepeater(self):
-        return Repeater(self.accessor, self.service)
+        return Repeater(self.accessor, self.service, self.roles)
 
     def asElectricalEnergySystem(self):
-        return ElectricalEnergySystem(self.accessor, self.service)
+        return ElectricalEnergySystem(self.accessor, self.service, self.roles)
 
     def asGateway(self):
-        return Gateway(self.accessor, self.service)
+        return Gateway(self.accessor, self.service, self.roles)
 
     def asVentilation(self):
-        return VentilationDevice(self.accessor, self.service)
+        return VentilationDevice(self.accessor, self.service, self.roles)
 
     def getConfig(self):
         return self.accessor
@@ -101,8 +102,11 @@ def getDeviceType(self):
     def getRoles(self):
         return self.roles
 
+    def hasRoles(self, requested_roles):
+        return hasRoles(requested_roles, self.roles)
+
     def isGateway(self):
-        return self.service._isGateway()  # pylint: disable=protected-access
+        return is_gateway_role(self.roles)
 
     # see: https://vitodata300.viessmann.com/vd300/ApplicationHelp/VD300/1031_de_DE/Ger%C3%A4teliste.html
     def asAutoDetectDevice(self):
@@ -131,7 +135,7 @@ def asAutoDetectDevice(self):
         ]
 
         for (creator_method, type_name, roles) in device_types:
-            if re.search(type_name, self.device_model) or self.service.hasRoles(roles):
+            if re.search(type_name, self.device_model) or self.hasRoles(roles):
                 logger.info("detected %s %s", self.device_model, creator_method.__name__)
                 device = creator_method()
                 if isinstance(device, (GazBoiler, HeatPump)) and not isinstance(device, Hybrid):