Devices with internal edges which are not 'unidirectionalEdge' cause an error

[jamiegosling]

Describe the Issue
When using topology.get_device eg. device = app.topology.get_device(device_id="device18") if the device has a internal edges which are not of type unidrectionalEdge it causes a Key Error, I think because these edges are not referenced in the corresponding nGraphElements call

To Reproduce
For me the code example is device = app.topology.get_device(device_id="device18"), but the device in question returns elements like this in the JSON

{
    "data": {
        "status": {
            "network": {
                "edgesByDevice": {
                    "_items": [
                        {
                            "_id": "device18",
                            "_vid": "device18",
<snip>
                            "resource_device18.67ea8d1dd6927425ae16fc6e-e14c6cec39f0840f20a004fb2d815cc329142d5dfe80ce44.36ec730c3143a03aae4167de83af1c3dbe5edec3b62e47b1::device18.67ea8d1dd6927425ae16fc6e-e14c6cec39f0840f20a004fb2d815cc329142d5dfe80ce44.80a5e80ffaa50ff7c5a49510ac9f8510fceb6e72587a183e": {
                                "active": true,
                                "descriptor": {
                                    "desc": "",
                                    "label": ""
                                },
                                "fDescriptor": {
                                    "desc": "",
                                    "label": "Resource from SRT to SRT"
                                },
                                "fResourceIds": [
                                    "SRT_SRT"
                                ],
                                "fromId": "device18.67ea8d1dd6927425ae16fc6e-e14c6cec39f0840f20a004fb2d815cc329142d5dfe80ce44.36ec730c3143a03aae4167de83af1c3dbe5edec3b62e47b1",
                                "resourceIds": [],
                                "tags": [],
                                "toId": "device18.67ea8d1dd6927425ae16fc6e-e14c6cec39f0840f20a004fb2d815cc329142d5dfe80ce44.80a5e80ffaa50ff7c5a49510ac9f8510fceb6e72587a183e",
                                "type": "nGraphResourceTransform"
                            }

Expected Behavior
Should be able to get edges

Logs & Error Messages

Traceback (most recent call last):
  File "/mnt/c/wsl/test-ipath-config/test.py", line 41, in <module>
    device = app.topology.get_device(device_id="device18")
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/c/wsl/test-ipath-config/venv/lib/python3.12/site-packages/videoipath_automation_tool/apps/topology/topology_app.py", line 68, in get_device
    return self._topology_api.get_device_from_topology(device_id)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/c/wsl/test-ipath-config/venv/lib/python3.12/site-packages/videoipath_automation_tool/apps/topology/topology_api.py", line 52, in get_device_from_topology
    device_configuration = self._fetch_device_configuration_from_topology(device_id)
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/c/wsl/test-ipath-config/venv/lib/python3.12/site-packages/videoipath_automation_tool/apps/topology/topology_api.py", line 134, in _fetch_device_configuration_from_topology
    vertex_list = self._fetch_all_nGraphElements_by_device_id(device_id)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/c/wsl/test-ipath-config/venv/lib/python3.12/site-packages/videoipath_automation_tool/apps/topology/topology_api.py", line 171, in _fetch_all_nGraphElements_by_device_id
    unidirectional_edges = self._fetch_filtered_nGraph_elements_by_device_id(device_id, "unidirectionalEdge")
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/c/wsl/test-ipath-config/venv/lib/python3.12/site-packages/videoipath_automation_tool/apps/topology/topology_api.py", line 332, in _fetch_filtered_nGraph_elements_by_device_id
    edge["_rev"] = rev_dict[edge["_id"]]
                   ~~~~~~~~^^^^^^^^^^^^^
KeyError: 'resource_device18.67ea8d1dd6927425ae16fc6e-48e07734e3bda1d9b96b2de263dac66670d3cc2f111c4aa3.591c5e135a258870b8ebdd000d1c96b4eb56d4432f6b0b03::device18.67ea8d1dd6927425ae16fc6e-48e07734e3bda1d9b96b2de263dac66670d3cc2f111c4aa3.941ea61088ab8b7d6ca6f497074ca22f034cbfdf085272c8'

Potential fix may be to filter by type = unidirectionalEdge as well as device ID when fetching edgesByDevice in line 312 of tpology_api.py:

            revision_data = self.vip_connector.rest.get(
                "/rest/v2/data/config/network/nGraphElements/* where type = 'unidirectionalEdge' /id,rev,vid"
            )
            rev_dict = {}
            for item in revision_data.data["config"]["network"]["nGraphElements"]["_items"]:
                id = item["_id"]
                rev = item["_rev"]
                rev_dict[id] = rev

            # 2. Fetch edge data
            api_response = self.vip_connector.rest.get(
                f"/rest/v2/data/status/network/edgesByDevice/* where _id='{device_id}' /**"
            )

Desktop (please complete the following information):

• OS: Windows
• Python Version: 3.11
• **Module Version:**0.4.0

Environment Details (if applicable):

• VideoIPath Server Version: [e.g., 2024.1.4]

Additional context
Add any other context about the problem here.

[PaulWinterstein]

Hi @jamiegosling,
thanks for reporting this issue — that kind of feedback is very helpful.

The element type nGraphResourceTransform hasn’t been supported so far.
I’ll update the data model and the relevant API methods accordingly. The fix will be part of the next release.

Best,
Paul

[PaulWinterstein]

Hi @jamiegosling,
could you try if the fix resolves the issue?

pip install --upgrade --force-reinstall videoipath-automation-tool==0.4.0.dev271

[jamiegosling]

Hi @PaulWinterstein, I've just tested that version and it resolves the issue, thanks!