FPV camera can not send flying drone images back (only the nonFLY drone camera work)

[hbzhang]

Two drones are in racing. I setup camera images like the following. But only the non fly drone send back FPV images. i want to visualize the flying drone real time tracjetory

I use the following callback function:

def image_callback(self):
    # get uncompressed fpv cam image
    request = [asim.ImageRequest("fpv_cam", asim.ImageType.Scene, False, False)]
    response = self.airsim_client_images.simGetImages(request)
    img_rgb_1d = np.fromstring(response[0].image_data_uint8, dtype=np.uint8)
    img_rgb = img_rgb_1d.reshape(response[0].height, response[0].width, 3)
    cv2.imshow("img_rgb", img_rgb)
    cv2.waitKey(1)

[madratman]

You'll have to add a camera to drone_2 in setting.json:

{
  "ClockSpeed": 1,
  "SeeDocsAt": "https://github.com/Microsoft/AirSim/blob/master/docs/settings.md",
  "SettingsVersion": 1.2,
  "SimMode": "Multirotor",
  "Vehicles": {
    "drone_1": {
      "Cameras": {
        "fpv_cam": {
          "CaptureSettings": [
            {
              "FOV_Degrees": 90,
              "Height": 240,
              "ImageType": 0,
              "Width": 320
            }
          ],
          "Pitch": 0.0,
          "Roll": 0.0,
          "X": 0.25,
          "Y": 0.0,
          "Yaw": 0.0,
          "Z": 0.0
        }
      },
      "Pitch": 0.0,
      "Roll": 0.0,
      "VehicleType": "SimpleFlight",
      "X": 0.0,
      "Y": 0.0,
      "Yaw": 0.0,
      "Z": 0.0
    },
    "drone_2": {
      "Cameras": {
        "fpv_cam_2": {
          "CaptureSettings": [
            {
            "FOV_Degrees": 90,
            "Height": 240,
            "ImageType": 0,
            "Width": 320
            }
          ],
          "Pitch": 0.0,
          "Roll": 0.0,
          "X": 0.25,
          "Y": 0.0,
          "Yaw": 0.0,
          "Z": 0.0
        }
      },
      "Pitch": 0.0,
      "Roll": 0.0,
      "VehicleType": "SimpleFlight",
      "X": 0.0,
      "Y": 0.0,
      "Yaw": 0.0,
      "Z": 0
    }
  }
}

Note that airsim doesn't allow for same camera names, hence, it's called fpv_cam_2.

then you add it to the request list:
request = [airsim.ImageRequest("fpv_cam", asim.ImageType.Scene, False, False), airsim.ImageRequest("fpv_cam_2", asim.ImageType.Scene, False, False)]

then you can get the image from fpv_cam_2 from second object in response list (response[1])

[hbzhang]

Thanks for the suggestion.

I changed the setting.json to this:
{
"ClockSpeed": 1,
"SeeDocsAt": "https://github.com/Microsoft/AirSim/blob/master/docs/settings.md",
"SettingsVersion": 1.2,
"SimMode": "Multirotor",
"Vehicles": {
"drone_1": {
"Cameras": {
"fpv_cam": {
"CaptureSettings": [
{
"FOV_Degrees": 90,
"Height": 240,
"ImageType": 0,
"Width": 320
}
],
"Pitch": 0.0,
"Roll": 0.0,
"X": 0.25,
"Y": 0.0,
"Yaw": 0.0,
"Z": 0.0
}
},
"Pitch": 0.0,
"Roll": 0.0,
"VehicleType": "SimpleFlight",
"X": 0.0,
"Y": 0.0,
"Yaw": 0.0,
"Z": 0.0
},
"drone_2": {
"Cameras": {
"fpv_cam_2": {
"CaptureSettings": [
{
"FOV_Degrees": 90,
"Height": 240,
"ImageType": 0,
"Width": 320
}
],
"Pitch": 0.0,
"Roll": 0.0,
"X": 0.25,
"Y": 0.0,
"Yaw": 0.0,
"Z": 0.0
}
},
"Pitch": 0.0,
"Roll": 0.0,
"VehicleType": "SimpleFlight",
"X": 0.0,
"Y": 0.0,
"Yaw": 0.0,
"Z": 0.0
}
}
}

and image call back like this:

def image_callback(self):
# get uncompressed fpv cam image
request = [asim.ImageRequest("fpv_cam_2", asim.ImageType.Scene, False, False)]
response = self.airsim_client_images.simGetImages(request)
img_rgb_1d = np.fromstring(response[0].image_data_uint8, dtype=np.uint8)
print(f"image heigh is {response[0].height}")
img_rgb = img_rgb_1d.reshape(response[0].height, response[0].width, 3)
cv2.imshow("img_rgb", img_rgb)
cv2.waitKey(1)

But looks like the response = self.airsim_client_images.simGetImages(request) hangs and the CV2 never able to render the response as shown below: