load image from s3 and apply transform

[j-sieger]

I want to load nii.gz file from s3 and apply transform on that image.
LoadImaged in compose need file path as input. But when i load image as object. So i feel transfrom is failing as LoadImaged cant accept objects.

Below code works fine when i run on sagemaker. As i don't need to download image from s3

val_transforms = Compose(
        [
            LoadImaged(keys=["image"]),
            EnsureChannelFirstd(keys=["image"]),
            Spacingd(keys=["image"], pixdim=(
                1.5, 1.5, 2.0), mode=("bilinear")),
            Orientationd(keys=["image"], axcodes="RAS"),
            ScaleIntensityRanged(
                keys=["image"], a_min=-57, a_max=164,
                b_min=0.0, b_max=1.0, clip=True,
                ),
            CropForegroundd(keys=["image"], source_key="image"),
            EnsureTyped(keys=["image"]),
        ]
    )
test_data = {"image": filepath}
Transformed_data = val_transforms(test_data)

Below code is same as above with extra lines of code to download object from s3.

val_transforms = Compose(
        [
            LoadImaged(keys=["image"]),
            EnsureChannelFirstd(keys=["image"]),
            Spacingd(keys=["image"], pixdim=(
                1.5, 1.5, 2.0), mode=("bilinear")),
            Orientationd(keys=["image"], axcodes="RAS"),
            ScaleIntensityRanged(
                keys=["image"], a_min=-57, a_max=164,
                b_min=0.0, b_max=1.0, clip=True,
                ),
            CropForegroundd(keys=["image"], source_key="image"),
            EnsureTyped(keys=["image"]),
        ]
    )
object = bucket.Object(key)
response = object.get()
file_stream = response['body']
test_data = {"image": file_stream}
Transformed_data = val_transforms(test_data)

Please suggest me the different ways to load the images and apply transform.
Note: i cant get the path of the image to pass it to LoadImaged or LoadImage

[Nic-Ma]

Hi @j-sieger ,

Thanks for your interest and feedback!
Yes, we don't support reading from the stream so far, we can consider to support it in future.
@wyli Can we use Nibabel or ITK to read remote files with URI directly?

Thanks.

[j-sieger]

@Nic-Ma i have tried Nibabel but after that i try applying transform but getting below error
img = nib.load('/home/ec2-user/SageMaker/efs/datasets/Task09_Spleen/imagesTr/spleen_61.nii.gz')
val_transforms({"image":img.get_fdata()})
Error::

---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
~/anaconda3/envs/pytorch_latest_p37/lib/python3.7/site-packages/monai/transforms/transform.py in apply_transform(transform, data, map_items, unpack_items)
     91             return [_apply_transform(transform, item, unpack_items) for item in data]
---> 92         return _apply_transform(transform, data, unpack_items)
     93     except Exception as e:

~/anaconda3/envs/pytorch_latest_p37/lib/python3.7/site-packages/monai/transforms/transform.py in _apply_transform(transform, parameters, unpack_parameters)
     59 
---> 60     return transform(parameters)
     61 

~/anaconda3/envs/pytorch_latest_p37/lib/python3.7/site-packages/monai/transforms/utility/dictionary.py in __call__(self, data)
    309         for key, meta_key, meta_key_postfix in zip(self.keys, self.meta_keys, self.meta_key_postfix):
--> 310             d[key] = self.adjuster(d[key], d[meta_key or f"{key}_{meta_key_postfix}"])
    311         return d

KeyError: 'image_meta_dict'

The above exception was the direct cause of the following exception:

RuntimeError                              Traceback (most recent call last)
/tmp/ipykernel_11054/3074050110.py in <module>
     16 
     17 #val_transforms(np.expand_dims(np.moveaxis(img.get_fdata(),2,0),axis=0))
---> 18 val_transforms({"image":img.get_fdata()})

~/anaconda3/envs/pytorch_latest_p37/lib/python3.7/site-packages/monai/transforms/compose.py in __call__(self, input_)
    158     def __call__(self, input_):
    159         for _transform in self.transforms:
--> 160             input_ = apply_transform(_transform, input_, self.map_items, self.unpack_items)
    161         return input_
    162 

~/anaconda3/envs/pytorch_latest_p37/lib/python3.7/site-packages/monai/transforms/transform.py in apply_transform(transform, data, map_items, unpack_items)
    114             else:
    115                 _log_stats(data=data)
--> 116         raise RuntimeError(f"applying transform {transform}") from e
    117 
    118 

RuntimeError: applying transform <monai.transforms.utility.dictionary.EnsureChannelFirstd object at 0x7f4e50c5b410>

So what is the expected format of data input to transform when we use data loaded with nibabel

[Nic-Ma]

Hi @j-sieger ,

For EnsureChannelFirstd(keys=["image"]), it depends on some meta data from LoadImage to automatically adjust the channel dim. As you removed LoadImage and used your own loaded data, so it caused an error.
You can change EnsureChannelFirstdto use AddChanneld transform, then I think it should work.

Thanks.

[j-sieger]

Hi @Nic-Ma ,
It worked after replacing EnsureChannelFirstd to AddChanneld. Thanks for the suggestion

But the real problem is with loading s3 object and applying transformation. I will check the ways to do the same. You can also suggest me if there is any way. With out this i wont able able to deploy my MONAI trained model on AWS.

[wyli]

duplicate of Project-MONAI/MONAI#1656?

[j-sieger]

@Nic-Ma , @wyli
**I have another issue during prediction this time after successfully applying transform on the file loaded with **
My code:

val_transforms = Compose(
    [
        AsChannelFirstd(keys=["image"]),
        AddChanneld(keys=["image"]),
        Spacingd(keys=["image"], pixdim=(
            1.5, 1.5, 2.0), mode=("bilinear")),
        Orientationd(keys=["image"], axcodes="RAS"),
        ScaleIntensityRanged(
            keys=["image"], a_min=-57, a_max=164,
            b_min=0.0, b_max=1.0, clip=True,
        ),
        CropForegroundd(keys=["image"], source_key="image"),
        EnsureTyped(keys=["image"]),
    ]
)
val_data=val_transforms({"image":img.get_fdata()})

roi_size = (160, 160, 160)
sw_batch_size = 4
val_outputs = sliding_window_inference(
            val_data["image"].to(device), roi_size, sw_batch_size, model
        )

ERROR:

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
/tmp/ipykernel_15519/3487396053.py in <module>
      2 sw_batch_size = 4
      3 val_outputs = sliding_window_inference(
----> 4             val_data["image"].to(device), roi_size, sw_batch_size, model
      5         )

~/anaconda3/envs/pytorch_latest_p37/lib/python3.7/site-packages/monai/inferers/utils.py in sliding_window_inference(inputs, roi_size, sw_batch_size, predictor, overlap, mode, sigma_scale, padding_mode, cval, sw_device, device, *args, **kwargs)
     96         sw_device = inputs.device
     97 
---> 98     roi_size = fall_back_tuple(roi_size, image_size_)
     99     # in case that image size is smaller than roi size
    100     image_size = tuple(max(image_size_[i], roi_size[i]) for i in range(num_spatial_dims))

~/anaconda3/envs/pytorch_latest_p37/lib/python3.7/site-packages/monai/utils/misc.py in fall_back_tuple(user_provided, default, func)
    177     """
    178     ndim = len(default)
--> 179     user = ensure_tuple_rep(user_provided, ndim)
    180     return tuple(  # use the default values if user provided is not valid
    181         user_c if func(user_c) else default_c for default_c, user_c in zip(default, user)

~/anaconda3/envs/pytorch_latest_p37/lib/python3.7/site-packages/monai/utils/misc.py in ensure_tuple_rep(tup, dim)
    134         return tuple(tup)
    135 
--> 136     raise ValueError(f"Sequence must have length {dim}, got {len(tup)}.")
    137 
    138 

ValueError: Sequence must have length 2, got 3.

[wyli]

could you ensure that{"image":img.get_fdata()} is 4D and the 4th dimension represents channel

[j-sieger]

print(img.get_fdata().shape) --> (512, 512, 31) >> looks like 3rd dimension is the channels

print(val_data['image'].shape) --> torch.Size([1, 21, 342, 208]) >> here 2nd dimension represents the channel as iam using AsChannelFirstd ==> In this case i am facing above error like sequence must have lenght 2.

---

@wyli
I have also tried to make sure 4th dimension as channel by removing AsChannelFirstd. As per ur above suggestion. But still i am facing same error like sequence must have length 2.

[wyli]

I see, assuming this is about 3D image processing... given that you have print(img.get_fdata().shape) --> (512, 512, 31)

the script would be

val_transforms = Compose(
    [
        # AsChannelFirstd(keys=["image"]),
        AddChanneld(keys=["image"]),
        Spacingd(keys=["image"], pixdim=(
            1.5, 1.5, 2.0), mode=("bilinear")),
        Orientationd(keys=["image"], axcodes="RAS"),
        ScaleIntensityRanged(
            keys=["image"], a_min=-57, a_max=164,
            b_min=0.0, b_max=1.0, clip=True,
        ),
        CropForegroundd(keys=["image"], source_key="image"),
        EnsureTyped(keys=["image"]),
    ]
)
val_data=val_transforms({"image":img.get_fdata()})

roi_size = (160, 160, 160)
sw_batch_size = 4
val_outputs = sliding_window_inference(
            val_data["image"].to(device), roi_size, sw_batch_size, model
        )

the AddChanneld transform with make val_data['image'] a 4D array. and the rest of the pipeline should work with the 4D array.