Skip to content

README.md

Latest commit

 

History

History
142 lines (99 loc) · 4.33 KB

README.md

File metadata and controls

142 lines (99 loc) · 4.33 KB

Low-Data Supervised Adaptation Outperforms Prompting for Cloud Segmentation Under Domain Shift

Harshith Kethavath, Weiming Hu

Lab for Geoinformatics and AI Modeling (GAIM), Department of Geography, The University of Georgia

📄 Paper  |  🤗 Models

Overview

We evaluate CLIPSeg on the CloudSEN12+ dataset for four-class cloud segmentation (clear, thick cloud, thin cloud, cloud shadow). We systematically evaluate 60 prompt variants and compare against LoRA and full fine-tuning under varying amounts of labeled data.

Key findings:

  • Prompt engineering fails comprehensively under domain shift; CLIPSeg's frozen encoder is largely insensitive to prompt variation for satellite imagery.
  • Full fine-tuning (FFT) and Low Rank Adaptation (LoRA) with 0.1% labeled data (~8 images) outperforms all prompting approaches.
  • 5–10% labeled data recovers ~85% of full fine-tuning performance.
  • FFT consistently outperforms LoRA, with the gap most pronounced for spectrally ambiguous classes (thin cloud, cloud shadow).

Installation

Note: Install PyTorch for your CUDA version first from pytorch.org, then install this package.

pip install cloudprompts

Or from source:

git clone https://github.com/uga-gaim/2026_CVPRW_CloudPrompts.git
cd 2026_CVPRW_CloudPrompts
pip install -e .

Dataset

Download and preprocess CloudSEN12+:

# Step 1: Download (defaults to ./cloudsen12plus_p509)
bash data/cloudsen12plus/download_cloudsen12plus_p509.sh

# Step 2: Export to NPZ format
python data/cloudsen12plus/export_p509_high_npz.py \
  --root ./cloudsen12plus_p509 \
  --out /path/to/data

The expected directory structure after preprocessing:

/path/to/data/
├── train/
│   ├── images/   # *.npz, key: 'image' (3, H, W) float32 in [0, 1]
│   └── masks/    # *.npz, key: 'mask'  (H, W) int64 class IDs
├── val/
│   ├── images/
│   └── masks/
└── test/
    ├── images/
    └── masks/

Class IDs: 0 = clear, 1 = thick cloud, 2 = thin cloud, 3 = cloud shadow.

Usage

Inference (Zero-Shot)

cloudprompts-infer \
  --data_root /path/to/data/test/images \
  --out_root /path/to/output \
  --checkpoint_type base

Inference (Fine-Tuned Models)

Load a fine-tuned model from HuggingFace:

# Full fine-tuned
huggingface-cli download uga-gaim/clipseg-cloudsen12plus-fft --local-dir ./clipseg-fft

cloudprompts-infer \
  --data_root /path/to/data/test/images \
  --out_root /path/to/output \
  --checkpoint_type full \
  --checkpoint_dir ./clipseg-fft

# LoRA adapter
huggingface-cli download uga-gaim/clipseg-cloudsen12plus-lora --local-dir ./clipseg-lora

cloudprompts-infer \
  --data_root /path/to/data/test/images \
  --out_root /path/to/output \
  --checkpoint_type adapter \
  --checkpoint_dir ./clipseg-lora

Pretrained Models

Fine-tuned models are available on HuggingFace under the uga-gaim organization.

Model Type mIoU
clipseg-cloudsen12plus-fft Full fine-tune 0.6572
clipseg-cloudsen12plus-lora LoRA adapter 0.5991

Citation

If you use this code or models in your research, please cite:

@InProceedings{Kethavath_2026_CVPR,
    author    = {Kethavath, Harshith and Hu, Weiming},
    title     = {Low-Data Supervised Adaptation Outperforms Prompting for Cloud Segmentation Under Domain Shift},
    booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops},
    month     = {June},
    year      = {2026},
    pages     = {7960-7969}
}

License

This code is licensed under the Creative Commons Attribution 4.0 International License (CC BY 4.0).