Skip to content

Update README for the new structure #32

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
KyberNull wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Update README for the new structure #32

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
59 changes: 45 additions & 14 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ This project builds a deep learning pipeline for **semantic segmentation** of dr
- **SegFormer Backbone:** Transformer-based segmentation with a practical training stack
- **Config-Driven Workflow:** Shared and task-specific settings under `config/`
- **Robust Evaluation:** Quantitative metrics plus qualitative visualization outputs
- **Large-Raster Inference:** Patch-wise TIFF inference with vectorized shapefile export
- **Reliable Checkpointing:** Resume-safe transitions with model head compatibility handling

## 🗂️ Project Structure
Expand All @@ -23,6 +24,7 @@ This project builds a deep learning pipeline for **semantic segmentation** of dr
├── pretrain.py # Pretrain entrypoint
├── train.py # Train entrypoint
├── evaluate.py # Evaluation + visualization entrypoint
├── main.py # Large-image TIFF inference + vector export entrypoint
├── model.py # SegFormer model definition
├── losses.py # Losses and segmentation metrics
├── utils.py # Device/logging/checkpoint helper utilities
Expand All @@ -40,7 +42,7 @@ This project builds a deep learning pipeline for **semantic segmentation** of dr
├── training/
│ ├── train.py # Core training loop
│ ├── pretrain.py # Pretrain-related training helpers
│ ├── phase_io.py # Checkpoint/data IO helpers
│ ├── io.py # Checkpoint/data IO helpers
│ └── primitives.py # Shared training primitives
└── data/ # Local dataset roots and demos
```
Expand All @@ -63,43 +65,68 @@ Runtime behavior is controlled through files in `config/` (not CLI flags).
- `LEARNING_RATE = 6e-5`
- `WEIGHT_DECAY = 0.01`
- `WARMUP_EPOCHS = 5`
- `BATCH_SIZE = 8`
- `NUM_WORKERS = 2`
- `BATCH_SIZE = 1`
- `VAL_BATCH_SIZE = 1`
- `NUM_WORKERS = 1`
- `PREFETCH_FACTOR = 1`
- `VAL_INTERVAL = 1`
- `GRAD_ACCUM_STEPS = 1`
- `GRAD_ACCUM_STEPS = 8`
- `USE_GRADIENT_CHECKPOINTING = False`
- `USE_TORCH_COMPILE = True`
- `MODEL_PATH = "model.pt"`

**Pretrain defaults** (`config/pretrain.py`):

- `NUM_CLASSES_PRETRAIN = 8`
- `NUM_EPOCHS_PRETRAIN = 20`
- `NUM_VAL_SAMPLES_PRETRAIN = 150`
- `PRETRAIN_DATA_ROOT = "./data/phase-2"`
- `PRETRAIN_DATA_ROOT = "./data/pretrain"`
- `PRETRAIN_SCENES = ["rural", "urban"]`

**Train defaults** (`config/train.py`):

- `NUM_CLASSES_TRAIN = 4`
- `NUM_EPOCHS_TRAIN = NUM_EPOCHS_PRETRAIN + 50`
- `NUM_VAL_SAMPLES_TRAIN = 280`
- `TRAIN_IMG_DIR = "data/phase-3/TrainningDataset/processed_datasets"`
- `TRAIN_MASK_DIR = "data/phase-3/TrainningDataset/processed_masks"`
- `VAL_IMG_DIR = "data/phase-3/ValidationDataset/processed_datasets"`
- `VAL_MASK_DIR = "data/phase-3/ValidationDataset/processed_masks"`
- `TRAIN_IMG_DIR = "data/train/training_dataset/processed_datasets"`
- `TRAIN_MASK_DIR = "data/train/training_dataset/processed_masks"`
- `VAL_IMG_DIR = "data/train/validation_dataset/processed_datasets"`
- `VAL_MASK_DIR = "data/train/validation_dataset/processed_masks"`

**Evaluation defaults** (`config/eval.py`):

- `NUM_CLASSES_EVAL = 4`
- `NUM_BATCHES_EVAL = 8`
- `MAX_EXAMPLES_EVAL = 5`
- `IGNORE_LABEL = 255`
- `INPUT_DIR = "data/train/testing_dataset/processed_datasets"`
- `MASK_DIR = "data/train/testing_dataset/processed_masks"`

**Inference defaults** (`config/inference.py`):

- `PATCH_SIZE = 1024`
- `STRIDE = PATCH_SIZE`
- `NUM_CLASSES_INFERENCE = 4`
- `TEMP_DATASET_DIR = "data/input_demo"`
- `TEMP_MASK_DIR = "data/output_demo"`
- `CLEANUP_TEMP_DIRS = True`

## 💾 Data Setup

**Pretrain data (LoveDA)**

- Expected root: `data/phase-2/`
- Expected root: `data/pretrain/`
- Default layout (already present in this repo): `Train/Rural`, `Train/Urban`, `Val/Rural`, `Val/Urban`
- Scenes configured by default: `rural`, `urban`

**Train data (target geospatial dataset)**

- Train images: `data/phase-3/TrainningDataset/processed_datasets`
- Train masks: `data/phase-3/TrainningDataset/processed_masks`
- Val images: `data/phase-3/ValidationDataset/processed_datasets`
- Val masks: `data/phase-3/ValidationDataset/processed_masks`
- Train images: `data/train/training_dataset/processed_datasets`
- Train masks: `data/train/training_dataset/processed_masks`
- Val images: `data/train/validation_dataset/processed_datasets`
- Val masks: `data/train/validation_dataset/processed_masks`
- Test images (evaluation): `data/train/testing_dataset/processed_datasets`
- Test masks (evaluation): `data/train/testing_dataset/processed_masks`

**Mask convention**

Expand All @@ -118,6 +145,9 @@ uv run train.py

# 3) Evaluate
uv run evaluate.py

# 4) Large-image inference (interactive .tiff path prompt)
uv run main.py
```

Equivalent VS Code tasks are available: `Pretrain`, `Train`, and `Evaluate`.
Expand All @@ -128,6 +158,7 @@ Equivalent VS Code tasks are available: `Pretrain`, `Train`, and `Evaluate`.
- **Primary metrics:** mIoU, per-class IoU, pixel accuracy
- **Evaluation script outputs:** Mean Pixel Accuracy, Mean IoU, and processed-mask variants
- **Checkpoints:** Pretrain/Train flows read and write `model.pt`
- **Inference outputs:** Class-wise shapefiles (`Road.shp`, `BuildUpArea.shp`, `WaterBodies.shp`)
- **Resume behavior:** On class-count mismatch, incompatible segmentation head state is dropped to allow clean continuation

## 📊 Results & Visualizations
Expand Down