upload latest slides

Author: elenaivadreyer

.gitignore

@@ -158,4 +158,7 @@ data/
 # Exclude not needed notebooks files
 notebooks/exploration.ipynb
 notebooks/class_template.ipynb
-notebooks/few_shot_approaches_setup.ipynb
+notebooks/few_shot_approaches_setup.ipynb
+
+presentation/tutorial-new-tutorial-group-1-NARRATIVE.html
+presentation/figures/tutorial-new-tutorial-group-1-NARRATIVE.ipynb

presentation/dimmery.scss

@@ -357,7 +357,7 @@ $presentation-h3-font-size: 1.3em;
 
 /* Make references tiny */
 .refs-super-small {
-  font-size: 0.6em !important;
+  font-size: 0.5em !important;
   line-height: 1.1em !important;
 }
 

presentation/figures/meta_training_loss.png

@@ -1,7 +1,7 @@
 ---
 title: "Few-Shot Learning for Rooftop Detection in Satellite Imagery"
 subtitle: "Deep Learning Tutorial"
-author: "Giorgio Coppala, Nadine Daum, Elena Dreyer, Nico Reichardt"
+author: "Giorgio Coppola, Nadine Daum, Elena Dreyer, Nico Reichardt"
 bibliography: refs.bib
 
 
@@ -31,17 +31,17 @@ format:
 ## Policy Relevance
 
 - Many public auhorities face the problem of **limited labeled data**
-  (annotation is expensive, slow, or requires domain expertise)
+  - Annotation is expensive, slow, or requires domain expertise
 
 - **Applications:**
-  - medical sector: **rare disease detection**
-  - emergency management: **flood extent mapping**
-  - climate & energy: **solar PV rooftop assessment**
-  - urban planning: **building footprints & infrastructure mapping**
+  - Medical sector: **rare disease detection**
+  - Emergency management: **flood extent mapping**
+  - Climate & energy: **solar PV rooftop assessment**
+  - Urban planning: **building footprints & infrastructure mapping**
 
 - **Few-shot learning (FSL)** can help:
   - Learns to **generalize** from *1–5 labeled support examples per class*
-  - (in our case) learns a **feature embedding** and constructs **class prototypes**
+  - (In our case) learns **feature embeddings** and constructs **class prototypes**
   - Enables segmentation in a **new city** with *minimal additional annotation*
 
 
@@ -63,16 +63,15 @@ rooftop shapes and textures from a small Geneva subset
 :::
 
 ::: {.column width="45%"}
-![](figures/picture_use_case.png){width="100%" style="margin-top: 1rem;"}
 
-<div style="font-size: 0.75rem; color:#666; text-align:center; margin-top:0.2rem;">
-Demonstration use case (self-made visualization)
-</div>
+![](figures/picture_use_case.png){width=90% style="margin-left: 30%;"}
+
 :::
 
 :::
 
 
+
 ## Dataset: [Roofs of Geneva](https://huggingface.co/datasets/raphaelattias/overfitteam-geneva-satellite-images)
 
 - **Size**: 1,050 labeled image-mask pairs
@@ -97,6 +96,84 @@ Geneva Animation: raw image → overlay rooftop → binary mask
 </div>
 
 
+
+## Few Shot Learning in General
+
+#### Few-Shot Learning (FSL)
+- Learning new **tasks, labels, or segmentations** from very few labeled examples
+  *(N-way, K-shot)*
+
+#### Few-Shot Semantic Segmentation (FSSS)
+- **Goal**: Segment novel object classes using only a few annotated examples
+- Assigning a class label to **every pixel**
+
+
+---
+
+## Prototypical Networks (ProtNets)
+
+* Learn a shared **embedding space** via a backbone model
+* Pixels belonging to the same class are **close in feature space**
+* Class representations are formed as **prototypes**
+* Training follows an **episodic framework**
+* Each episode consists of:
+  - **Support set**:
+    Few images with **pixel-level masks**
+    Defines the target classes
+  - **Query image**:
+    Image where the model must segment the target classes
+
+## Prototypical Network Overview
+
+#### Workflow
+* Support Image → Prototype → Similarity → Query Segmentation
+
+
+#### Feature Extraction
+* **Backbone:** ResNet-18 CNN, pretrained on ImageNet
+* **Projection:** feature maps → embedding dimension (256 channels)
+
+
+#### Evaluation Metric
+$$
+\mathrm{IoU} = \frac{|A \cap B|}{|A \cup B|}
+$$
+
+
+---
+
+## Prototypical Network Overview
+
+![](figures/illustration_prototypical_network.png){width=100% fig-align="center"}
+
+<div style="font-size:0.75rem; text-align:center; color:#666; margin-top:0.5rem;">
+Modified figure from <a href="https://arxiv.org/abs/2210.16829">(Ding et al. 2022)</a>
+</div>
+
+
+---
+
+## (Preliminary) Results
+
+#### (1) Meta training loss
+
+The “avg episode loss” at each epoch is the average cross-entropy error over all support–query tasks in that epoch. The encoder is successfully learning a feature space where prototype-based segmentation works increasingly well.
+
+![](figures/meta_training_loss.png){width="50%" fig-align="center"}
+
+
+---
+
+## (Preliminary) Results
+
+#### (2) Predicted masks
+
+With 5-shot learning, the predicted masks have a mean IoU over 102 test samples of 0.485.
+
+Here an example:
+
+![](figures/predicted_mask.png){width=80% fig-align="center"}
+
 ## Discussion
 
 **Room for improvement:**
@@ -128,7 +205,6 @@ Geneva Animation: raw image → overlay rooftop → binary mask
 </div>
 
 
-
 ## References
 
 ::: {.refs-super-small}

presentation/figures/predicted_mask.png

@@ -1,13 +1,12 @@
 ---
 title: "Few-Shot Learning for Rooftop Detection in Satellite Imagery"
 subtitle: "Deep Learning Tutorial"
-author: "Giorgio Coppala, Nadine Daum, Elena Dreyer, Nico Reichardt"
+author: "Giorgio Coppola, Nadine Daum, Elena Dreyer, Nico Reichardt"
 bibliography: refs.bib
 
 
 resources:
   - img/**
-  - figures/**
 
 format:
   revealjs:
@@ -29,86 +28,151 @@ format:
 ---
 
 
+## Policy Relevance
+
+- Many public auhorities face the problem of **limited labeled data**
+  - Annotation is expensive, slow, or requires domain expertise
+
+- **Applications:**
+  - Medical sector: **rare disease detection**
+  - Emergency management: **flood extent mapping**
+  - Climate & energy: **solar PV rooftop assessment**
+  - Urban planning: **building footprints & infrastructure mapping**
+
+- **Few-shot learning (FSL)** can help:
+  - Learns to **generalize** from *1–5 labeled support examples per class*
+  - (In our case) learns **feature embeddings** and constructs **class prototypes**
+  - Enables segmentation in a **new city** with *minimal additional annotation*
+
+
 
 ## Problem Setting
 
-::: columns
-::: column
-- Cities need accurate rooftop maps to plan and scale **solar PV installations**
+::: {.columns}
 
-- Manual rooftop labeling is **slow** and **costly**
+::: {.column width="55%"}
 
-- Every city looks different → traditional models do not **generalize** well
+- Goal of the tutorial: apply **Prototypical Networks** to
+rooftop segmentation using only a few labeled tiles
 
-### Idea:
+- **Few-shot segmentation** allows the model to learn characteristic
+rooftop shapes and textures from a small Geneva subset
 
-- **Few-shot learning** makes segmentation possible with only a handful of labeled examples
+- Demonstrates how rooftop maps can be produced for solar potential estimation in a **new geographic setting** with limited labels
 
 :::
 
-::: column
+::: {.column width="45%"}
+
 ![](figures/picture_use_case.png){width=90% style="margin-left: 30%;"}
+
 :::
+
 :::
 
 
-## Dataset: [Rooftops of Geneva](https://huggingface.co/datasets/raphaelattias/overfitteam-geneva-satellite-images)
 
-::: {.two-col-60-40}
-::: {.col-left}
+## Dataset: [Roofs of Geneva](https://huggingface.co/datasets/raphaelattias/overfitteam-geneva-satellite-images)
 
-- **Satellite Images**: High-resolution RGB satellite images of Geneva available on Huggingface
 - **Size**: 1,050 labeled image-mask pairs
+
 - **Task**: Binary segmentation masks (rooftop vs background)
+
 - **Geographic splits**: 3 grids/ neighborhoods (North, Center, South)
+
 - **Image size**: 250x250 pixels
+
 - **Categories**: Industrial, Residential
 
-:::
 
-::: {.col-right}
+## Inside the dataset
 
-![](figures/grids_animation.gif){width=80%}
-![](figures/geneva-map-gif.gif){width=80%}
+<div style="text-align:center;">
+![](figures/grids_animation.gif){width="50%"}
+</div>
+
+<div style="font-size:0.75rem; text-align:center; color:#666; margin-top:0.5rem;">
+Geneva Animation: raw image → overlay rooftop → binary mask
+</div>
 
-:::
-:::
 
 
 ## Few Shot Learning in General
 
-tbd
+#### Few-Shot Learning (FSL)
+- Learning new **tasks, labels, or segmentations** from very few labeled examples
+  *(N-way, K-shot)*
 
+#### Few-Shot Semantic Segmentation (FSSS)
+- **Goal**: Segment novel object classes using only a few annotated examples
+- Assigning a class label to **every pixel**
 
-## Prototypical Network
 
-::: {.two-col-80-20}
-::: {.col-left}
-![](figures/illustration_prototypical_network.png){width=100%}
+---
 
-*(modified from)* [SRPNet](https://arxiv.org/abs/2210.16829)
+## Prototypical Networks (ProtNets)
 
-:::
+* Learn a shared **embedding space** via a backbone model
+* Pixels belonging to the same class are **close in feature space**
+* Class representations are formed as **prototypes**
+* Training follows an **episodic framework**
+* Each episode consists of:
+  - **Support set**:
+    Few images with **pixel-level masks**
+    Defines the target classes
+  - **Query image**:
+    Image where the model must segment the target classes
 
-::: {.col-right}
-- high-level schematic (support → prototype → similarity → segmentation)
-- 1-way-1-shot → explain what it means
-- Data preprocessing (augmentation, geographic splits)
-- Model architecture (feature extraction, CNN layers, backbone)
-- Training strategy
-  - Loss function
-  - Evaluation metrics
+## Prototypical Network Overview
 
-:::
-:::
+#### Workflow
+* Support Image → Prototype → Similarity → Query Segmentation
+
+
+#### Feature Extraction
+* **Backbone:** ResNet-18 CNN, pretrained on ImageNet
+* **Projection:** feature maps → embedding dimension (256 channels)
+
+
+#### Evaluation Metric
+$$
+\mathrm{IoU} = \frac{|A \cap B|}{|A \cup B|}
+$$
+
+
+---
+
+## Prototypical Network Overview
+
+![](figures/illustration_prototypical_network.png){width=100% fig-align="center"}
+
+<div style="font-size:0.75rem; text-align:center; color:#666; margin-top:0.5rem;">
+Modified figure from <a href="https://arxiv.org/abs/2210.16829">(Ding et al. 2022)</a>
+</div>
+
+
+---
+
+## (Preliminary) Results
+
+#### (1) Meta training loss
+
+The “avg episode loss” at each epoch is the average cross-entropy error over all support–query tasks in that epoch. The encoder is successfully learning a feature space where prototype-based segmentation works increasingly well.
+
+![](figures/meta_training_loss.png){width="50%" fig-align="center"}
+
+
+---
 
 ## (Preliminary) Results
 
-- Show performance for 1-shot / 5-shot / full-data comparison
+#### (2) Predicted masks
 
-- Show predicted masks
+With 5-shot learning, the predicted masks have a mean IoU over 102 test samples of 0.485.
 
+Here an example:
 
+![](figures/predicted_mask.png){width=80% fig-align="center"}
 
 ## Discussion
 
@@ -141,7 +205,6 @@ tbd
 </div>
 
 
-
 ## References
 
 ::: {.refs-super-small}
@@ -174,10 +237,3 @@ tbd
 :::
 
 
-
-
-
-
-
-
-