Merge remote-tracking branch 'origin/fix_coco_scipts'

Author: paren8esis

README.md

@@ -3,7 +3,8 @@
 
 **Contributors:** [Sykas D.](https://github.com/dimsyk), [Zografakis D.](https://github.com/dimzog), [Sdraka M.](https://github.com/paren8esis)
 
-**This repository contains the models and training scripts for reproducing the experiments presented in [add publication].**
+**This repository contains the models and training scripts for reproducing the experiments presented in:**\
+[A Sentinel-2 multi-year, multi-country benchmark dataset for crop classification and segmentation with deep learning](https://ieeexplore.ieee.org/document/9749916).
 
 ### Requirements
 
@@ -36,7 +37,6 @@ For OAD:
 S4A-models\
     L dataset\
         L netcdf\
-        L annotations\
         L oad\
     L coco_files\
     L logs\
@@ -47,18 +47,14 @@ S4A-models\
             L mappings\
 ```
 
-#### Annotations
+#### COCO files
 
-In the `dataset/annotations/` folder are the annotations required for exporting the COCO files.
+In the `coco_files/` folder are the COCO files required for training, validating and testing the models. Some proof-of-concept COCO files are also given (with the refix `poc_`) just for testing and playing around.
 
 #### NetCDF4 files
 
 In the `dataset/netcdf/` folder you should place the netCDF4 files.
 
-#### COCO files
-
-In the `coco_files/` folder are the COCO files required for training, validating and testing the models. Some proof-of-concept COCO files are also given (with the refix `poc_`) just for testing and playing around.
-
 #### OAD files
 
 In the `dataset/oad/` folder you should place the exported files containing the OAD statistics.
@@ -72,13 +68,13 @@ Every script inherits settings from the aforementioned files.
 
 #### Essential scripts
 
-- `coco_data_split.py`: Uses the nectCDF4 data and the annotations to produce three COCO files for training, validation and testing.
-- `export_medians_multi.py`: Uses the netCDF4 data and the COCO files to compute the median image per month.
+- `coco_data_split.py`: Uses the nectCDF4 data to produce three COCO files for training, validation and testing.
+- `export_medians_multi.py`: Uses the netCDF4 data and the COCO files to compute the median image per month and export them to the disk.
 - `compute_class_weights.py`: Computes the class weights based on the exported medians, to account for class imbalance.
 - `object-based-csv.py`: Uses the netCDF4 data to compute the statistics required for OAD.
 - `pad_experiments.py`: The main script for training/testing the PAD models.
 - `oad_experiments.py`: The main script for training/testing the OAD models.
-- `visualize_predictions.py`: Produces a visualization of the ground truth and the prediction of a given model for a given image.
+- `visualize_predictions.py`: Produces a visualization of the ground truth and the prediction of a given model for a given image. Only relevant for PAD models.
 
 #### Using the repo
 
@@ -95,7 +91,7 @@ Every script inherits settings from the aforementioned files.
    python pad_experiments.py --train --model convlstm --parcel_loss --weighted_loss --root_path_coco <coco_folder_path> --prefix_coco <coco_file_prefix> --prefix <run_prefix> --num_epochs 10 --batch_size 32 --bands B02 B03 B04 B08 --saved_medians --img_size 61 61 --requires_norm --num_workers 16 --num_gpus 1 --window_len 12
    ```
    The above command is for training the **ConvLSTM** model using the **weighted parcel loss** described in the associated publication. Training will continue for **10 epochs** with **batch size 32**, using the Sentinel-2 **bands Blue (B02), Green (B03), Red (B04) and NIR (B08)**. The **input image size is 61x61**, the **precomputed medians are used** to speed up training and all input data are **normalized**. The **window length is 12**, including all months. Please use the `--help` argument to find information on all available parameters.
-2. Optionally, run `visualize_predictions.py` to visualize the image, ground truth and prediction for a specific model and image.
+2. Optionally, after training run `visualize_predictions.py` to visualize the image, ground truth and prediction for a specific model and image.
 
 **For OAD:**
 1. Run `oad_experiments.py` with the appropriate arguments. Example:
@@ -135,3 +131,18 @@ Scenario | Model | Acc. W. (%) | F1 W. (%) | Precision W. (%)
 3  | LSTM | **60.60** | **63.96** | **70.55**  
 3  | Transformer | 51.21 | 56.71 | 67.76
 3  | TempCNN | 52.32 | 57.38 | 68.35
+
+### Citation
+
+If you use our work, please cite:
+
+```
+@ARTICLE{
+  9749916,
+  author={Sykas, Dimitrios and Sdraka, Maria and Zografakis, Dimitrios and Papoutsis, Ioannis},
+  journal={IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing},
+  title={A Sentinel-2 multi-year, multi-country benchmark dataset for crop classification and segmentation with deep learning},
+  year={2022},
+  doi={10.1109/JSTARS.2022.3164771}
+}
+```

coco_data_split.py

@@ -28,6 +28,7 @@
 from utils.settings.mappings.mappings_cat import SAMPLE_TILES as CAT_TILES
 from utils.settings.mappings.mappings_fr import SAMPLE_TILES as FR_TILES
 from utils.settings.mappings.encodings_en import CROP_ENCODING
+from utils.settings.config import LINEAR_ENCODER
 
 from sklearn.preprocessing import MultiLabelBinarizer
 from skmultilearn.model_selection.iterative_stratification import IterativeStratification
@@ -73,6 +74,11 @@ def create_dataframe(data_path, tiles, years, common_labels=None):
 
         unique_labels = set(np.unique(labels.labels.data))
 
+        # Keep only labels contained in LINEAR_ENCODER
+        unique_labels = unique_labels & set(LINEAR_ENCODER.keys())
+
+        if (unique_labels) == 0: continue
+
         if common_labels is not None:
             # We want the train/val and test tiles to have common labels
             if unique_labels.isdisjoint(common_labels): continue
@@ -90,12 +96,8 @@ def create_dataframe(data_path, tiles, years, common_labels=None):
 
     parser.add_argument('--how', type=str, required=True, help='Perform a stratified split.',
                         choices=['stratified', 'random'])
-    parser.add_argument('--ann_limit', type=int, required=False, default=None,
-                        help='Maximum number of annotations per patch. Patches exceeding this limit, are dismissed')
     parser.add_argument('--data_path', type=str, default='dataset/netcdf/', required=False,
                         help='The path containing the data in netCDF format. Default "dataset/netcdf/".')
-    parser.add_argument('--data_ann', type=str, default='dataset/annotations/', required=False,
-                        help='The path containing subfolders with the annotations files. Default "dataset/annotations/".')
     parser.add_argument('--coco_path', type=str, default='coco_files/', required=False,
                         help='The path to export the COCO files into. Default "coco_files/"')
     parser.add_argument('--ratio', nargs='+', default=['60', '20', '20'], required=False,
@@ -130,7 +132,6 @@ def create_dataframe(data_path, tiles, years, common_labels=None):
     # Define paths
     data_path = Path(args.data_path)
     coco_path = Path(args.coco_path)
-    ann_path = Path(args.data_ann)
 
     # Ignore tile/year filtering in case an explicit experiment scheme is selected
     if args.experiment is not None:
@@ -253,41 +254,32 @@ def create_dataframe(data_path, tiles, years, common_labels=None):
         # Export COCO files
         create_coco_dataframe(df=X_train,
                               path_coco=coco_path / f'{prefix}_coco_train.json',
-                              ann_path=ann_path,
-                              ann_limit=args.ann_limit,
                               keep_tiles=train_tiles,
                               keep_years=train_years,
                               common_labels=common_lbls
                               )
 
         create_coco_dataframe(df=X_val,
                               path_coco=coco_path / f'{prefix}_coco_val.json',
-                              ann_path=ann_path,
-                              ann_limit=args.ann_limit,
                               keep_tiles=train_tiles,
                               keep_years=train_years,
                               common_labels=common_lbls
                               )
 
         create_coco_dataframe(df=X_test,
                               path_coco=coco_path / f'{prefix}_coco_test.json',
-                              ann_path=ann_path,
-                              ann_limit=args.ann_limit,
                               keep_tiles=test_tiles,
                               keep_years=test_years,
                               common_labels=common_lbls
                               )
 
     elif args.how == 'random':
         create_coco_netcdf(netcdf_path=data_path,
-                           ann_path=ann_path,
                            path_train=coco_path / f'{prefix}_coco_train.json',
                            path_test=coco_path / f'{prefix}_coco_test.json',
                            path_val=coco_path / f'{prefix}_coco_val.json',
-                           having_annotations=False,
                            train_r=train_r,
                            val_r=val_r,
-                           ann_limit=args.ann_limit,
                            keep_tiles=args.tiles,
                            keep_years=args.years,
                            experiment=args.experiment,

utils/PAD_dataset.py

@@ -328,11 +328,6 @@ def get_labels(self, netcdf: netCDF4.Dataset, start_bin: int) -> np.ndarray:
         return labels
 
 
-    def get_annotations(self, coco: COCO) -> dict:
-        # https://medium.com/fullstackai/how-to-train-an-object-detector-with-your-own-coco-dataset-in-pytorch-319e7090da5
-        raise NotImplementedError('')
-
-
     def load_medians(self, path: Path, subpatch_id: int, start_bin: int) -> Tuple[np.ndarray, np.ndarray]:
         """
         Loads precomputed medians for requested path.