Merge pull request #483 from OjasChaudhari23/topic-recognition

Topic recognition

Author: shakes76

recognition/s4694189_UNET/README.md

@@ -0,0 +1,58 @@
+<h2> Name: Ojas Madhusudan Chaudhari </h2>
+
+<h2> Student Number: 46941893 </h2>
+
+<h2>Student Email: o.chaudhari@uqconnect.edu.au </h2>
+
+<h2>Project: Segmentation using UNet </h2>
+
+
+<h1> Segmentation of ISIC data with improved UNet </h1>
+
+This project develops a solution to the ISIC challenge using UNet. The objective of the project is to train convolutional neural network to segment the ISIC images using improved UNet. The architecture of improved UNet has been referred from F. Isensee, P. Kickingereder, W. Wick, M. Bendszus, and K. H. Maier-Hein’s “Brain Tumor Segmentation and Radiomics Survival Prediction: Contribution to the BRATS 2017 Challenge,” research paper. This paper gives the architecture of improved UNet. It consists of two parts: -
+* Downsampling
+* Upsampling
+
+Following diagram exactly depicts the architecture
+![Improved UNet](https://github.com/OjasChaudhari23/PatternFlow/blob/topic-recognition/recognition/s4694189_UNET/improvedunet.png)
+
+The neural network starts with giving an input layer of 256*256*3. The actual size of the images are 128*128. The images are then resized, normalized and given as array to the network. The neural network starts with contraction at start. It starts with 16 layers and increases in each step. Each step consists of two equal features. After it reaches to 256, Expansion starts. In this project Transposecv has been used for the expansion. In this code I have used dropout(0.1) as it was giving me a good result but it can be changed.
+
+The program starts with importing the data and making the appropriate pre-processing. The Unet structure takes the input in size 256*256*3. In downsampling part, the depth increases to 256 while in upsampling part, the model outputs the image in the same dimension as that of input. Maxpooling is used after each depth to half the size of image in downsampling part. The model has been trained on the whole training data, it is then validated with validation data and the results are observed on test data. Training data has not been split into train-validation-test data.  
+
+<h2>Running the program: </h2>
+Please download the modules.ipynb file to run the program or alternatively download modules.py, dataset.py, train.py and predict.py files and run train and predict files. The modules.ipynb has all the result data with the merged code and all other files have been created according to their functionality.
+
+<h2> File structure </h2>
+ * <b> dataset.py </b>:- This file loads the dataset and give the x and y array values with the pre-processing of images.
+ * <b> module.py </b>:- This file has UNET neural network defined in it and it returns the model
+ * <b> train.py </b> :- This file trains the model and plots the result. The model uses dice similarity measure to evaluate the model's performance.
+ * <b> predict.py </b> :- This file evaluates the performance on test data.
+
+<h2> Plots and visualizations</h2>
+
+ After building the model. coefficient loss of training and validation data has been visualized with respect to epochs
+ 
+ ![Loss](https://github.com/OjasChaudhari23/PatternFlow/blob/topic-recognition/recognition/s4694189_UNET/loss_picture.jpg)
+ 
+ Similarly dice coefficient values have been visualized
+ ![Dice](https://github.com/OjasChaudhari23/PatternFlow/blob/topic-recognition/recognition/s4694189_UNET/dice_coefficient.jpg)
+ 
+ In this project 3 data files which have been given in 2017 ISIC Challenge have been used. Train, validation and test images with their masks have been used for the study. Also the result on test data is quite good and it is more than 0.8
+ ![evaluation](https://github.com/OjasChaudhari23/PatternFlow/blob/topic-recognition/recognition/s4694189_UNET/prediction.jpg)
+ 
+ After visualizing the truth images with predicted images it is quite evident that the improved UNet has performed realy well on the test images.
+ ![Truth_vs_prediction](https://github.com/OjasChaudhari23/PatternFlow/blob/topic-recognition/recognition/s4694189_UNET/truth_vs_prediction.jpg)
+ 
+ This project uses tensorflow version 2.1 version and cv2 for image processing operations. Also Improved Unet has given better image segmentation result that Unet with greater than 80% dice similarity on the test set.
+ 
+ 
+
+
+
+
+
+
+
+ 
+

recognition/s4694189_UNET/dataset.py

@@ -0,0 +1,62 @@
+# ## Import the libraries
+from keras.utils import normalize
+import os
+import glob
+import cv2
+import numpy as np
+import matplotlib.pyplot as plt
+
+
+# ## Loading the images
+
+#Import the images and perform transformation on images
+
+transformed_X = 256
+transformed_Y = 256
+def load_images(path):
+    image_list = []
+    for fi in os.listdir(path):
+    #print(fi)
+        if fi.endswith(".csv") or "superpixels" in fi:
+            continue
+        img = cv2.imread(os.path.join(path, fi),cv2.IMREAD_COLOR)
+        img = cv2.resize(img,(transformed_Y,transformed_X))
+        img = img / 255.0
+        img = img.astype(np.float32)
+        image_list.append(img)
+    image_list = np.array(image_list)
+    return image_list
+
+
+# ## Load mask images
+
+#Import the mask dataset
+def load_masks(path):
+    masks_list = []
+    for fi in os.listdir(path):
+    #print(fi)
+        if fi.endswith(".csv") or "superpixels" in fi:
+            continue
+        mask = cv2.imread(os.path.join(path, fi),cv2.IMREAD_GRAYSCALE)
+        mask = cv2.resize(mask,(transformed_Y,transformed_X),interpolation = cv2.INTER_NEAREST)
+        mask = mask / 255.0
+        mask = mask.astype(np.float32)
+        masks_list.append(mask)
+    masks_list = np.array(masks_list)
+    return masks_list
+
+
+# ## Training data
+
+X_train = load_images("ISIC-2017_Training_Data/")
+masks_train_images = load_masks("ISIC-2017_Training_Part1_GroundTruth/")
+
+
+# ## Validation data
+x_validate = load_images("ISIC-2017_Validation_Data/")
+masks_valid_images = load_masks("ISIC-2017_Validation_Part1_GroundTruth")
+
+
+x_test = load_images("ISIC-2017_Test_v2_Data/")
+masks_test_images = load_images("ISIC-2017_Test_v2_Part1_GroundTruth")
+

recognition/s4694189_UNET/dice_coefficient.jpg

@@ -0,0 +1,79 @@
+
+from tensorflow.keras.layers import Conv2D, BatchNormalization, Activation, MaxPooling2D
+from tensorflow.keras.models import Model
+from tensorflow.keras.layers import Input, Conv2D, UpSampling2D, concatenate, Conv2DTranspose, Dropout
+from dataset import *
+
+def unet_model():
+    inputs = Input((256, 256, 3))
+    x = inputs 
+    # Contraction starts
+    ## 1st downsampled network
+    conv1 = Conv2D(16,(3,3), padding="same", activation="relu", kernel_initializer="he_normal")(x)
+    conv1 = Dropout(0.1)(conv1)
+    conv1 = Conv2D(16,(3,3), padding="same", activation="relu", kernel_initializer="he_normal")(conv1)
+    pool1 = MaxPooling2D((2,2))(conv1)
+    
+    ## 2nd downsampled network
+    conv2 = Conv2D(32,(3,3), padding="same", activation="relu", kernel_initializer="he_normal")(pool1)
+    conv2 = Dropout(0.1)(conv2)
+    conv2 = Conv2D(32,(3,3), padding="same", activation="relu", kernel_initializer="he_normal")(conv2)
+    pool2 = MaxPooling2D((2,2))(conv2)
+    
+    ## 3rd downsampled network
+    conv3 = Conv2D(64,(3,3), padding="same", activation="relu", kernel_initializer="he_normal")(pool2)
+    conv3 = Dropout(0.1)(conv3)
+    conv3 = Conv2D(64,(3,3), padding="same", activation="relu", kernel_initializer="he_normal")(conv3)
+    pool3 = MaxPooling2D((2,2))(conv3)
+    
+    ## 4th downsampled network
+    conv4 = Conv2D(128,(3,3), padding="same", activation="relu", kernel_initializer="he_normal")(pool3)
+    conv4 = Dropout(0.1)(conv4)
+    conv4 = Conv2D(128,(3,3), padding="same", activation="relu", kernel_initializer="he_normal")(conv4)
+    pool4 = MaxPooling2D((2,2))(conv4)
+    
+    ## 5th downsampled network
+    conv5 = Conv2D(256,(3,3), padding="same", activation="relu", kernel_initializer="he_normal")(pool4)
+    conv5 = Dropout(0.1)(conv5)
+    conv5 = Conv2D(256,(3,3), padding="same", activation="relu", kernel_initializer="he_normal")(conv5)
+    pool5 = MaxPooling2D((2,2))(conv5)
+    
+    ##xpansion starts
+    ## 1st upsampled network
+    upconv6 = Conv2DTranspose(128,(2,2),strides = (2,2),padding="same")(conv5)
+    upconv6 = concatenate([upconv6, conv4])
+    conv6 = Conv2D(128,(3,3), padding="same", activation="relu", kernel_initializer="he_normal")(upconv6)
+    conv6 = Dropout(0.1)(conv6)
+    conv6 = Conv2D(128,(3,3), padding="same", activation="relu", kernel_initializer="he_normal")(conv6)
+    
+    ## 2nd upsampled network
+    upconv7 = Conv2DTranspose(64,(2,2),strides = (2,2),padding="same")(conv6)
+    upconv7 = concatenate([upconv7, conv3])
+    conv7 = Conv2D(64,(3,3), padding="same", activation="relu", kernel_initializer="he_normal")(upconv7)
+    conv7 = Dropout(0.1)(conv7)
+    conv7 = Conv2D(64,(3,3), padding="same", activation="relu", kernel_initializer="he_normal")(conv7)
+    
+    ## 3rd upsampled network
+    upconv8 = Conv2DTranspose(32,(2,2),strides = (2,2),padding="same")(conv7)
+    upconv8 = concatenate([upconv8, conv2])
+    conv8 = Conv2D(32,(3,3), padding="same", activation="relu", kernel_initializer="he_normal")(upconv8)
+    conv8 = Dropout(0.1)(conv8)
+    conv8 = Conv2D(32,(3,3), padding="same", activation="relu", kernel_initializer="he_normal")(conv8)
+    
+    ## 4th upsampled network
+    upconv9 = Conv2DTranspose(16,(2,2),strides = (2,2),padding="same")(conv8)
+    upconv9 = concatenate([upconv9, conv1],axis=3)
+    conv9 = Conv2D(16,(3,3), padding="same", activation="relu", kernel_initializer="he_normal")(upconv9)
+    conv9 = Dropout(0.1)(conv9)
+    conv9 = Conv2D(16,(3,3), padding="same", activation="relu", kernel_initializer="he_normal")(conv9)
+    
+    ##Output layer
+    outputs = Conv2D(1,(1,1),activation="sigmoid")(conv9)
+
+    cnn_model = Model(inputs = [inputs], outputs=[outputs])
+    return cnn_model
+
+
+
+
+