Skip to content

Commit 9e710a4

Browse files
shakes76shakes76
authored
Merge pull request #487 from hotzan0301/topic-recognition
New pull request with correct branch
2 parents 1781121 + 16a231f commit 9e710a4

6 files changed

Lines changed: 474 additions & 0 deletions

File tree

‎recognition/MinsooHan_42570893/README.md‎

Lines changed: 66 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,66 @@
1+
# ISIC 2017 Skin lesion segementation using an improved U-Net
2+
3+
## Dataset
4+
ISIC2017 dataset can be found on the following link
5+
6+
ISIC2017 dataset : https://challenge.isic-archive.com/data/#2017
7+
8+
## Description of the algorithm
9+
![KakaoTalk_Snapshot_20221021_175510](https://user-images.githubusercontent.com/59554674/197143729-01160b28-8c62-4da2-b7b7-4b9041676450.png)
10+
11+
We applied the network architecture as seen in the image above. The network architecture was referenced from 'Brain Tumor Segmentation and Radiomics
12+
Survival Prediction: Contribution to the BRATS 2017 Challenge' by Isensee, et al. The link will be shared below.
13+
14+
Basically, this improved U-net is based on the existing U-net architectures. However, its core architecure inside the network is rather diffrent to the others.
15+
And the diffrences are that it has context, localization, and upsampling modules:
16+
17+
Context module consists of: Two 3 x 3 convolutional layers and a dropout layer with rate 0.3.
18+
Localization module consists of: A 3 x 3 convolutional layer and a 1 x 1 convolutional layer.
19+
Upsampling modules consists of: An upsampling layer and a 3 x 3 convolutional layer.
20+
21+
In this architecture, there are two pathways, which are the context pathway(left) and the localization pathway(right).
22+
Through the context pathway, the U-net reduces the resolution of the feature maps by using the context module.
23+
Through the localiztion pathway, the U-net take features for the lower level to the higher level by using the localization module that upsampling the low resolution feature maps. After this, the concatenation between the upsampled features and the context aggregation in the same level is done.
24+
25+
Finally, the output forms via element-wise sum between the concatenation above and the segmentation layers.
26+
27+
reference : https://arxiv.org/abs/1802.10508v1
28+
29+
## Task
30+
31+
Train a model for the image segmentation of skin lesion using the improved U-net architecture.
32+
33+
## How it works
34+
35+
### Data preprocessing
36+
Firstly, we needed to resize all the images to 256 x 256 because the images of the dataset did not have the same size. And saved all the resized images into new directories. Then, created blank arrays to store the resized training, validation, and test images and joined the resized images to the blank arrays respectively.
37+
This process has taken some time. And created data frames to store the excel files containing the names of the images. After that, loaded and generated the normalized images and masks of training, validation, and test data.
38+
39+
### Training
40+
After data preprocessing, we had train_x, train_y, validation_x, and validation_y. And loaded an improve U-net model that takes an image with 256 x 256 x 3 created by Input fucntion in Keras library. And we set the learning rate as 0.0005 and the decay rate as learning rate x 0.985 as the paper stated. And we compiled a model using Adam for optimization, and dice similarity loss wss used as a metric. And we fitted the model with train_x, train_y, validation_x, and validation_y. The batch size was 8, and epochs was 300.
41+
42+
### Results
43+
44+
We achieved a dice similarity of 0.51.
45+
46+
reference : https://arxiv.org/pdf/1606.04797v1.pdf
47+
48+
![KakaoTalk_Snapshot_20221021_224451](https://user-images.githubusercontent.com/59554674/197198882-7c2e081a-90f5-4db8-9886-19d771081ea4.png)
49+
50+
The images below are good cases and bad cases.
51+
#### Good results
52+
![Result_1](https://user-images.githubusercontent.com/59554674/197197864-a2404488-0146-4190-8e3e-35ded7f01f7f.png)
53+
![Result2](https://user-images.githubusercontent.com/59554674/197197901-ed236ae2-3b32-4811-8a2a-bc9b1f16d160.png)
54+
![Result3](https://user-images.githubusercontent.com/59554674/197197930-1021a9c5-31b7-4f7a-927a-2a42e103a797.png)
55+
56+
#### Bad results
57+
![BadResult](https://user-images.githubusercontent.com/59554674/197198195-f4304b79-409e-40ef-bf39-ff10fe5f41a8.png)
58+
![BadResult2](https://user-images.githubusercontent.com/59554674/197198811-41bb423e-4ad1-45af-bef6-54296ea50c1b.png)
59+
60+
## Environment
61+
OS : Windows10
62+
Python : 3.7
63+
Cuda : 10.1
64+
Tensorflow: 2.1.0
65+
66+

‎recognition/MinsooHan_42570893/dataset.py‎

Lines changed: 129 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,129 @@
1+
import os
2+
import cv2 as cv
3+
import SimpleITK as itk
4+
from skimage import io
5+
6+
7+
# Preprocesses the data and stores the preprocessed data into the directories.
8+
# And returns train_x, train_y, test_x, and test_y respectively.
9+
def data_preprocessing():
10+
print("Data preprocessing started")
11+
print("Data loading started")
12+
# Paths of the training, validation, and test images of ISIC dataset.
13+
14+
training_images_path = 'E:/Uni/COMP3710/ISIC-2017_Training_Data/'
15+
training_images = os.listdir(training_images_path)
16+
17+
training_ground_truth_images_path = 'E:/Uni/COMP3710/ISIC-2017_Training_Part1_GroundTruth/'
18+
training_ground_truth_images = os.listdir(training_ground_truth_images_path)
19+
20+
validation_images_path = 'E:/Uni/COMP3710/ISIC-2017_Validation_Data/'
21+
validation_images = os.listdir(validation_images_path)
22+
23+
validation_ground_truth_images_path = 'E:/Uni/COMP3710/ISIC-2017_Validation_Part1_GroundTruth/'
24+
validation_ground_truth_images = os.listdir(validation_ground_truth_images_path)
25+
26+
test_images_path = 'E:/Uni/COMP3710/ISIC-2017_Test_v2_Data/'
27+
test_images = os.listdir(test_images_path)
28+
29+
test_ground_truth_images_path = 'E:/Uni/COMP3710/ISIC-2017_Test_v2_Part1_GroundTruth/'
30+
test_ground_truth_images = os.listdir(test_ground_truth_images_path)
31+
print("Data loading ended")
32+
print("Data saving started")
33+
# Path for saving the data. Change the paths to where you are going to save the data.
34+
save_train_data = 'E:/Uni/COMP3710/Assignment/PatternFlow/recognition/MinsooHan_42570893/train_data/'
35+
save_training_ground_truth_data = 'E:/Uni/COMP3710/Assignment/PatternFlow/recognition/MinsooHan_42570893/train_ground_truth_data/'
36+
save_validation_data = 'E:/Uni/COMP3710/Assignment/PatternFlow/recognition/MinsooHan_42570893/validation_data/'
37+
save_validation_ground_truth_data = 'E:/Uni/COMP3710/Assignment/PatternFlow/recognition/MinsooHan_42570893/validation_ground_truth_data/'
38+
save_test_data = 'E:/Uni/COMP3710/Assignment/PatternFlow/recognition/MinsooHan_42570893/test_data/'
39+
save_test_ground_truth_data = 'E:/Uni/COMP3710/Assignment/PatternFlow/recognition/MinsooHan_42570893/test_ground_truth_data/'
40+
print("Data saving ended")
41+
# Create directories for saving.
42+
43+
if not os.path.exists(save_train_data):
44+
os.mkdir(save_train_data)
45+
if not os.path.exists(save_training_ground_truth_data):
46+
os.mkdir(save_training_ground_truth_data)
47+
if not os.path.exists(save_validation_data):
48+
os.mkdir(save_validation_data)
49+
if not os.path.exists(save_validation_ground_truth_data):
50+
os.mkdir(save_validation_ground_truth_data)
51+
if not os.path.exists(save_test_data):
52+
os.mkdir(save_test_data)
53+
if not os.path.exists(save_test_ground_truth_data):
54+
os.mkdir(save_test_ground_truth_data)
55+
56+
image_size = 256
57+
58+
# Define functions for creating training, validation, and test images.
59+
60+
def create_training_images():
61+
data = []
62+
for image in training_images:
63+
read_image = itk.ReadImage(os.path.join(training_images_path, image))
64+
image_array = itk.GetArrayFromImage(read_image)
65+
resized_array = cv.resize(image_array, (image_size, image_size))
66+
data.append([resized_array])
67+
io.imsave(save_train_data + image[:-4] + '.png', resized_array)
68+
return data
69+
70+
def create_training_ground_truth_images():
71+
data = []
72+
for image in training_ground_truth_images:
73+
read_image = itk.ReadImage(os.path.join(training_ground_truth_images_path, image))
74+
image_array = itk.GetArrayFromImage(read_image)
75+
resized_array = cv.resize(image_array, (image_size, image_size))
76+
data.append([resized_array])
77+
io.imsave(save_training_ground_truth_data + image[:-4] + '.png', resized_array)
78+
return data
79+
80+
def create_validation_images():
81+
data = []
82+
for image in validation_images:
83+
read_image = itk.ReadImage(os.path.join(validation_images_path, image))
84+
image_array = itk.GetArrayFromImage(read_image)
85+
resized_array = cv.resize(image_array, (image_size, image_size))
86+
data.append([resized_array])
87+
io.imsave(save_validation_data + image[:-4] + '.png', resized_array)
88+
return data
89+
90+
def create_validation_ground_truth_images():
91+
data = []
92+
for image in validation_ground_truth_images:
93+
read_image = itk.ReadImage(os.path.join(validation_ground_truth_images_path, image))
94+
image_array = itk.GetArrayFromImage(read_image)
95+
resized_array = cv.resize(image_array, (image_size, image_size))
96+
data.append([resized_array])
97+
io.imsave(save_validation_ground_truth_data + image[:-4] + '.png', resized_array)
98+
return data
99+
100+
def create_test_images():
101+
data = []
102+
for image in test_images:
103+
read_image = itk.ReadImage(os.path.join(test_images_path, image))
104+
image_array = itk.GetArrayFromImage(read_image)
105+
resized_array = cv.resize(image_array, (image_size, image_size))
106+
data.append([resized_array])
107+
io.imsave(save_test_data + image[:-4] + '.png', resized_array)
108+
return data
109+
110+
def create_test_ground_truth_images():
111+
data = []
112+
for image in test_ground_truth_images:
113+
read_image = itk.ReadImage(os.path.join(test_ground_truth_images_path, image))
114+
image_array = itk.GetArrayFromImage(read_image)
115+
resized_array = cv.resize(image_array, (image_size, image_size))
116+
data.append([resized_array])
117+
io.imsave(save_test_ground_truth_data + image[:-4] + '.png', resized_array)
118+
return data
119+
120+
print("Creating images started")
121+
# Create the images
122+
123+
create_training_images()
124+
create_training_ground_truth_images()
125+
create_validation_images()
126+
create_validation_ground_truth_images()
127+
create_test_images()
128+
create_test_ground_truth_images()
129+
print("Creating images ended")

‎recognition/MinsooHan_42570893/modules.py‎

Lines changed: 96 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,96 @@
1+
2+
from tensorflow.keras.models import Model
3+
from tensorflow.keras.layers import Conv2D, MaxPool2D, Dense, Concatenate, UpSampling2D, Input, Activation, add, \
4+
BatchNormalization, Dropout, Softmax, LeakyReLU
5+
6+
7+
# Create a context_module
8+
def context_module(input_image, filters, kernel_size=(3, 3), padding="same", strides=1):
9+
block = Conv2D(filters, kernel_size, strides, padding, activation=LeakyReLU(alpha=0.02))(input_image)
10+
block = BatchNormalization()(block)
11+
block = Conv2D(filters, kernel_size, strides, padding, activation=LeakyReLU(alpha=0.02))(block)
12+
block = BatchNormalization()(block)
13+
block = Dropout(rate=0.3)(block)
14+
return block
15+
16+
17+
# Create a localization_module
18+
def localization_module(input_image, filters):
19+
block = Conv2D(filters, kernel_size=(3, 3), padding="same", strides=1, activation=LeakyReLU(alpha=0.02))(input_image)
20+
block = BatchNormalization()(block)
21+
block = Conv2D(filters, kernel_size=(1, 1), padding="same", strides=1, activation=LeakyReLU(alpha=0.02))(block)
22+
block = BatchNormalization()(block)
23+
return block
24+
25+
26+
# Create an upsampling_module
27+
def upsampling_module(input_image, filters):
28+
block = UpSampling2D((2, 2))(input_image)
29+
block = Conv2D(filters, kernel_size=(3, 3), strides=1, padding="same", activation=LeakyReLU(alpha=0.02))(block)
30+
block = BatchNormalization()(block)
31+
return block
32+
33+
34+
# improved_Unet
35+
def improved_Unet(input_image):
36+
# Contracting path
37+
enc1_1 = Conv2D(filters=16, kernel_size=(3, 3), padding="same", strides=1, activation=LeakyReLU(alpha=0.02))(input_image)
38+
enc1_1 = BatchNormalization()(enc1_1)
39+
enc1_2 = context_module(enc1_1, filters=16)
40+
enc1 = add([enc1_1, enc1_2])
41+
42+
enc2_1 = Conv2D(filters=32, kernel_size=(3, 3), padding="same", strides=2, activation=LeakyReLU(alpha=0.02))(enc1)
43+
enc2_1 = BatchNormalization()(enc2_1)
44+
enc2_2 = context_module(enc2_1, filters=32)
45+
enc2 = add([enc2_1, enc2_2])
46+
47+
enc3_1 = Conv2D(filters=64, kernel_size=(3, 3), padding="same", strides=2, activation=LeakyReLU(alpha=0.02))(enc2)
48+
enc3_1 = BatchNormalization()(enc3_1)
49+
enc3_2 = context_module(enc3_1, filters=64)
50+
enc3 = add([enc3_1, enc3_2])
51+
52+
enc4_1 = Conv2D(filters=128, kernel_size=(3, 3), padding="same", strides=2, activation=LeakyReLU(alpha=0.02))(enc3)
53+
enc4_1 = BatchNormalization()(enc4_1)
54+
enc4_2 = context_module(enc4_1, filters=128)
55+
enc4 = add([enc4_1, enc4_2])
56+
57+
enc5_1 = Conv2D(filters=256, kernel_size=(3, 3), padding="same", strides=2, activation=LeakyReLU(alpha=0.02))(enc4)
58+
enc5_1 = BatchNormalization()(enc5_1)
59+
enc5_2 = context_module(enc5_1, filters=256)
60+
enc5_3 = add([enc5_1, enc5_2])
61+
enc5 = upsampling_module(enc5_3, filters=128)
62+
63+
# Expansive path
64+
dec1_1 = Concatenate()([enc4, enc5])
65+
dec1_2 = localization_module(dec1_1, filters=128)
66+
dec1 = upsampling_module(dec1_2, filters=64)
67+
68+
dec2_1 = Concatenate()([enc3, dec1])
69+
dec2_2 = localization_module(dec2_1, filters=64)
70+
dec2 = upsampling_module(dec2_2, filters=32)
71+
72+
dec3_1 = Concatenate()([enc2_2, dec2])
73+
dec3_2 = localization_module(dec3_1, filters=32)
74+
dec3 = upsampling_module(dec3_2, filters=16)
75+
76+
dec4_1 = Concatenate()([enc1_2, dec3])
77+
dec4_2 = Conv2D(filters=32, kernel_size=(3, 3), strides=1, padding="same", activation=LeakyReLU(alpha=0.02))(dec4_1)
78+
dec4_2 = BatchNormalization()(dec4_2)
79+
dec4 = Conv2D(filters=32, kernel_size=(1, 1), strides=1, padding="same", activation=LeakyReLU(alpha=0.02))(dec4_2)
80+
dec4 = BatchNormalization()(dec4)
81+
82+
# Element-wise sum between segmentation layers
83+
seg1 = Conv2D(filters=32,kernel_size=(1, 1), strides=1, padding="same", activation=LeakyReLU(alpha=0.02))(dec2_2)
84+
seg1 = BatchNormalization()(seg1)
85+
seg1 = UpSampling2D((2, 2))(seg1)
86+
seg2 = Conv2D(filters=32,kernel_size=(1, 1), strides=1, padding="same", activation=LeakyReLU(alpha=0.02))(dec3_2)
87+
seg2 = BatchNormalization()(seg2)
88+
seg3 = add([seg1, seg2])
89+
seg3 = UpSampling2D((2, 2))(seg3)
90+
seg4 = Concatenate()([dec4, seg3])
91+
92+
output = Conv2D(3, (1, 1), padding="same", activation="softmax")(seg4)
93+
model = Model(input_image, output)
94+
return model
95+
96+

‎recognition/MinsooHan_42570893/predict.py‎

Lines changed: 85 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,85 @@
1+
import os
2+
import utils
3+
import SimpleITK as itk
4+
import numpy as np
5+
import pandas as pd
6+
from skimage import io
7+
import tensorflow
8+
from tensorflow.keras.models import load_model
9+
from matplotlib import pyplot as plt
10+
11+
# Path of test, test GT, and test metadata
12+
save_test_data = 'E:/Uni/COMP3710/Assignment/PatternFlow/recognition/MinsooHan_42570893/test_data/'
13+
save_test_ground_truth_data = 'E:/Uni/COMP3710/Assignment/PatternFlow/recognition/MinsooHan_42570893/test_ground_truth_data/'
14+
metadata_test = pd.read_csv('E:/Uni/COMP3710/ISIC-2017_Test_v2_Data_metadata.csv')
15+
16+
17+
# Define a function that generates test data
18+
def test_data():
19+
x, y = [], []
20+
for index, cell in metadata_test.iterrows():
21+
read_image = itk.ReadImage(save_test_data + cell[0] + '.png')
22+
image_array = itk.GetArrayFromImage(read_image) / 255.0
23+
mask = io.imread(save_test_ground_truth_data + cell[0] + '_segmentation.png') / 255.0
24+
x.append(image_array)
25+
y.append(mask)
26+
return x, y
27+
28+
29+
# Define a function that loads an image and the corresponding GT image at index i.
30+
def visualize_image(i):
31+
image_list = os.listdir(save_test_data)
32+
index = i
33+
image_GT_list = os.listdir(save_test_ground_truth_data)
34+
image = itk.ReadImage(os.path.join(save_test_data, image_list[index]))
35+
image_array = itk.GetArrayFromImage(image) / 255.0
36+
37+
image_GT = io.imread(os.path.join(save_test_ground_truth_data, image_GT_list[index])) / 255.0
38+
39+
return image_array, image_GT
40+
41+
42+
# Generate test_x and test_y
43+
test_x, test_y = test_data()
44+
test_x = np.array(test_x)
45+
test_y = np.array(test_y)
46+
test_y = np.expand_dims(test_y, axis=-1)
47+
z3 = np.zeros(test_y.shape[:-1] + (2,), dtype=test_y.dtype)
48+
test_y = np.concatenate((test_y, z3), axis=-1)
49+
50+
# Load the model 'improvedUnet'
51+
model = tensorflow.keras.models.load_model(
52+
'improvedUnet.h5',
53+
custom_objects={
54+
"dice_coef_loss": utils.dice_coef_loss,
55+
"dice_coef": utils.dice_coef,
56+
"LeakyReLU": tensorflow.keras.layers.LeakyReLU,
57+
},
58+
)
59+
60+
# Visualize original test image, test GT, and a predicted GT image.
61+
figure = plt.figure()
62+
figure.subplots_adjust(hspace=0.4, wspace=0.4)
63+
64+
# Run several times changing the parameter of visualize_image()
65+
image_array, image_GT = visualize_image(100)
66+
test_image = np.expand_dims(image_array, axis=0)
67+
result = model.predict(test_image)
68+
result = result > 0.5
69+
result = np.squeeze(result, axis=0)
70+
71+
show_image = figure.add_subplot(1, 3, 1)
72+
show_image.imshow(image_array)
73+
plt.title("Test image")
74+
75+
show_image = figure.add_subplot(1, 3, 2)
76+
show_image.imshow(image_GT, cmap="gray")
77+
plt.title("Test GT image")
78+
79+
show_image = figure.add_subplot(1, 3, 3)
80+
show_image.imshow(result * 255, cmap="gray")
81+
plt.title("Predicted GT")
82+
plt.show()
83+
84+
# Store the scores.
85+
loss, dice_coef = model.evaluate(test_x, test_y)

0 commit comments

Comments
 (0)