Skip to content

Commit e7efab1

Browse files
ZhongkuiMaZhongkuiMa
committed
First commit
0 parents  commit e7efab1

76 files changed

Lines changed: 15981 additions & 0 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.gitignore‎

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
1+
# ignore the setting of pycharm
2+
.idea
3+
4+
# ignore the model files of gurobi
5+
.ilp
6+
7+
# ignore the log files
8+
*.log
9+
*.txt
10+
# Do not ignore the requirements.txt file
11+
!requirements.txt
12+
13+
# ignore the setting of mac
14+
.DS_Store
15+
16+
# ignore all the python cache
17+
*.pyc
18+
19+
# ignore the incompleted files.
20+
tests
21+
22+
23+
# ignore the cache directory
24+
__pycache__
25+
.temp/
26+
.nets/
27+
28+
.txt
29+
.pdf

‎LICENSE‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
MIT License
2+
3+
Copyright (c) 2025 Zhongkui Ma
4+
5+
Permission is hereby granted, free of charge, to any person obtaining a copy
6+
of this software and associated documentation files (the "Software"), to deal
7+
in the Software without restriction, including without limitation the rights
8+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9+
copies of the Software, and to permit persons to whom the Software is
10+
furnished to do so, subject to the following conditions:
11+
12+
The above copyright notice and this permission notice shall be included in all
13+
copies or substantial portions of the Software.
14+
15+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21+
SOFTWARE.

‎README.md‎

Lines changed: 344 additions & 0 deletions
Large diffs are not rendered by default.

‎archived_logs/.gitkeep‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
# This folder is intentionally left empty to ensure that the directory is included in version control.
2+
# You can download the our archived_logs from the following Google Drive link:
3+
# https://drive.google.com/drive/folders/1C4kYaKb_Pd3xCo6aCy6W80tw43CM8Nn8?usp=sharing

‎evaluation_verification/cifar10_relulike.sh‎

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
#!/bin/bash
2+
3+
python3 exp.py --net_file_path ../nets/leaky_relu/cifar_leaky_relu_3_500.onnx --dataset cifar10 --perturbation_radius 0.004 --bp rover --log_name cifar_leaky_relu_3_500_ROVER
4+
python3 exp.py --net_file_path ../nets/leaky_relu/cifar_leaky_relu_6_500.onnx --dataset cifar10 --perturbation_radius 0.0025 --bp rover --log_name cifar_leaky_relu_6_500_ROVER
5+
python3 exp.py --net_file_path ../nets/leaky_relu/cifar_leaky_relu_convsmall.onnx --dataset cifar10 --perturbation_radius 0.004 --bp rover --log_name cifar_leaky_relu_convsmall_ROVER
6+
python3 exp.py --net_file_path ../nets/leaky_relu/cifar_leaky_relu_convmed.onnx --dataset cifar10 --perturbation_radius 0.004 --bp rover --log_name cifar_leaky_relu_convmed_ROVER
7+
python3 exp.py --net_file_path ../nets/elu/cifar_elu_3_500.onnx --dataset cifar10 --perturbation_radius 0.003 --bp rover --log_name cifar_elu_3_500_ROVER
8+
python3 exp.py --net_file_path ../nets/elu/cifar_elu_6_500.onnx --dataset cifar10 --perturbation_radius 0.003 --bp rover --log_name cifar_elu_6_500_ROVER
9+
python3 exp.py --net_file_path ../nets/elu/cifar_elu_convsmall.onnx --dataset cifar10 --perturbation_radius 0.0035 --bp rover --log_name cifar_elu_convsmall_ROVER
10+
python3 exp.py --net_file_path ../nets/elu/cifar_elu_convmed.onnx --dataset cifar10 --perturbation_radius 0.0035 --bp rover --log_name cifar_elu_convmed_ROVER
11+
12+
13+
python3 exp.py --net_file_path ../nets/leaky_relu/cifar_leaky_relu_3_500.onnx --dataset cifar10 --perturbation_radius 0.004 --bp rover --opt lp --log_name cifar_leaky_relu_3_500_ROVER_LP
14+
python3 exp.py --net_file_path ../nets/leaky_relu/cifar_leaky_relu_6_500.onnx --dataset cifar10 --perturbation_radius 0.0025 --bp rover --opt lp --log_name cifar_leaky_relu_6_500_ROVER_LP
15+
python3 exp.py --net_file_path ../nets/leaky_relu/cifar_leaky_relu_convsmall.onnx --dataset cifar10 --perturbation_radius 0.004 --bp rover --opt lp --log_name cifar_leaky_relu_convsmall_ROVER_LP
16+
python3 exp.py --net_file_path ../nets/leaky_relu/cifar_leaky_relu_convmed.onnx --dataset cifar10 --perturbation_radius 0.004 --bp rover --opt lp --log_name cifar_leaky_relu_convmed_ROVER_LP
17+
python3 exp.py --net_file_path ../nets/elu/cifar_elu_3_500.onnx --dataset cifar10 --perturbation_radius 0.003 --bp rover --opt lp --log_name cifar_elu_3_500_ROVER_LP
18+
python3 exp.py --net_file_path ../nets/elu/cifar_elu_6_500.onnx --dataset cifar10 --perturbation_radius 0.003 --bp rover --opt lp --log_name cifar_elu_6_500_ROVER_LP
19+
python3 exp.py --net_file_path ../nets/elu/cifar_elu_convsmall.onnx --dataset cifar10 --perturbation_radius 0.0035 --bp rover --opt lp --log_name cifar_elu_convsmall_ROVER_LP
20+
python3 exp.py --net_file_path ../nets/elu/cifar_elu_convmed.onnx --dataset cifar10 --perturbation_radius 0.0035 --bp rover --opt lp --log_name cifar_elu_convmed_ROVER_LP
21+
22+
23+
python3 exp.py --net_file_path ../nets/leaky_relu/cifar_leaky_relu_3_500.onnx --dataset cifar10 --perturbation_radius 0.004 --bp rover --opt mnlp --log_name cifar_leaky_relu_3_500_ROVER_MNLP
24+
python3 exp.py --net_file_path ../nets/leaky_relu/cifar_leaky_relu_6_500.onnx --dataset cifar10 --perturbation_radius 0.0025 --bp rover --opt mnlp --log_name cifar_leaky_relu_6_500_ROVER_MNLP
25+
python3 exp.py --net_file_path ../nets/leaky_relu/cifar_leaky_relu_convsmall.onnx --dataset cifar10 --perturbation_radius 0.004 --bp rover --opt mnlp --log_name cifar_leaky_relu_convsmall_ROVER_MNLP
26+
python3 exp.py --net_file_path ../nets/leaky_relu/cifar_leaky_relu_convmed.onnx --dataset cifar10 --perturbation_radius 0.004 --bp rover --opt mnlp --log_name cifar_leaky_relu_convmed_ROVER_MNLP
27+
python3 exp.py --net_file_path ../nets/elu/cifar_elu_3_500.onnx --dataset cifar10 --perturbation_radius 0.003 --bp rover --opt mnlp --log_name cifar_elu_3_500_ROVER_MNLP
28+
python3 exp.py --net_file_path ../nets/elu/cifar_elu_6_500.onnx --dataset cifar10 --perturbation_radius 0.003 --bp rover --opt mnlp --log_name cifar_elu_6_500_ROVER_MNLP
29+
python3 exp.py --net_file_path ../nets/elu/cifar_elu_convsmall.onnx --dataset cifar10 --perturbation_radius 0.0035 --bp rover --opt mnlp --log_name cifar_elu_convsmall_ROVER_MNLP
30+
python3 exp.py --net_file_path ../nets/elu/cifar_elu_convmed.onnx --dataset cifar10 --perturbation_radius 0.0035 --bp rover --opt mnlp --log_name cifar_elu_convmed_ROVER_MNLP

‎evaluation_verification/cifar10_sshape.sh‎

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
#!/bin/bash
2+
3+
python3 exp.py --net_file_path ../nets/sigmoid/cifar_sigmoid_3_500.onnx --dataset cifar10 --perturbation_radius 0.004 --bp crown --opt mnlp --log_name cifar_sigmoid_3_500_CROWN_MNLP
4+
python3 exp.py --net_file_path ../nets/sigmoid/cifar_sigmoid_6_500.onnx --dataset cifar10 --perturbation_radius 0.003 --bp crown --opt mnlp --log_name cifar_sigmoid_6_500_CROWN_MNLP
5+
python3 exp.py --net_file_path ../nets/sigmoid/cifar_sigmoid_convsmall.onnx --dataset cifar10 --perturbation_radius 0.004 --bp crown --opt mnlp --log_name cifar_sigmoid_convsmall_CROWN_MNLP
6+
python3 exp.py --net_file_path ../nets/sigmoid/cifar_sigmoid_convmed.onnx --dataset cifar10 --perturbation_radius 0.01 --bp crown --opt mnlp --log_name cifar_sigmoid_convmed_CROWN_MNLP
7+
8+
9+
python3 exp.py --net_file_path ../nets/sigmoid/cifar_sigmoid_3_500.onnx --dataset cifar10 --perturbation_radius 0.004 --bp deeppoly --opt mnlp --log_name cifar_sigmoid_3_500_DEEPPOLY_MNLP
10+
python3 exp.py --net_file_path ../nets/sigmoid/cifar_sigmoid_6_500.onnx --dataset cifar10 --perturbation_radius 0.003 --bp deeppoly --opt mnlp --log_name cifar_sigmoid_6_500_DEEPPOLY_MNLP
11+
python3 exp.py --net_file_path ../nets/sigmoid/cifar_sigmoid_convsmall.onnx --dataset cifar10 --perturbation_radius 0.004 --bp deeppoly --opt mnlp --log_name cifar_sigmoid_convsmall_DEEPPOLY_MNLP
12+
python3 exp.py --net_file_path ../nets/sigmoid/cifar_sigmoid_convmed.onnx --dataset cifar10 --perturbation_radius 0.01 --bp deeppoly --opt mnlp --log_name cifar_sigmoid_convmed_DEEPPOLY_MNLP
13+
14+
15+
python3 exp.py --net_file_path ../nets/tanh/cifar_tanh_3_500.onnx --dataset cifar10 --perturbation_radius 0.0015 --bp crown --opt mnlp --log_name cifar_tanh_3_500_CROWN_MNLP
16+
python3 exp.py --net_file_path ../nets/tanh/cifar_tanh_6_500.onnx --dataset cifar10 --perturbation_radius 0.0020 --bp crown --opt mnlp --log_name cifar_tanh_6_500_CROWN_MNLP
17+
python3 exp.py --net_file_path ../nets/tanh/cifar_tanh_convsmall.onnx --dataset cifar10 --perturbation_radius 0.0015 --bp crown --opt mnlp --log_name cifar_tanh_convsmall_CROWN_MNLP
18+
python3 exp.py --net_file_path ../nets/tanh/cifar_tanh_convmed.onnx --dataset cifar10 --perturbation_radius 0.0025 --bp crown --opt mnlp --log_name cifar_tanh_convmed_CROWN_MNLP
19+
20+
21+
python3 exp.py --net_file_path ../nets/tanh/cifar_tanh_3_500.onnx --dataset cifar10 --perturbation_radius 0.0015 --bp deeppoly --opt mnlp --log_name cifar_tanh_3_500_DEEPPOLY_MNLP
22+
python3 exp.py --net_file_path ../nets/tanh/cifar_tanh_6_500.onnx --dataset cifar10 --perturbation_radius 0.0020 --bp deeppoly --opt mnlp --log_name cifar_tanh_6_500_DEEPPOLY_MNLP
23+
python3 exp.py --net_file_path ../nets/tanh/cifar_tanh_convsmall.onnx --dataset cifar10 --perturbation_radius 0.0015 --bp deeppoly --opt mnlp --log_name cifar_tanh_convsmall_DEEPPOLY_MNLP
24+
python3 exp.py --net_file_path ../nets/tanh/cifar_tanh_convmed.onnx --dataset cifar10 --perturbation_radius 0.0025 --bp deeppoly --opt mnlp --log_name cifar_tanh_convmed_DEEPPOLY_MNLP
25+

‎evaluation_verification/exp.py‎

Lines changed: 71 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,71 @@
1+
"""
2+
This script is used to run the verification experiment for a neural network model.
3+
"""
4+
import argparse
5+
import sys
6+
7+
sys.path.append("../")
8+
9+
import logging
10+
from datetime import datetime
11+
12+
from src import *
13+
14+
15+
def main(args):
16+
17+
if args.bp == "deeppoly":
18+
bp_method = ActRelaxationMode.DEEPPOLY
19+
elif args.bp == "crown":
20+
bp_method = ActRelaxationMode.CROWN
21+
elif args.bp == "rover":
22+
bp_method = ActRelaxationMode.ROVER_SN
23+
else:
24+
raise ValueError(f"Invalid BP method: {args.bp}")
25+
26+
if args.opt == "lp":
27+
opt_method = OptimizationMethod.LP
28+
elif args.opt == "mnlp":
29+
opt_method = OptimizationMethod.MNLP
30+
else:
31+
opt_method = None
32+
33+
log_file = None
34+
if args.log_name is not None:
35+
current_time = datetime.now().strftime("%Y%m%d%H%M%S")
36+
log_file = args.log_name
37+
log_file = "./logs/" + log_file + f"_{current_time}.log"
38+
39+
# Set arguments for the model.
40+
args = Arguments(
41+
net_file_path=args.net_file_path,
42+
dataset=args.dataset,
43+
perturbation_radius=args.perturbation_radius,
44+
log_level=logging.INFO,
45+
act_relax_mode=bp_method,
46+
log_file=log_file,
47+
optimization_method=opt_method,
48+
first_sample_index=0,
49+
num_samples=100,
50+
device="cuda:0", # noqa
51+
dtype="float64",
52+
)
53+
54+
# Create the model and verify it.
55+
model = ModelFactory(args)
56+
model.prepare()
57+
model.build()
58+
model.verify()
59+
60+
61+
if __name__ == "__main__":
62+
parser = argparse.ArgumentParser(description="Run the experiment.")
63+
parser.add_argument("--net_file_path", type=str, default=None)
64+
parser.add_argument("--dataset", type=str, default="mnist")
65+
parser.add_argument("--perturbation_radius", type=float, default=0.01)
66+
parser.add_argument("--bp", type=str, default=None)
67+
parser.add_argument("--opt", type=str, default=None)
68+
parser.add_argument("--log_name", type=str, default=None)
69+
70+
args = parser.parse_args()
71+
main(args)

‎evaluation_verification/exp_test.py‎

Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,39 @@
1+
"""
2+
This script is used to test the verification of a neural network model by one small
3+
sample.
4+
"""
5+
import sys
6+
7+
sys.path.append("../")
8+
9+
import logging
10+
from datetime import datetime
11+
12+
from src import *
13+
14+
15+
def main():
16+
current_time = datetime.now().strftime("%Y%m%d%H%M%S")
17+
# Set arguments for the model.
18+
args = Arguments(
19+
net_file_path="../nets/leaky_relu/mnist_leaky_relu_3_500.onnx",
20+
dataset="mnist",
21+
perturbation_radius=0.03,
22+
log_level=logging.INFO,
23+
act_relax_mode=ActRelaxationMode.CROWN,
24+
optimization_method=OptimizationMethod.MNLP,
25+
first_sample_index=0,
26+
num_samples=1,
27+
device="cuda:0",
28+
dtype="float64",
29+
)
30+
31+
# Create the model and verify it.
32+
model = ModelFactory(args)
33+
model.prepare()
34+
model.build()
35+
model.verify()
36+
37+
38+
if __name__ == "__main__":
39+
main()

‎evaluation_verification/maxpool.sh‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
#!/bin/bash
2+
3+
python3 exp.py --net_file_path ../nets/maxpool/mnist_conv_maxpool.onnx --dataset mnist --perturbation_radius 0.01 --bp crown --opt mnlp --log_name mnist_conv_maxpool_CROWN_MNLP
4+
python3 exp.py --net_file_path ../nets/maxpool/mnist_conv_maxpool.onnx --dataset mnist --perturbation_radius 0.01 --bp deeppoly --opt mnlp --log_name mnist_conv_maxpool_DEEPPOLY_MNLP
5+
6+
7+
python3 exp.py --net_file_path ../nets/maxpool/cifar_conv_maxpool.onnx --dataset cifar10 --perturbation_radius 0.001 --bp crown --opt mnlp --log_name cifar_conv_maxpool_CROWN_MNLP
8+
python3 exp.py --net_file_path ../nets/maxpool/cifar_conv_maxpool.onnx --dataset cifar10 --perturbation_radius 0.001 --bp deeppoly --opt mnlp --log_name cifar_conv_maxpool_DEEPPOLY_MNLP

‎evaluation_verification/mnist_relulike.sh‎

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
#!/bin/bash
2+
3+
python3 exp.py --net_file_path ../nets/leaky_relu/mnist_leaky_relu_3_500.onnx --dataset mnist --perturbation_radius 0.03 --bp rover --log_name mnist_leaky_relu_3_500_ROVER
4+
python3 exp.py --net_file_path ../nets/leaky_relu/mnist_leaky_relu_6_500.onnx --dataset mnist --perturbation_radius 0.015 --bp rover --log_name mnist_leaky_relu_6_500_ROVER
5+
python3 exp.py --net_file_path ../nets/leaky_relu/mnist_leaky_relu_convsmall.onnx --dataset mnist --perturbation_radius 0.02 --bp rover --log_name mnist_leaky_relu_convsmall_ROVER
6+
python3 exp.py --net_file_path ../nets/leaky_relu/mnist_leaky_relu_convmed.onnx --dataset mnist --perturbation_radius 0.02 --bp rover --log_name mnist_leaky_relu_convmed_ROVER
7+
python3 exp.py --net_file_path ../nets/elu/mnist_elu_3_500.onnx --dataset mnist --perturbation_radius 0.03 --bp rover --log_name mnist_elu_3_500_ROVER
8+
python3 exp.py --net_file_path ../nets/elu/mnist_elu_6_500.onnx --dataset mnist --perturbation_radius 0.02 --bp rover --log_name mnist_elu_6_500_ROVER
9+
python3 exp.py --net_file_path ../nets/elu/mnist_elu_convsmall.onnx --dataset mnist --perturbation_radius 0.03 --bp rover --log_name mnist_elu_convsmall_ROVER
10+
python3 exp.py --net_file_path ../nets/elu/mnist_elu_convmed.onnx --dataset mnist --perturbation_radius 0.03 --bp rover --log_name mnist_elu_convmed_ROVER
11+
12+
13+
python3 exp.py --net_file_path ../nets/leaky_relu/mnist_leaky_relu_3_500.onnx --dataset mnist --perturbation_radius 0.03 --bp rover --opt lp --log_name mnist_leaky_relu_3_500_ROVER_LP
14+
python3 exp.py --net_file_path ../nets/leaky_relu/mnist_leaky_relu_6_500.onnx --dataset mnist --perturbation_radius 0.015 --bp rover --opt lp --log_name mnist_leaky_relu_6_500_ROVER_LP
15+
python3 exp.py --net_file_path ../nets/leaky_relu/mnist_leaky_relu_convsmall.onnx --dataset mnist --perturbation_radius 0.02 --bp rover --opt lp --log_name mnist_leaky_relu_convsmall_ROVER_LP
16+
python3 exp.py --net_file_path ../nets/leaky_relu/mnist_leaky_relu_convmed.onnx --dataset mnist --perturbation_radius 0.02 --bp rover --opt lp --log_name mnist_leaky_relu_convmed_ROVER_LP
17+
python3 exp.py --net_file_path ../nets/elu/mnist_elu_3_500.onnx --dataset mnist --perturbation_radius 0.03 --bp rover --opt lp --log_name mnist_elu_3_500_ROVER_LP
18+
python3 exp.py --net_file_path ../nets/elu/mnist_elu_6_500.onnx --dataset mnist --perturbation_radius 0.02 --bp rover --opt lp --log_name mnist_elu_6_500_ROVER_LP
19+
python3 exp.py --net_file_path ../nets/elu/mnist_elu_convsmall.onnx --dataset mnist --perturbation_radius 0.03 --bp rover --opt lp --log_name mnist_elu_convsmall_ROVER_LP
20+
python3 exp.py --net_file_path ../nets/elu/mnist_elu_convmed.onnx --dataset mnist --perturbation_radius 0.03 --bp rover --opt lp --log_name mnist_elu_convmed_ROVER_LP
21+
22+
23+
python3 exp.py --net_file_path ../nets/leaky_relu/mnist_leaky_relu_3_500.onnx --dataset mnist --perturbation_radius 0.03 --bp rover --opt mnlp --log_name mnist_leaky_relu_3_500_ROVER_MNLP
24+
python3 exp.py --net_file_path ../nets/leaky_relu/mnist_leaky_relu_6_500.onnx --dataset mnist --perturbation_radius 0.015 --bp rover --opt mnlp --log_name mnist_leaky_relu_6_500_ROVER_MNLP
25+
python3 exp.py --net_file_path ../nets/leaky_relu/mnist_leaky_relu_convsmall.onnx --dataset mnist --perturbation_radius 0.02 --bp rover --opt mnlp --log_name mnist_leaky_relu_convsmall_ROVER_MNLP
26+
python3 exp.py --net_file_path ../nets/leaky_relu/mnist_leaky_relu_convmed.onnx --dataset mnist --perturbation_radius 0.02 --bp rover --opt mnlp --log_name mnist_leaky_relu_convmed_ROVER_MNLP
27+
python3 exp.py --net_file_path ../nets/elu/mnist_elu_3_500.onnx --dataset mnist --perturbation_radius 0.03 --bp rover --opt mnlp --log_name mnist_elu_3_500_ROVER_MNLP
28+
python3 exp.py --net_file_path ../nets/elu/mnist_elu_6_500.onnx --dataset mnist --perturbation_radius 0.02 --bp rover --opt mnlp --log_name mnist_elu_6_500_ROVER_MNLP
29+
python3 exp.py --net_file_path ../nets/elu/mnist_elu_convmed.onnx --dataset mnist --perturbation_radius 0.03 --bp rover --opt mnlp --log_name mnist_elu_convmed_ROVER_MNLP
30+
python3 exp.py --net_file_path ../nets/elu/mnist_elu_convsmall.onnx --dataset mnist --perturbation_radius 0.03 --bp rover --opt mnlp --log_name mnist_elu_convsmall_ROVER_MNLP

0 commit comments

Comments
 (0)