Skip to content

Commit 38477b4

Browse files
author
Welt Xing
committed
Update example
1 parent adf5ff5 commit 38477b4

21 files changed

Lines changed: 1182 additions & 344 deletions

MANIFEST.in

Lines changed: 0 additions & 2 deletions
This file was deleted.

README.md

Lines changed: 15 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,7 @@
1616
**2025.8.12**: 实现了纯推理的llama3 (6-layer Transformer, vocab-size=32000). 参考了[这里](https://github.com/likejazz/llama3.np)的NumPy实现和数据集. 将数据集下载到`llama`文件夹即可运行:
1717

1818
```bash
19-
>>> python .\llama\infer.py
19+
>>> python -m llama.infer
2020
There was a boy named Timmy. He loved to play with hi toy and run around outside. One day, Timmy' mom asked him to help her with the laundry. Timmy didn't want to help because he wanted to play. But hi mom said, "Timmy, you need to help me. It' important to help out."
2121
Timmy didn't want to help, but he knew he had to. So, he put on hi shoe and went outside to help hi mom. A they were folding the clothe, Timmy saw a big pile of laundry on the floor. He wanted to help, so he started to pick it up. But then, he accidentally knocked over a pile of clothe and they fell on him. Timmy wa okay, but he felt bad.
2222
Hi mom saw what happened and said, "Timmy, you need to be more careful. You could have hurt yourself." Timmy felt bad and said sorry. Hi mom hugged him and said, "It' okay, accident happen. Let' clean up the laundry together." Timmy learned that it' important to be careful and help out when you need it.
@@ -54,58 +54,58 @@ python setup.py install
5454

5555
## Example
5656

57-
[examples](./examples/)中是一些例子运行`python examples/XXX.py`即可:
57+
[examples/pydynet](./examples/pydynet)中是一些例子[examples/pytorch](./examples/pytorch)给出等价的pytorch实现. 运行`python examples.pydynet.xxx`即可:
5858

5959
### AutoDiff
6060

61-
[autodiff1d.py](examples/autodiff1d.py)利用自动微分,对一个一维凸函数进行梯度下降:
61+
[autodiff1d.py](examples/pydynet/autodiff1d.py)利用自动微分,对一个一维凸函数进行梯度下降:
6262

6363
<img src="imgs/ad1d.png" alt="ad1" style="zoom:67%;" />
6464

65-
以及一个多元凸函数的例子: [autodiff2d.py](examples/autodiff2d.py)
65+
以及一个多元凸函数的例子: [autodiff2d.py](examples/pydynet/autodiff2d.py)
6666

6767
<img src="imgs/ad2d.png" alt="ad2" style="zoom:67%;" />
6868

6969
### MLP & LeNet
7070

71-
[mlp_cnn.py](examples/mlp_cnn.py)使用全连接网络(三层+残差)和LeNet对MNIST进行分类. 训练准确率和测试准确率:
71+
[mlp_cnn.py](examples/pydynet/mnist.py)使用MLP和LeNet对MNIST进行分类. 训练准确率和测试准确率:
7272

7373
<img src="imgs/mlp_cnn.png" alt="dnn" style="zoom:67%;" />
7474

7575
### Dropout & BN
7676

77-
[mlp_dropout_bn.py](examples/mlp_dropout_bn.py)使用三种网络对`fetch_olivetti_faces`人脸(64×64)数据集进行分类并进行性能对比:
77+
[mlp_dropout_bn.py](examples/pydynet/dropout_bn.py)使用三种网络对`fetch_olivetti_faces`人脸(64×64)数据集进行分类并进行性能对比:
7878

7979
1. 三层MLP;
8080
2. 三层MLP + Dropout;
8181
3. 三层MLP + BatchNormalization.
8282

8383
学习效果对比:
8484

85-
<img src="imgs/dropout_BN.png" alt="cnn" style="zoom:67%;" />
85+
<img src="imgs/dropout_bn.png" alt="cnn" style="zoom:67%;" />
8686

8787
### RNN
8888

89-
[rnn_sin.py](examples/ts_prediction.py)中是一个用GRU做时序预测例子:
89+
[rnn_sin.py](examples/pydynet/ts_prediction.py)中是一个用GRU做时序预测例子:
9090

9191
<img src="imgs/rnn.png" alt="RNN" style="zoom:67%;" />
9292

9393
### Transformer
9494

95-
[transformer.py](examples/transformer.py)中是一个用Transformer训练文本分类模型的例子. 训练结果:
95+
[transformer.py](examples/pydynet/transformer.py)中是一个用Transformer训练文本分类模型的例子. 训练结果:
9696

9797
<img src="imgs/transformer.png" alt="transformer" style="zoom:67%;" />
9898

9999
> 数据集 (CoLA) 链接: <https://nyu-mll.github.io/CoLA/cola_public_1.1.zip>
100100
101101
## cuda加速
102102

103-
在训练batch size为128, 测试batch size为512情况下,模型在CPU和GPU上的训练速度比较:
103+
在训练batch size为256, 测试batch size为1024情况下,模型在CPU和GPU上的训练速度比较:
104104

105-
| Net | Dataset | CPU time (s) per Epoch | GPU time (s) per Epoch |
105+
| Network structure | Dataset | CPU time (s) per epoch | GPU time (s) per epoch |
106106
| :-----------------: | :---------------: | :--------------------: | :--------------------: |
107-
| ResidualMLP | MNIST (80000×574) | 20.256±0.138 | 2.903±.018 |
108-
| LeNet | MNIST (80000×574) | 239.664±2.108 | 10.148±0.026 |
109-
| 1-layer Transformer | CoLA (8551×45×64) | 17.503±0.251 | 1.125±0.002 |
107+
| 3-layer MLP | MNIST (80000×574) | 7.256±0.138 | 1.203±.0181 |
108+
| LeNet | MNIST (80000×574) | 239.664±2.108 | 2.841±0.026 |
109+
| 1-layer Transformer (dim=512, head=4) | CoLA (8551×45×64) | 17.503±0.251 | 1.075±0.002 |
110110

111-
设备: Nvidia GeForce RTX 3090.
111+
设备: Nvidia GeForce RTX 4090.

examples/autograd1d.py

Lines changed: 0 additions & 95 deletions
This file was deleted.

examples/mlp_cnn.py

Lines changed: 0 additions & 180 deletions
This file was deleted.

0 commit comments

Comments
 (0)