Skip to content

Commit b95c6c9

Browse files
HenryChang213HenryChang213
committed
init
0 parents  commit b95c6c9

142 files changed

Lines changed: 74216 additions & 0 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.gitignore‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
__pycache__/
2+
SlurmOutput/
3+
temp/
4+
.vscode/
5+
*.ncu-rep

‎.gitmodules‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
[submodule "cost_model/supply_chain"]
2+
path = cost_model/supply_chain
3+
url = https://github.com/PrincetonUniversity/ttm-cas

‎Dockerfile‎

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
# Start with a base image that includes Miniconda to manage our environment
2+
FROM continuumio/miniconda3
3+
4+
# Set the working directory in the container to /app
5+
WORKDIR /app
6+
7+
# Create the conda environment
8+
COPY environment.yml /app/environment.yml
9+
RUN conda env create -f /app/environment.yml
10+
11+
# Initialize conda in bash shell
12+
RUN echo "source activate llmcompass_ae" > ~/.bashrc
13+
ENV PATH /opt/conda/envs/llmcompass_ae/bin:$PATH
14+
15+
# Clone your GitHub repository
16+
RUN git clone https://github.com/HenryChang213/LLMCompass_ISCA_AE.git /app/LLMCompass_ISCA_AE
17+
RUN cd /app/LLMCompass_ISCA_AE && git submodule init && git submodule update --recursive
18+
19+
# Expose the port your app runs on
20+
EXPOSE 8000
21+
22+

‎LICENSE‎

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
BSD 3-Clause License
2+
3+
Copyright (c) 2024, Princeton University
4+
5+
Redistribution and use in source and binary forms, with or without
6+
modification, are permitted provided that the following conditions are met:
7+
8+
1. Redistributions of source code must retain the above copyright notice, this
9+
list of conditions and the following disclaimer.
10+
11+
2. Redistributions in binary form must reproduce the above copyright notice,
12+
this list of conditions and the following disclaimer in the documentation
13+
and/or other materials provided with the distribution.
14+
15+
3. Neither the name of the copyright holder nor the names of its
16+
contributors may be used to endorse or promote products derived from
17+
this software without specific prior written permission.
18+
19+
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
20+
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
21+
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
22+
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
23+
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
24+
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
25+
SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
26+
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
27+
OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
28+
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

‎LLMCompass_AE_Appendix.pdf‎

118 KB
Binary file not shown.

‎README.md‎

Lines changed: 83 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,83 @@
1+
[![DOI](https://zenodo.org/badge/779008229.svg)](https://zenodo.org/doi/10.5281/zenodo.10892431)
2+
3+
# LLMCompass
4+
5+
This repository provides the implementation of **LLMCompass** from the following papers:
6+
7+
**LLMCompass: Enabling Efficient Hardware Design for Large Language Model Inference**
8+
9+
*Hengrui Zhang, August Ning, Rohan Prabhakar, David Wentzlaff*
10+
11+
12+
## Set up the environment
13+
14+
```
15+
$ conda create -n llmcompass_ae python=3.9
16+
$ conda activate llmcompass_ae
17+
$ pip3 install scalesim
18+
$ conda install pytorch==2.0.0 -c pytorch
19+
$ pip3 install matplotlib
20+
$ pip3 install seaborn
21+
$ pip3 install scipy
22+
```
23+
24+
## Installation
25+
26+
### If using Github
27+
```
28+
$ git clone https://github.com/HenryChang213/LLMCompass_ISCA_AE.git
29+
$ cd LLMCompass_ISCA_AE
30+
$ git submodule init
31+
$ git submodule update --recursive
32+
```
33+
34+
### If using Zenodo
35+
Unzip the file and download from https://github.com/PrincetonUniversity/ttm-cas.git as `cost_model\supply_chain`
36+
37+
38+
### If using Docker
39+
A Dockerfile has been provided (`./Dockerfile`), including all the software dependencies and the LLMCompass source code.
40+
41+
A docker image has been provided [here](https://github.com/HenryChang213/LLMCompass_ISCA_AE_docker).
42+
43+
## Experiment workflow
44+
```
45+
# Figure 5 (around 100 min)
46+
$ cd ae/figure5
47+
$ bash run_figure5.sh
48+
49+
# Figure 6 (around 1 min)
50+
$ cd ae/figure6
51+
$ bash run_figure6.sh
52+
53+
# Figure 7 (around 20 min)
54+
$ cd ae/figure7
55+
$ bash run_figure7.sh
56+
57+
# Figure 8 (around 40 min)
58+
$ cd ae/figure8
59+
$ bash run_figure8.sh
60+
61+
# Figure 9 (around 30 min)
62+
$ cd ae/figure9
63+
$ bash run_figure9.sh
64+
65+
# Figure 10 (around 45 min)
66+
$ cd ae/figure10
67+
$ bash run_figure10.sh
68+
69+
# Figure 11 (around 5 min)
70+
$ cd ae/figure11
71+
$ bash run_figure11.sh
72+
73+
# Figure 12 (around 4 hours)
74+
$ cd ae/figure12
75+
$ bash run_figure12.sh
76+
```
77+
78+
## Expected result
79+
80+
After running each script above, the corresponding figures
81+
will be generated under the corresponding directory as suggested by its name.
82+
83+
For comparison, a copy of the expected results can be found in `ae\expected_results`

‎__init__.py‎

Whitespace-only changes.

‎ae/.gitignore‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
*.pdf
2+
*.csv
3+
!**/real_hardware/**/*.csv
4+
!expected_results/*

‎ae/__init__.py‎

Whitespace-only changes.

‎ae/expected_results/figure10.pdf‎

25.1 KB
Binary file not shown.

0 commit comments

Comments
 (0)