Skip to content

Commit 57f105d

Browse files
XeratecXeratec
authored
Fix Debug Print Topology Pass and Code Transformation (pulp-platform#75)
* Fix `DebugPrint` topology pass * Fix `PrintInput` code transformations * Update Changelog
1 parent 0cdd680 commit 57f105d

7 files changed

Lines changed: 310 additions & 43 deletions

File tree

‎.github/workflows/CI.yml‎

Lines changed: 37 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
name: CI
22

3-
on:
3+
on:
44
push:
55
pull_request:
66
workflow_dispatch:
@@ -33,8 +33,8 @@ jobs:
3333
submodules: recursive
3434
- name: Build Deeploy
3535
run: pip install -e .
36-
37-
36+
37+
3838
### Generic Tests ###
3939
generic-kernels:
4040
uses: ./.github/workflows/TestRunnerGeneric.yml
@@ -76,7 +76,7 @@ jobs:
7676
Quant
7777
Dequant
7878
QuantizedLinear
79-
79+
8080
8181
generic-models:
8282
uses: ./.github/workflows/TestRunnerGeneric.yml
@@ -94,10 +94,10 @@ jobs:
9494
miniMobileNet
9595
miniMobileNetv2
9696
CCT/CCT_1_16_16_8
97-
97+
9898
9999
### CortexM Tests ###
100-
cortexm-kernels:
100+
cortexm-kernels:
101101
uses: ./.github/workflows/TestRunnerCortexM.yml
102102
needs: select-docker-image
103103
with:
@@ -115,14 +115,14 @@ jobs:
115115
testReduceMean
116116
testSlice
117117
118-
cortexm-models:
118+
cortexm-models:
119119
uses: ./.github/workflows/TestRunnerCortexM.yml
120120
needs: select-docker-image
121121
with:
122122
docker-image: ${{ needs.select-docker-image.outputs.image }}
123123
test-names: |
124124
simpleRegression
125-
WaveFormer
125+
WaveFormer
126126
127127
### Snitch Tests ###
128128
snitch-kernels:
@@ -453,12 +453,12 @@ jobs:
453453
]
454454
num-cores: 8
455455
double-buffer: true
456-
456+
457457
siracusa-models-tiled-singlebuffer-L2:
458458
strategy:
459459
fail-fast: false
460460
matrix:
461-
test-data:
461+
test-data:
462462
- name: "simpleRegression"
463463
L1: [45000, 30000, 15000]
464464
- name: "miniMobileNet"
@@ -497,7 +497,7 @@ jobs:
497497
strategy:
498498
fail-fast: false
499499
matrix:
500-
test-data:
500+
test-data:
501501
- name: "simpleRegression"
502502
L1: [45000, 30000, 16000] # SCHEREMO: 15000 leads to non-2d transfers in L3!
503503
- name: "miniMobileNet"
@@ -538,7 +538,7 @@ jobs:
538538
strategy:
539539
fail-fast: false
540540
matrix:
541-
test-data:
541+
test-data:
542542
- name: "simpleRegression"
543543
L1: [60000, 45000, 30000]
544544
- name: "miniMobileNet"
@@ -638,7 +638,7 @@ jobs:
638638
strategy:
639639
fail-fast: false
640640
matrix:
641-
test-data:
641+
test-data:
642642
- name: "miniMobileNet"
643643
L1: [2000] # LMACAN: 1000 leads to non-2d transfers in L3!
644644
- name: "Attention"
@@ -665,7 +665,7 @@ jobs:
665665
strategy:
666666
fail-fast: false
667667
matrix:
668-
test-data:
668+
test-data:
669669
- name: "miniMobileNet"
670670
L1: [2000] # LMACAN: 1000 leads to non-2d transfers in L3!
671671
- name: "Attention"
@@ -720,7 +720,7 @@ jobs:
720720
strategy:
721721
fail-fast: false
722722
matrix:
723-
test-data:
723+
test-data:
724724
- name: "miniMobileNet"
725725
L1: [2000] # LMACAN: 1000 leads to non-2d transfers in L3!
726726
- name: "Attention"
@@ -878,6 +878,26 @@ jobs:
878878
python testTypes.py
879879
shell: bash
880880

881+
deeploy-debug:
882+
runs-on: ubuntu-22.04
883+
needs: select-docker-image
884+
container:
885+
image: ${{ needs.select-docker-image.outputs.image }}
886+
steps:
887+
- name: Checkout Repo
888+
uses: actions/checkout@v4
889+
with:
890+
submodules: recursive
891+
- name: Build Deeploy
892+
run: pip install -e .
893+
- name: Run Test
894+
run: |
895+
cd DeeployTest
896+
python testPrintInputOutputTransformation.py -p Generic -t ./Tests/simpleRegression
897+
python testPrintInputOutputTransformation.py -p Siracusa -t ./Tests/simpleRegression
898+
python testDebugPrintPass.py -p Generic -t ./Tests/simpleRegression
899+
shell: bash
900+
881901
deeploy-regex-matching:
882902
runs-on: ubuntu-22.04
883903
needs: select-docker-image
@@ -925,11 +945,11 @@ jobs:
925945
shell: bash
926946
- name: Format Python Licenses
927947
run: |
928-
grep -Lr "SPDX-License-Identifier: Apache-2.0" --exclude-dir="toolchain" --exclude-dir="install" --exclude-dir=".git" . --exclude-dir="third_party" --exclude-dir="TEST_*" --exclude "run_clang_format.py" | grep ".*\.py$" || [[ $? == 1 ]]
948+
grep -Lr "SPDX-License-Identifier: Apache-2.0" --exclude-dir="toolchain" --exclude-dir="install" --exclude-dir=".git" . --exclude-dir="third_party" --exclude-dir="TEST_*" --exclude "run_clang_format.py" | grep ".*\.py$" || [[ $? == 1 ]]
929949
shell: bash
930950
- name: Format C Licenses
931951
run: |
932-
grep -Lr "SPDX-License-Identifier: Apache-2.0" --exclude-dir="toolchain" --exclude-dir="install" --exclude-dir=".git" . --exclude-dir="third_party" --exclude-dir="TEST_*" --exclude-dir="runtime" | grep ".*\.c$" || [[ $? == 1 ]]
952+
grep -Lr "SPDX-License-Identifier: Apache-2.0" --exclude-dir="toolchain" --exclude-dir="install" --exclude-dir=".git" . --exclude-dir="third_party" --exclude-dir="TEST_*" --exclude-dir="runtime" | grep ".*\.c$" || [[ $? == 1 ]]
933953
shell: bash
934954
- name: Format C Header Licenses
935955
run: |

‎CHANGELOG.md‎

Lines changed: 14 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,18 @@
22

33
## Unreleased
44

5+
## Fix Debug Print Topology Pass and Code Transformation
6+
7+
### Added
8+
- Test for the `DebugPrintTopologyPass`.
9+
- Test for `PrintInputGeneration`, `PrintOutputGeneration`, `MemoryAwarePrintInputGeneration`, `MemoryAwarePrintOutputGeneration`
10+
11+
### Changed
12+
- Refactored` IntrospectiveCodeTransformationMixIn` to allow extracting dynamic references to global variables
13+
14+
### Fixed
15+
- Fix `DebugPrint` topology pass
16+
- Fix `PrintInput` code transformations to work with global variables
517

618
## Bunch of fixes and changes
719

@@ -100,7 +112,7 @@
100112
- cmake flow for gvsoc
101113
- CI tests regarding Snitch run on GVSOC as well
102114

103-
### Changed
115+
### Changed
104116
- Add the RTL library to the snitch_cluster build process in the Makefile, required for GVSOC simulation
105117

106118

@@ -176,7 +188,7 @@ Change main.c to use OUTPUTTYPE instead of float
176188

177189
### Added
178190
- A visualization of the memory allocation solution generated by Deeploy at each level of memory. I use Plotpy to generate a static `html` file and save it to the `DeeployState` directory.
179-
- An initialization strategy for the variable in the tiling to randomize the variables related to the permutation matrix.
191+
- An initialization strategy for the variable in the tiling to randomize the variables related to the permutation matrix.
180192
- New interface to `testRunner_tiled_siracusa` to control the generation of the memory allocation visualization, the memory allocation strategy, and the search strategy.
181193
- Export a new docker container with `plotpy` as dependency.
182194

‎Deeploy/CommonExtensions/CodeTransformationPasses/IntrospectiveCodeTransformation.py‎

Lines changed: 44 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -32,7 +32,7 @@
3232
from mako.parsetree import Expression, TemplateNode
3333

3434
from Deeploy.AbstractDataTypes import Pointer, Struct
35-
from Deeploy.DeeployTypes import ExecutionBlock, NetworkContext, NodeTemplate, OperatorRepresentation
35+
from Deeploy.DeeployTypes import ExecutionBlock, NetworkContext, NodeTemplate, OperatorRepresentation, VariableBuffer
3636

3737
_NULL: str = "NULL"
3838

@@ -105,13 +105,15 @@ def fixupParseTree(parseTree: TemplateNode) -> TemplateNode:
105105
def extractDynamicReferences(self,
106106
ctxt: NetworkContext,
107107
executionBlock: ExecutionBlock = None,
108-
unrollStructs = False):
108+
unrollStructs = False,
109+
includeGobalReferences = False):
109110

110111
makoDynamicReferences = []
111112
for codeSnippet in executionBlock.codeSnippets:
112113
template, operatorRepresentation = codeSnippet.template, codeSnippet.operatorRepresentation
113114

114-
newRefs = self._extractDynamicExpressions(ctxt, operatorRepresentation, template, unrollStructs)
115+
newRefs = self._extractDynamicExpressions(ctxt, operatorRepresentation, template, unrollStructs,
116+
includeGobalReferences)
115117

116118
makoDynamicReferences += newRefs
117119

@@ -131,7 +133,8 @@ def _extractDynamicExpressions(self,
131133
ctxt: NetworkContext,
132134
operatorRepresentation: OperatorRepresentation,
133135
template: NodeTemplate,
134-
unrollStructs = False):
136+
unrollStructs = False,
137+
includeGobalReferences = False):
135138

136139
codeHash = hash(template.template._source)
137140

@@ -145,13 +148,20 @@ def _extractDynamicExpressions(self,
145148
# Filter parsing tree for expressions
146149
makoExpressions = [node.text for node in makoParseTree.nodes if type(node) == Expression]
147150

148-
# Filter expressions for variables contained in operatorRepresentation
149-
makoReferences = [
151+
# Filter expressions for local variables contained in operatorRepresentation
152+
makoLocalReferences = [
150153
node for node in makoExpressions
151154
if ((node in operatorRepresentation) and type(operatorRepresentation[node]) == str and (
152155
operatorRepresentation[node] in ctxt.localObjects.keys()))
153156
]
154157

158+
# Filter expressions for global variables contained in operatorRepresentation
159+
makoGlobalReferences = [
160+
node for node in makoExpressions
161+
if ((node in operatorRepresentation) and type(operatorRepresentation[node]) == str and (
162+
operatorRepresentation[node] in ctxt.globalObjects.keys()))
163+
]
164+
155165
def _unrollStructReferences(val) -> List[str]:
156166
# Unroll struct references
157167
structReferences = []
@@ -163,17 +173,35 @@ def _unrollStructReferences(val) -> List[str]:
163173
structReferences.append(val.value[key].referenceName)
164174
return structReferences
165175

166-
references = []
167-
structReferences = []
168-
for ref in makoReferences:
169-
references.append(operatorRepresentation[ref])
176+
# Unroll local struct references
177+
localReferences = []
178+
localStructReferences = []
179+
for ref in makoLocalReferences:
180+
localReferences.append(operatorRepresentation[ref])
170181
if unrollStructs:
171-
if (ctxt.is_local(operatorRepresentation[ref])
172-
or ctxt.is_global(operatorRepresentation[ref])) and hasattr(
173-
ctxt.lookup(operatorRepresentation[ref]), "structDict"):
174-
structReferences += _unrollStructReferences(ctxt.lookup(operatorRepresentation[ref]).structDict)
182+
if ctxt.is_local(operatorRepresentation[ref]) and hasattr(ctxt.lookup(operatorRepresentation[ref]),
183+
"structDict"):
184+
localStructReferences += _unrollStructReferences(
185+
ctxt.lookup(operatorRepresentation[ref]).structDict)
186+
187+
# Unroll global struct references
188+
globalReferences = []
189+
globalStructReferences = []
190+
for ref in makoGlobalReferences:
191+
globalReferences.append(operatorRepresentation[ref])
192+
if unrollStructs:
193+
if ctxt.is_global(operatorRepresentation[ref]) and hasattr(ctxt.lookup(operatorRepresentation[ref]),
194+
"structDict"):
195+
globalStructReferences += _unrollStructReferences(
196+
ctxt.lookup(operatorRepresentation[ref]).structDict)
175197

176198
# Filter for dynamically allocated tensors
199+
dynamicLocalReferences = [ref for ref in localReferences + localStructReferences if ctxt.lookup(ref)._deploy]
200+
dynamicGlobalReferences = [
201+
ref for ref in globalReferences + globalStructReferences if isinstance(ctxt.lookup(ref), VariableBuffer)
202+
]
177203

178-
dynamicReferences = [ref for ref in references + structReferences if (ctxt.lookup(ref)._deploy)]
179-
return dynamicReferences
204+
if includeGobalReferences:
205+
return dynamicLocalReferences + dynamicGlobalReferences
206+
else:
207+
return dynamicLocalReferences

‎Deeploy/CommonExtensions/CodeTransformationPasses/PrintInputs.py‎

Lines changed: 24 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -80,7 +80,10 @@ def apply(self,
8080
name: str,
8181
verbose: CodeGenVerbosity = _NoVerbosity) -> Tuple[NetworkContext, ExecutionBlock]:
8282

83-
references = self.extractDynamicReferences(ctxt, executionBlock, True)
83+
references = self.extractDynamicReferences(ctxt,
84+
executionBlock,
85+
unrollStructs = True,
86+
includeGobalReferences = True)
8487

8588
for ref in references:
8689
refDict = self._getRepDict(ctxt, ref, name)
@@ -120,7 +123,10 @@ def apply(self,
120123
name: str,
121124
verbose: CodeGenVerbosity = _NoVerbosity) -> Tuple[NetworkContext, ExecutionBlock]:
122125

123-
references = self.extractDynamicReferences(ctxt, executionBlock, True)
126+
references = self.extractDynamicReferences(ctxt,
127+
executionBlock,
128+
unrollStructs = True,
129+
includeGobalReferences = True)
124130

125131
filteredReferences = [ref for ref in references if self._matchesRegex(ctxt, ref)]
126132

@@ -158,7 +164,10 @@ def apply(self,
158164
name: str,
159165
verbose: CodeGenVerbosity = _NoVerbosity) -> Tuple[NetworkContext, ExecutionBlock]:
160166

161-
references = self.extractDynamicReferences(ctxt, executionBlock, True)
167+
references = self.extractDynamicReferences(ctxt,
168+
executionBlock,
169+
unrollStructs = True,
170+
includeGobalReferences = True)
162171

163172
for ref in references:
164173
rep = self._getRepDict(ctxt, ref, name)
@@ -176,7 +185,10 @@ def apply(self,
176185
name: str,
177186
verbose: CodeGenVerbosity = _NoVerbosity) -> Tuple[NetworkContext, ExecutionBlock]:
178187

179-
references = self.extractDynamicReferences(ctxt, executionBlock, True)
188+
references = self.extractDynamicReferences(ctxt,
189+
executionBlock,
190+
unrollStructs = True,
191+
includeGobalReferences = True)
180192

181193
filteredReferences = [ref for ref in references if self._matchesRegex(ctxt, ref)]
182194

@@ -205,7 +217,10 @@ def _getRepDict(self, ctxt: NetworkContext, ref: str, name: str):
205217
def apply(self, ctxt: NetworkContext, executionBlock: ExecutionBlock,
206218
name: str) -> Tuple[NetworkContext, ExecutionBlock]:
207219

208-
references = self.extractDynamicReferences(ctxt, executionBlock, True)
220+
references = self.extractDynamicReferences(ctxt,
221+
executionBlock,
222+
unrollStructs = True,
223+
includeGobalReferences = True)
209224

210225
for ref in references:
211226
rep = self._getRepDict(ctxt, ref, name)
@@ -223,7 +238,10 @@ def apply(self,
223238
name: str,
224239
verbose: CodeGenVerbosity = _NoVerbosity) -> Tuple[NetworkContext, ExecutionBlock]:
225240

226-
references = self.extractDynamicReferences(ctxt, executionBlock, True)
241+
references = self.extractDynamicReferences(ctxt,
242+
executionBlock,
243+
unrollStructs = True,
244+
includeGobalReferences = True)
227245

228246
filteredReferences = [ref for ref in references if self._matchesRegex(ctxt, ref)]
229247

‎Deeploy/Targets/Generic/Templates/DebugPrintTemplate.py‎

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -44,8 +44,7 @@ def alignToContext(self, ctxt: NetworkContext,
4444
operatorRepresentation['data_in_signed'] = data_in._signed
4545
operatorRepresentation['offset'] = (data_in._signed == 0) * int(data_in.nLevels / 2)
4646

47-
operatorRepresentation['output_name'] = ctxt._mangle("outputs") + "[0]" if ctxt.outputs(
48-
)[0].name == data_out.name else ctxt._mangle(data_out.name)
47+
operatorRepresentation['output_name'] = ctxt._mangle(data_out.name)
4948

5049
return ctxt, operatorRepresentation, []
5150

0 commit comments

Comments
 (0)