Skip to content

Commit 20d1312

Browse files
jbluehjblueh
committed
Merge branch 'develop'
Extend OpDiLib instrumentation. - instrumentation of master constructs - instrumentation of passive barriers - instrumentation of passive parallel regions - revise and extend reverse pass instrumentation Balance the tape activity calls received by thread 0.
2 parents 9686235 + 09ac56b commit 20d1312

14 files changed

Lines changed: 339 additions & 33 deletions

‎README.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,7 @@ OpDiLib supports all directives, clauses and runtime functions of the OpenMP 2.5
1919

2020
If you have a code that is differentiated with a serial AD tool and parallelize it using OpenMP, the procedure of obtaining an efficient parallel differentiated code with OpDiLib is as follows.
2121

22-
1. **Couple OpDiLib with your AD tool.** This step can be skipped if you use an AD tool that already has OpDiLib bindings, for example [CoDiPack](https://www.scicomp.uni-kl.de/software/codi/). OpDiLib support is available in CoDiPack's [develop branch](https://github.com/scicompkl/codipack/tree/develop).
22+
1. **Couple OpDiLib with your AD tool.** This step can be skipped if you use an AD tool that already has OpDiLib bindings, for example [CoDiPack](https://www.scicomp.uni-kl.de/software/codi/), which has OpDiLib support since [version 2.1](https://github.com/SciCompKL/CoDiPack/releases/tag/v2.1.0).
2323
2. **Obtain a first parallel differentiated version of your code.** If your compiler supports OMPT, it suffices to add a few lines of code for the initialization and finalization of OpDiLib. Otherwise, you have to use OpDiLib's macro backend, which involves rewriting your OpenMP constructs according to OpDiLib's macro interface. Both approaches are demonstrated in the minimal example below.
2424
3. **Optimize the performance of the parallel reverse pass.** Check your parallel forward code for parts that do not involve shared reading. Use OpDiLib's adjoint access control tools to disable atomic adjoints for these parts. You may also revise your data access patterns to eliminate additional instances of shared reading.
2525

@@ -50,7 +50,7 @@ If you use OpDiLib in one of your applications and write a paper, please cite us
5050

5151
## Minimal Example
5252

53-
The following minimal example assumes that [CoDiPack](https://www.scicomp.uni-kl.de/software/codi/) is used as the underlying AD tool. Specifically, you need CoDiPack's [develop branch](https://github.com/scicompkl/codipack/tree/develop). For additional examples, please refer to OpDiLib's test suite.
53+
The following minimal example assumes that [CoDiPack](https://www.scicomp.uni-kl.de/software/codi/) is used as the underlying AD tool. You need CoDiPack [version 2.1](https://github.com/SciCompKL/CoDiPack/releases/tag/v2.1.0) or newer. For additional examples, please refer to OpDiLib's test suite.
5454

5555
### OMPT Backend
5656

‎include/opdi/backend/macro/macros.hpp‎

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -140,9 +140,13 @@
140140
#define OPDI_END_SECTION
141141

142142
#define OPDI_MASTER(...) \
143-
OPDI_PRAGMA(omp master __VA_ARGS__)
143+
OPDI_PRAGMA(omp master __VA_ARGS__) \
144+
{ \
145+
opdi::logic->onMaster(opdi::LogicInterface::ScopeEndpoint::Begin);
144146

145-
#define OPDI_END_MASTER
147+
#define OPDI_END_MASTER \
148+
opdi::logic->onMaster(opdi::LogicInterface::ScopeEndpoint::End); \
149+
}
146150

147151
// standalone macros
148152

‎include/opdi/backend/ompt/masterCallbacks.hpp‎

Lines changed: 77 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,77 @@
1+
/*
2+
* OpDiLib, an Open Multiprocessing Differentiation Library
3+
*
4+
* Copyright (C) 2020-2022 Chair for Scientific Computing (SciComp), TU Kaiserslautern
5+
* Copyright (C) 2023 Chair for Scientific Computing (SciComp), University of Kaiserslautern-Landau
6+
* Homepage: http://www.scicomp.uni-kl.de
7+
* Contact: Prof. Nicolas R. Gauger (opdi@scicomp.uni-kl.de)
8+
*
9+
* Lead developer: Johannes Blühdorn (SciComp, University of Kaiserslautern-Landau)
10+
*
11+
* This file is part of OpDiLib (http://www.scicomp.uni-kl.de/software/opdi).
12+
*
13+
* OpDiLib is free software: you can redistribute it and/or
14+
* modify it under the terms of the GNU General Public License
15+
* as published by the Free Software Foundation, either version 3 of the
16+
* License, or (at your option) any later version.
17+
*
18+
* OpDiLib is distributed in the hope that it will be useful,
19+
* but WITHOUT ANY WARRANTY; without even the implied warranty
20+
* of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
21+
*
22+
* See the GNU General Public License for more details.
23+
* You should have received a copy of the GNU
24+
* General Public License along with OpDiLib.
25+
* If not, see <http://www.gnu.org/licenses/>.
26+
*
27+
* For other licensing options please contact us.
28+
*
29+
*/
30+
31+
#pragma once
32+
33+
#include "../../helpers/exceptions.hpp"
34+
#include "../../helpers/macros.hpp"
35+
#include "../../logic/logicInterface.hpp"
36+
37+
#include "callbacksBase.hpp"
38+
39+
namespace opdi {
40+
41+
struct MasterCallbacks : public virtual CallbacksBase {
42+
43+
private:
44+
45+
static void onMaster(ompt_scope_endpoint_t _endpoint,
46+
ompt_data_t* parallelData,
47+
ompt_data_t* taskData,
48+
void const* codeptr) {
49+
OPDI_UNUSED(parallelData);
50+
OPDI_UNUSED(taskData);
51+
OPDI_UNUSED(codeptr);
52+
53+
LogicInterface::ScopeEndpoint endpoint;
54+
if (ompt_scope_begin == _endpoint) {
55+
endpoint = LogicInterface::ScopeEndpoint::Begin;
56+
}
57+
else {
58+
endpoint = LogicInterface::ScopeEndpoint::End;
59+
}
60+
61+
logic->onMaster(endpoint);
62+
}
63+
64+
protected:
65+
66+
static void init() {
67+
OPDI_CHECK_ERROR(CallbacksBase::registerCallback(ompt_callback_master,
68+
(ompt_callback_t) MasterCallbacks::onMaster));
69+
}
70+
71+
static void finalize() {
72+
OPDI_CHECK_ERROR(CallbacksBase::clearCallback(ompt_callback_master));
73+
}
74+
75+
};
76+
77+
}

‎include/opdi/backend/ompt/omptBackend.hpp‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -48,6 +48,7 @@
4848

4949
#include "implicitTaskCallbacks.hpp"
5050
#include "macros.hpp"
51+
#include "masterCallbacks.hpp"
5152
#include "mutexCallbacks.hpp"
5253
#include "parallelCallbacks.hpp"
5354
#include "reductionCallbacks.hpp"
@@ -58,6 +59,7 @@
5859
namespace opdi {
5960

6061
struct OmptBackend : public ImplicitTaskCallbacks,
62+
public MasterCallbacks,
6163
public MutexCallbacks,
6264
public ParallelCallbacks,
6365
public ReductionCallbacks,
@@ -105,6 +107,7 @@ namespace opdi {
105107
SyncRegionCallbacks::init();
106108
MutexCallbacks::init();
107109
ReductionCallbacks::init();
110+
MasterCallbacks::init();
108111

109112
return 1; // success
110113
}
@@ -114,6 +117,7 @@ namespace opdi {
114117
OPDI_UNUSED(toolData);
115118

116119
// finalize callback structures
120+
MasterCallbacks::finalize();
117121
ReductionCallbacks::finalize();
118122
MutexCallbacks::finalize();
119123
SyncRegionCallbacks::finalize();

‎include/opdi/logic/logicInterface.hpp‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -30,6 +30,8 @@
3030

3131
#pragma once
3232

33+
#include <cstdlib>
34+
3335
namespace opdi {
3436

3537
struct LogicInterface
@@ -71,6 +73,8 @@ namespace opdi {
7173

7274
virtual void onWork(WorksharingKind kind, ScopeEndpoint endpoint) = 0;
7375

76+
virtual void onMaster(ScopeEndpoint endpoint) = 0;
77+
7478
virtual void onSyncRegion(SyncRegionKind kind, ScopeEndpoint endpoint) = 0;
7579

7680
virtual void init() = 0;

‎include/opdi/logic/omp/implicitTaskOmpLogic.cpp‎

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -48,7 +48,7 @@ void* opdi::ImplicitTaskOmpLogic::onImplicitTaskBegin(int actualParallelism, int
4848

4949
ParallelData* parallelData = (ParallelData*) parallelDataPtr;
5050

51-
if (parallelData != nullptr && tool->isActive(parallelData->masterTape)) {
51+
if (parallelData != nullptr) {
5252
if (index == 0) {
5353
parallelData->actualThreads = actualParallelism;
5454
}
@@ -107,6 +107,10 @@ void opdi::ImplicitTaskOmpLogic::onImplicitTaskEnd(void* dataPtr) {
107107

108108
tool->setActive(data->parallelData->tapes[data->index], false);
109109

110+
if (data->oldTape == data->parallelData->masterTape) {
111+
tool->setActive(data->oldTape, true);
112+
}
113+
110114
delete data;
111115
}
112116
}

‎include/opdi/logic/omp/instrument/ompLogicInstrumentInterface.hpp‎

Lines changed: 9 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -35,6 +35,7 @@
3535

3636
#include "../../logicInterface.hpp"
3737
#include "../implicitTaskOmpLogic.hpp"
38+
#include "../masterOmpLogic.hpp"
3839
#include "../mutexOmpLogic.hpp"
3940
#include "../parallelOmpLogic.hpp"
4041
#include "../syncRegionOmpLogic.hpp"
@@ -49,6 +50,9 @@ namespace opdi {
4950

5051
virtual void reverseFlush() {}
5152

53+
virtual void reverseImplicitTaskBegin(ParallelOmpLogic::Data* /*data*/, int /*threadNum*/) {}
54+
virtual void reverseImplicitTaskEnd(ParallelOmpLogic::Data* /*data*/, int /*threadNum*/) {}
55+
virtual void reverseImplicitTaskPart(ParallelOmpLogic::Data* /*data*/, int /*threadNum*/, std::size_t /*part*/) {}
5256
virtual void onImplicitTaskBegin(ImplicitTaskOmpLogic::Data* /*data*/) {}
5357
virtual void onImplicitTaskEnd(ImplicitTaskOmpLogic::Data* /*data*/) {}
5458

@@ -58,8 +62,8 @@ namespace opdi {
5862
virtual void onMutexAcquired(MutexOmpLogic::Data* /*data*/) {}
5963
virtual void onMutexReleased(MutexOmpLogic::Data* /*data*/) {}
6064

61-
virtual void reverseParallel(ParallelOmpLogic::Data* /*data*/) {}
62-
virtual void reverseParallelPart(ParallelOmpLogic::Data* /*data*/, std::size_t /*j*/) {}
65+
virtual void reverseParallelBegin(ParallelOmpLogic::Data* /*data*/) {}
66+
virtual void reverseParallelEnd(ParallelOmpLogic::Data* /*data*/) {}
6367
virtual void onParallelBegin(ParallelOmpLogic::Data* /*data*/) {}
6468
virtual void onParallelEnd(ParallelOmpLogic::Data* /*data*/) {}
6569

@@ -69,6 +73,9 @@ namespace opdi {
6973
virtual void reverseWork(WorkOmpLogic::Data* /*data*/) {}
7074
virtual void onWork(LogicInterface::WorksharingKind /*kind*/, LogicInterface::ScopeEndpoint /*endpoint*/) {}
7175

76+
virtual void reverseMaster(MasterOmpLogic::Data* /*data*/) {}
77+
virtual void onMaster(LogicInterface::ScopeEndpoint /*endpoint*/) {}
78+
7279
virtual void onSetAdjointAccessMode(LogicInterface::AdjointAccessMode /*adjointAccess*/) {}
7380
};
7481

‎include/opdi/logic/omp/instrument/ompLogicOutputInstrument.hpp‎

Lines changed: 55 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -45,6 +45,29 @@ namespace opdi {
4545
TapedOutput::print("R FLSH l", omp_get_level(), "t", omp_get_thread_num());
4646
}
4747

48+
virtual void reverseImplicitTaskBegin(ParallelOmpLogic::Data* data, int threadNum) {
49+
TapedOutput::print("R IMTB l", omp_get_level(),
50+
"t", threadNum,
51+
"tape", data->tapes[threadNum],
52+
"pos", tool->positionToString(data->positions[threadNum].back()));
53+
}
54+
55+
virtual void reverseImplicitTaskEnd(ParallelOmpLogic::Data* data, int threadNum) {
56+
TapedOutput::print("R IMTE l", omp_get_level(),
57+
"t", threadNum,
58+
"tape", data->tapes[threadNum],
59+
"pos", tool->positionToString(data->positions[threadNum].front()));
60+
}
61+
62+
virtual void reverseImplicitTaskPart(ParallelOmpLogic::Data* data, int threadNum, std::size_t part) {
63+
TapedOutput::print("R IMTP l", omp_get_level(),
64+
"t", threadNum,
65+
"tape", data->tapes[threadNum],
66+
"start", tool->positionToString(data->positions[threadNum][part]),
67+
"end", tool->positionToString(data->positions[threadNum][part - 1]),
68+
"mode", data->adjointAccessModes[threadNum][part - 1]);
69+
}
70+
4871
virtual void onImplicitTaskBegin(ImplicitTaskOmpLogic::Data* data) {
4972
TapedOutput::print("F IMTB l", omp_get_level(),
5073
"t", data->index,
@@ -94,32 +117,38 @@ namespace opdi {
94117
"at", data->traceValue);
95118
}
96119

97-
virtual void reverseParallel(ParallelOmpLogic::Data* data) {
98-
TapedOutput::print("R PARA t", omp_get_thread_num(),
99-
"tape", data->tapes[omp_get_thread_num()],
100-
"start", tool->positionToString(data->positions[omp_get_thread_num()].back()),
101-
"end", tool->positionToString(data->positions[omp_get_thread_num()].front()));
120+
virtual void reverseParallelBegin(ParallelOmpLogic::Data* data) {
121+
TapedOutput::print("R PARB l", omp_get_level(),
122+
"master", data->masterTape);
102123
}
103124

104-
virtual void reverseParallelPart(ParallelOmpLogic::Data* data, std::size_t j) {
105-
TapedOutput::print("R PARP t", omp_get_thread_num(),
106-
"tape", data->tapes[omp_get_thread_num()],
107-
"j", j,
108-
"start", tool->positionToString(data->positions[omp_get_thread_num()][j]),
109-
"end", tool->positionToString(data->positions[omp_get_thread_num()][j - 1]),
110-
"mode", data->adjointAccessModes[omp_get_thread_num()][j - 1]);
125+
virtual void reverseParallelEnd(ParallelOmpLogic::Data* data) {
126+
TapedOutput::print("R PARE l", omp_get_level(),
127+
"master", data->masterTape);
111128
}
112129

113130
virtual void onParallelBegin(ParallelOmpLogic::Data* data) {
114-
TapedOutput::print("F PARB l", omp_get_level(),
115-
"master", data->masterTape,
116-
"mode", data->outerAdjointAccessMode);
131+
if (data == nullptr) {
132+
TapedOutput::print("F PARB l", omp_get_level(),
133+
"(passive)");
134+
}
135+
else {
136+
TapedOutput::print("F PARB l", omp_get_level(),
137+
"master", data->masterTape,
138+
"mode", data->outerAdjointAccessMode);
139+
}
117140
}
118141

119142
virtual void onParallelEnd(ParallelOmpLogic::Data* data) {
120-
TapedOutput::print("F PARE l", omp_get_level(),
121-
"master", data->masterTape,
122-
"mode", data->outerAdjointAccessMode);
143+
if (data == nullptr) {
144+
TapedOutput::print("F PARE l", omp_get_level(),
145+
"(passive)");
146+
}
147+
else {
148+
TapedOutput::print("F PARE l", omp_get_level(),
149+
"master", data->masterTape,
150+
"mode", data->outerAdjointAccessMode);
151+
}
123152
}
124153

125154
virtual void reverseSyncRegion(SyncRegionOmpLogic::Data* data) {
@@ -137,5 +166,13 @@ namespace opdi {
137166
virtual void onWork(LogicInterface::WorksharingKind kind, LogicInterface::ScopeEndpoint endpoint) {
138167
TapedOutput::print("F WORK t", omp_get_thread_num(), "kind", kind, "endp", endpoint);
139168
}
169+
170+
virtual void reverseMaster(MasterOmpLogic::Data* data) {
171+
TapedOutput::print("R MAST t", omp_get_thread_num(), "endp", data->endpoint);
172+
}
173+
174+
virtual void onMaster(LogicInterface::ScopeEndpoint endpoint) {
175+
TapedOutput::print("F MAST t", omp_get_thread_num(), "endp", endpoint);
176+
}
140177
};
141178
}

0 commit comments

Comments
 (0)