Skip to content

Commit 4f55d49

Browse files
committed
add pipline design
1 parent c4fe63c commit 4f55d49

1 file changed

Lines changed: 134 additions & 0 deletions

File tree

doc/design.md

Lines changed: 134 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -186,3 +186,137 @@ We hope Fluid users could represent it by the following Python/Fluid code.
186186
```python
187187
def build_docker_image_from_git_source(
188188
```
189+
190+
### Pipeline
191+
192+
A Pipeline object is like function decleration.
193+
194+
A Pipeline in Tekton defined an ordered series of Tasks. A valid Pipeline declearation
195+
must include a reference to at last one `Task`, for example:
196+
197+
``` yaml
198+
tasks:
199+
- name: build-the-image
200+
taskRef:
201+
name: build-push
202+
```
203+
204+
The `Pipeline Tasks` in a Pipeline can be connected and run as a Directed Acyclic Graph(DAG), each of the Pipeline Tasks is a node, which can be connected with:
205+
206+
- `runAfter` clauses on the `Pipeline Tasks`.
207+
- `from` clauses on the `PipelineResources` needed by a `Task`.
208+
209+
For an example `Pipeline` spec with `runAfter`:
210+
211+
``` yaml
212+
- name: lint-repo
213+
taskRef:
214+
name: pylint
215+
resources:
216+
inputs:
217+
- name: workspace
218+
resource: my-repo
219+
- name: test-app
220+
taskRef:
221+
name: make-test
222+
resources:
223+
inputs:
224+
- name: workspace
225+
resource: my-repo
226+
- name: build-app
227+
taskRef:
228+
name: kaniko-build-app
229+
runAfter:
230+
- test-app
231+
resources:
232+
inputs:
233+
- name: workspace
234+
resource: my-repo
235+
outputs:
236+
- name: image
237+
resource: my-app-image
238+
- name: build-frontend
239+
taskRef:
240+
name: kaniko-build-frontend
241+
runAfter:
242+
- test-app
243+
resources:
244+
inputs:
245+
- name: workspace
246+
resource: my-repo
247+
outputs:
248+
- name: image
249+
resource: my-frontend-image
250+
- name: deploy-all
251+
taskRef:
252+
name: deploy-kubectl
253+
resources:
254+
inputs:
255+
- name: my-app-image
256+
resource: my-app-image
257+
from:
258+
- build-app
259+
- name: my-frontend-image
260+
resource: my-frontend-image
261+
from:
262+
- build-frontend
263+
```
264+
265+
This will result the following execution graph:
266+
267+
``` text
268+
| |
269+
v v
270+
test-app lint-repo
271+
/ \
272+
v v
273+
build-app build-frontend
274+
\ /
275+
v v
276+
deploy-all
277+
```
278+
279+
In Python, a function is like a node of the DAG, which connected by the input
280+
and of output of the functions.
281+
282+
We hope Fluid users can write the following program to express a DAG with `fluid.pipeline`:
283+
284+
``` python
285+
@fluid.task
286+
def pylint():
287+
fluid.step(...)
288+
289+
@fluid.task
290+
def make_test():
291+
fluid.step(...)
292+
293+
@fluid.task
294+
def kaniko_build_app():
295+
fluid.step(...)
296+
297+
@fluid.task
298+
def kaniko_build_frontend():
299+
fluid.step(...)
300+
301+
@fluid.task
302+
def deploy_kubectl():
303+
fluid.step(...)
304+
305+
@fluid.pipeline
306+
def dag_demo():
307+
lint_repo = pylint()
308+
test_app = make_test()
309+
build_app = kaniko_build_app().run_after(test_app)
310+
build_frontend = kaniko_build_frontend().run_after(test_app)
311+
deploy_all = deploy_kubectl()
312+
deploy_all.inputs.my_frontend_image.from(build_app)
313+
deploy_all.inputs.my_app_image.from(build_frontend)
314+
```
315+
316+
### PipelineRun
317+
318+
A PipelineRun object is like a function invocation:
319+
320+
``` python
321+
build_pipeline()
322+
```

0 commit comments

Comments
 (0)