Skip to content

Commit cc12621

Browse files
grussorussomatnar
andcommitted
Workflow: code review
Co-authored-by: Matteo Nardelli <matnar@gmail.com>
1 parent 2699027 commit cc12621

103 files changed

Lines changed: 4836 additions & 8129 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.github/workflows/makefile.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -23,4 +23,4 @@ jobs:
2323
run: make
2424

2525
- name: Test
26-
run: make unit_test
26+
run: make test

Makefile

Lines changed: 1 addition & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -14,9 +14,6 @@ serverledge-cli:
1414
executor:
1515
CGO_ENABLED=0 $(GO) build -o $(BIN)/$@ cmd/$@/executor.go
1616

17-
proto:
18-
protoc --go_out=. --go_opt=paths=source_relative --go-grpc_out=. --go-grpc_opt=paths=source_relative internal/scheduling/protobuf/solver.proto
19-
2017
DOCKERHUB_USER=grussorusso
2118
images: image-python310 image-nodejs17ng image-base
2219
image-python310:
@@ -26,33 +23,13 @@ image-base:
2623
image-nodejs17ng:
2724
docker build -t $(DOCKERHUB_USER)/serverledge-nodejs17ng -f images/nodejs17ng/Dockerfile .
2825

29-
images-multi-arch: image-python310-multi-arch image-nodejs17ng-multi-arch image-base-multi-arch
30-
PLATFORMS=linux/amd64,linux/arm64,linux/arm/v7
31-
image-python310-multi-arch:
32-
docker buildx build --platform $(PLATFORMS) -t $(DOCKERHUB_USER)/serverledge-python310 -f images/python310/Dockerfile --push .
33-
image-base-multi-arch:
34-
docker buildx build --platform $(PLATFORMS) -t $(DOCKERHUB_USER)/serverledge-base -f images/base-alpine/Dockerfile --push .
35-
image-nodejs17ng-multi-arch:
36-
docker buildx build --platform $(PLATFORMS) -t $(DOCKERHUB_USER)/serverledge-nodejs17ng -f images/nodejs17ng/Dockerfile --push .
37-
3826
push-images:
3927
docker push $(DOCKERHUB_USER)/serverledge-python310
4028
docker push $(DOCKERHUB_USER)/serverledge-base
4129
docker push $(DOCKERHUB_USER)/serverledge-nodejs17ng
4230

43-
unit_test:
44-
go test -v ./...
45-
4631
test:
47-
INTEGRATION=1 go test -v ./...
48-
49-
# to get coverage, install cover
50-
# go get golang.org/x/tools/cmd/cover
51-
test_coverage:
52-
go test -coverprofile coverage.txt ./...
53-
54-
coverage:
55-
go tool cover -html=coverage.txt
32+
go test -v ./...
5633

5734
.PHONY: serverledge serverledge-cli lb executor test images
5835

README.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -51,8 +51,8 @@ be installed on the host. Furthermore, Serverledge needs
5151
permissions to create containers.
5252

5353
You also need an `etcd` server to run Serverledge. To quickly start a local
54-
server, just download [a recent
55-
release](https://github.com/etcd-io/etcd/releases) of `etcd` and just run `./etcd`.
54+
server, just download [a recent release](https://github.com/etcd-io/etcd/releases)
55+
of `etcd` and just run `./etcd`.
5656

5757
Start a local Serverledge node:
5858

docs/custom_runtime.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ your function code, as explained below.
77

88
The easiest way to build a custom function image is by leveraging the
99
Serverledge base runtime image, i.e., `grussorusso/serverledge-base`.
10-
This image contains a simple implementation of the [Executor](https://github.com/grussorusso/serverledge/blob/main/docs/executor.md)
10+
This image contains a simple implementation of the [Executor](https://github.com/serverledge-faas/serverledge/blob/main/docs/executor.md)
1111
server. When the function is invoked, the Executor runs a user-specified
1212
command as a new process and sets a few environment variables that may be
1313
used by the called process:

docs/writing-function-compositions.md

Lines changed: 12 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -1,23 +1,23 @@
11
# Functions composition
22

3-
Serverledge accepts DAGs defined by users through a subset of the JSON-based *Amazon States Language*, currently in use by AWS Step Functions. Furthermore, you can define DAGs programmatically
4-
with the DagBuilder APIs.
3+
Serverledge accepts workflows defined by users through a subset of the JSON-based *Amazon States Language*, currently in use by AWS Step Functions. Furthermore, you can define workflows programmatically
4+
with the Builder APIs.
55

6-
Serverledge DAGs comprise 4 types of nodes:
6+
Serverledge workflows comprise 4 types of nodes:
77
- **SimpleNode**: a node that wraps a function. This is the only node that executes user-defined functions.
88
- **ChoiceNode**: a node with N conditions that transfers its input to the first branch whose condition is evaluated as true
99
- **FanOutNode**: a node with N outputs that copies (or scatters) the input to all the outputs (with subsequent nodes activated in parallel)
1010
- **FanInNode**: a node with N inputs that waits for the termination of all the parent nodes, and then merges the results in one output. The node fails after a specified timeout.
1111

1212
Three special nodes are always present and pre-built when using the APIs:
13-
- **StartNode**: the single node from which the DAG starts executing
14-
- **EndNode**: the final node of the DAG
15-
- **ErrorNode**: a node that terminates DAG execution with failure
13+
- **StartNode**: the single node from which the workflow starts executing
14+
- **EndNode**: the final node of the workflow
15+
- **ErrorNode**: a node that terminates workflow execution with failure
1616

1717

1818
## Signature
1919
Specifying a signature is optional for Serverledge functions. However,
20-
in order to use functions within DAGs, they must have an associated signature.
20+
in order to use functions within workflows, they must have an associated signature.
2121
A signature specifies the type of the inputs accepted by the function, as well as the type of the produced outputs. For instance, a *Fibonacci* function might have a single integer input, and produce a single integer output.
2222

2323
The signature can be specified when creating a function through the CLI.
@@ -41,10 +41,10 @@ Example:
4141

4242
TODO
4343

44-
## DagBuilder API
44+
## Builder API
4545

46-
It is possible to use the internal builder APIs to build complex DAGs programmatically in Go.
47-
Here is an example of a DAG made by two simple nodes and a choice node, with N alternative conditions
46+
It is possible to use the internal builder APIs to build complex workflows programmatically in Go.
47+
Here is an example of a workflow made by two simple nodes and a choice node, with N alternative conditions
4848

4949
N := 4
5050
function := function.Function{...}
@@ -53,11 +53,11 @@ Here is an example of a DAG made by two simple nodes and a choice node, with N a
5353
...
5454
condition[N-1] = Condition{...}
5555

56-
NewDagBuilder().
56+
NewBuilder().
5757
AddSimpleNode(&function).
5858
AddSimpleNode(&function2).
5959
AddChoiceNode(conditions).
60-
ForEach(NewDagBuilder().
60+
ForEach(NewBuilder().
6161
AddSimpleNode(&function).
6262
Build()).
6363
EndChoiceNode().

examples/inc.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,3 @@
11
def handler(params, context):
2+
print(f"Invoked inc with input: {params}")
23
return int(params["input"]) + 1

go.sum

Lines changed: 8 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -77,7 +77,7 @@ github.com/coreos/go-systemd/v22 v22.5.0/go.mod h1:Y58oyj3AT4RCenI/lSvhwexgC+NSV
7777
github.com/coreos/pkg v0.0.0-20180928190104-399ea9e2e55f/go.mod h1:E3G3o1h8I7cfcXa63jLwjI0eiQQMgzzUDFVpN/nH/eA=
7878
github.com/cornelk/hashmap v1.0.8 h1:nv0AWgw02n+iDcawr5It4CjQIAcdMMKRrs10HOJYlrc=
7979
github.com/cornelk/hashmap v1.0.8/go.mod h1:RfZb7JO3RviW/rT6emczVuC/oxpdz4UsSB2LJSclR1k=
80-
github.com/cpuguy83/go-md2man/v2 v2.0.0/go.mod h1:maD7wRr/U5Z6m/iR4s+kqSMx2CaBsrgA7czyZG/E6dU=
80+
github.com/cpuguy83/go-md2man/v2 v2.0.6/go.mod h1:oOW0eioCTA6cOiMLiUPZOpcVxMig6NIQQ7OS05n1F4g=
8181
github.com/creack/pty v1.1.9/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ33E=
8282
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
8383
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
@@ -93,8 +93,6 @@ github.com/docker/go-connections v0.4.0/go.mod h1:Gbd7IOopHjR8Iph03tsViu4nIes5Xh
9393
github.com/docker/go-units v0.5.0 h1:69rxXcBk27SvSaaxTtLh/8llcHD8vYHT7WSdRZ/jvr4=
9494
github.com/docker/go-units v0.5.0/go.mod h1:fgPhTUdO+D/Jk86RDLlptpiXQzgHJF7gydDDbaIK4Dk=
9595
github.com/dustin/go-humanize v1.0.0/go.mod h1:HtrtbFcZ19U5GC7JDqmcUSB87Iq5E25KnS6fMYU6eOk=
96-
github.com/enginyoyen/aslparser v0.2.0 h1:PJyxowHrDE/L8HvmoTEhT0LLHANSmL0hkxkLMgAy3tI=
97-
github.com/enginyoyen/aslparser v0.2.0/go.mod h1:UsMFe9B/voStSJSdG7fKYfzf7ppBX6zsUkjjRQRuP5s=
9896
github.com/envoyproxy/go-control-plane v0.9.0/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4=
9997
github.com/envoyproxy/go-control-plane v0.9.1-0.20191026205805-5f8ba28d4473/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4=
10098
github.com/envoyproxy/go-control-plane v0.9.4/go.mod h1:6rpuAdCZL397s3pYoYcLgu1mIlRU8Am5FuJP05cCM98=
@@ -199,8 +197,8 @@ github.com/hashicorp/hcl v1.0.0/go.mod h1:E5yfLk+7swimpb2L/Alb/PJmXilQ/rhwaUYs4T
199197
github.com/hexablock/vivaldi v0.0.0-20180727225019-07adad3f2b5f h1:vgMos6ed6qNnsswN5hB+l/y/UcuUxyq7OL6pXu8XkaI=
200198
github.com/hexablock/vivaldi v0.0.0-20180727225019-07adad3f2b5f/go.mod h1:oicL+P8ej+PDUzurbqC9ln05I+/+LP0YHljnBmv1HZw=
201199
github.com/ianlancetaylor/demangle v0.0.0-20181102032728-5e5cf60278f6/go.mod h1:aSSvb/t6k1mPoxDqO4vJh6VOCGPwU4O0C2/Eqndh1Sc=
202-
github.com/inconshreveable/mousetrap v1.0.0 h1:Z8tu5sraLXCXIcARxBp/8cbvlwVa7Z1NHg9XEKhtSvM=
203-
github.com/inconshreveable/mousetrap v1.0.0/go.mod h1:PxqpIevigyE2G7u3NXJIT2ANytuPF1OarO4DADm73n8=
200+
github.com/inconshreveable/mousetrap v1.1.0 h1:wN+x4NVGpMsO7ErUn/mUI3vEoE6Jt13X2s0bqwp9tc8=
201+
github.com/inconshreveable/mousetrap v1.1.0/go.mod h1:vpF70FUmC8bwa3OWnCshd2FqLfsEA9PFc4w1p2J65bw=
204202
github.com/jonboulle/clockwork v0.1.0/go.mod h1:Ii8DK3G1RaLaWxj9trq07+26W01tbo22gdxWY5EU2bo=
205203
github.com/jpillora/backoff v1.0.0/go.mod h1:J/6gKK9jxlEcS3zixgDgUAsiuZ7yrSoa/FX5e0EB2j4=
206204
github.com/json-iterator/go v1.1.6/go.mod h1:+SdeFBvtyEkXs7REEP0seUULqWtbJapLOCVDaaPEHmU=
@@ -247,7 +245,6 @@ github.com/matttproud/golang_protobuf_extensions v1.0.4 h1:mmDVorXM7PCGKw94cs5zk
247245
github.com/matttproud/golang_protobuf_extensions v1.0.4/go.mod h1:BSXmuO+STAnVfrANrmjBb36TMTDstsz7MSK+HVaYKv4=
248246
github.com/mikoim/go-loadavg v0.0.0-20150917074714-35ece5f6d547 h1:sKOBS3TQA6gIeu7xDDIJnH1cPmGAa3535gg2/cWrwC4=
249247
github.com/mikoim/go-loadavg v0.0.0-20150917074714-35ece5f6d547/go.mod h1:Gv1gEAo58s56eUbsb59IAFnEr6flyFg9lgryVQnKwhM=
250-
github.com/mitchellh/go-homedir v1.1.0/go.mod h1:SfyaCUpYCn1Vlf4IUYiD9fPX4A5wJrkLzIz1N1q0pr0=
251248
github.com/mitchellh/mapstructure v1.1.2 h1:fmNYVwqnSfB9mZU6OS2O6GsXM+wcskZDuKQzvN1EDeE=
252249
github.com/mitchellh/mapstructure v1.1.2/go.mod h1:FVVH3fgwuzCH5S8UJGiWEs2h04kUh9fWfEaFds41c1Y=
253250
github.com/moby/term v0.5.0 h1:xt8Q1nalod/v7BqbG21f8mQPqH+xAaC9C3N3wfWbVP0=
@@ -313,8 +310,7 @@ github.com/rogpeppe/go-internal v1.3.0/go.mod h1:M8bDsm7K2OlrFYOpmOWEs/qY81heoFR
313310
github.com/rogpeppe/go-internal v1.6.1/go.mod h1:xXDCJY+GAPziupqXw64V24skbSoqbTEfhy4qGm1nDQc=
314311
github.com/rogpeppe/go-internal v1.8.1 h1:geMPLpDpQOgVyCg5z5GoRwLHepNdb71NXb67XFkP+Eg=
315312
github.com/rogpeppe/go-internal v1.8.1/go.mod h1:JeRgkft04UBgHMgCIwADu4Pn6Mtm5d4nPKWu0nJ5d+o=
316-
github.com/russross/blackfriday/v2 v2.0.1/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM=
317-
github.com/shurcooL/sanitized_anchor_name v1.0.0/go.mod h1:1NzhyTcUVG4SuEtjjoZeVRXNmyL/1OwPU0+IJeTBvfc=
313+
github.com/russross/blackfriday/v2 v2.1.0/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM=
318314
github.com/sirupsen/logrus v1.2.0/go.mod h1:LxeOpSwHxABJmUn/MG1IvRgCAasNZTLOkJPxbbu5VWo=
319315
github.com/sirupsen/logrus v1.4.2/go.mod h1:tLMulIdttU9McNUspp0xgXVQah82FyeX6MwdIuYE2rE=
320316
github.com/sirupsen/logrus v1.6.0/go.mod h1:7uNnSEd1DgxDLC74fIahvMZmmYsHGZGEOFrfsX/uA88=
@@ -325,13 +321,13 @@ github.com/spf13/afero v1.2.2 h1:5jhuqJyZCZf2JRofRvN/nIFgIWNzPa3/Vz8mYylgbWc=
325321
github.com/spf13/afero v1.2.2/go.mod h1:9ZxEEn6pIJ8Rxe320qSDBk6AsU0r9pR7Q4OcevTdifk=
326322
github.com/spf13/cast v1.3.0 h1:oget//CVOEoFewqQxwr0Ej5yjygnqGkvggSE/gB35Q8=
327323
github.com/spf13/cast v1.3.0/go.mod h1:Qx5cxh0v+4UWYiBimWS+eyWzqEqokIECu5etghLkUJE=
328-
github.com/spf13/cobra v1.0.0 h1:6m/oheQuQ13N9ks4hubMG6BnvwOeaJrqSPLahSnczz8=
329-
github.com/spf13/cobra v1.0.0/go.mod h1:/6GTrnGXV9HjY+aR4k0oJ5tcvakLuG6EuKReYlHNrgE=
324+
github.com/spf13/cobra v1.9.1 h1:CXSaggrXdbHK9CF+8ywj8Amf7PBRmPCOJugH954Nnlo=
325+
github.com/spf13/cobra v1.9.1/go.mod h1:nDyEzZ8ogv936Cinf6g1RU9MRY64Ir93oCnqb9wxYW0=
330326
github.com/spf13/jwalterweatherman v1.0.0 h1:XHEdyB+EcvlqZamSM4ZOMGlc93t6AcsBEu9Gc1vn7yk=
331327
github.com/spf13/jwalterweatherman v1.0.0/go.mod h1:cQK4TGJAtQXfYWX+Ddv3mKDzgVb68N+wFjFa4jdeBTo=
332328
github.com/spf13/pflag v1.0.3/go.mod h1:DYY7MBk1bdzusC3SYhjObp+wFpr4gzcvqqNjLnInEg4=
333-
github.com/spf13/pflag v1.0.5 h1:iy+VFUOCP1a+8yFto/drg2CJ5u0yRoB7fZw3DKv/JXA=
334-
github.com/spf13/pflag v1.0.5/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
329+
github.com/spf13/pflag v1.0.6 h1:jFzHGLGAlb3ruxLB8MhbI6A8+AQX/2eW4qeyNZXNp2o=
330+
github.com/spf13/pflag v1.0.6/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
335331
github.com/spf13/viper v1.4.0 h1:yXHLWeravcrgGyFSyCgdYpXQ9dR9c/WED3pg1RhxqEU=
336332
github.com/spf13/viper v1.4.0/go.mod h1:PTJ7Z/lr49W6bUbkmS1V3by4uWynFiR9p7+dSq/yZzE=
337333
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
@@ -350,12 +346,6 @@ github.com/valyala/bytebufferpool v1.0.0/go.mod h1:6bBcMArwyJ5K/AmCkWv1jt77kVWyC
350346
github.com/valyala/fasttemplate v1.0.1/go.mod h1:UQGH1tvbgY+Nz5t2n7tXsz52dQxojPUpymEIMZ47gx8=
351347
github.com/valyala/fasttemplate v1.2.1 h1:TVEnxayobAdVkhQfrfes2IzOB6o+z4roRkPF52WA1u4=
352348
github.com/valyala/fasttemplate v1.2.1/go.mod h1:KHLXt3tVN2HBp8eijSv/kGJopbvo7S+qRAEEKiv+SiQ=
353-
github.com/xeipuuv/gojsonpointer v0.0.0-20180127040702-4e3ac2762d5f h1:J9EGpcZtP0E/raorCMxlFGSTBrsSlaDGf3jU/qvAE2c=
354-
github.com/xeipuuv/gojsonpointer v0.0.0-20180127040702-4e3ac2762d5f/go.mod h1:N2zxlSyiKSe5eX1tZViRH5QA0qijqEDrYZiPEAiq3wU=
355-
github.com/xeipuuv/gojsonreference v0.0.0-20180127040603-bd5ef7bd5415 h1:EzJWgHovont7NscjpAxXsDA8S8BMYve8Y5+7cuRE7R0=
356-
github.com/xeipuuv/gojsonreference v0.0.0-20180127040603-bd5ef7bd5415/go.mod h1:GwrjFmJcFw6At/Gs6z4yjiIwzuJ1/+UwLxMQDVQXShQ=
357-
github.com/xeipuuv/gojsonschema v1.2.0 h1:LhYJRs+L4fBtjZUfuSZIKGeVu0QRy8e5Xi7D17UxZ74=
358-
github.com/xeipuuv/gojsonschema v1.2.0/go.mod h1:anYRn/JVcOK2ZgGU+IjEV4nwlhoK5sQluxsYJ78Id3Y=
359349
github.com/xiang90/probing v0.0.0-20190116061207-43a291ad63a2/go.mod h1:UETIi67q53MR2AWcXfiuqkDkRtnGDLqkBTpCHuJHxtU=
360350
github.com/xordataexchange/crypt v0.0.3-0.20170626215501-b2862e3d0a77/go.mod h1:aYKd//L2LvnjZzWKhF00oedf4jCCReLcmhLdhm1A27Q=
361351
github.com/yuin/goldmark v1.1.25/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=

images/base-alpine/Dockerfile

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
FROM golang:1.20-alpine AS build
1+
FROM golang:1.21-alpine AS build
22

33
WORKDIR /sedge
44

0 commit comments

Comments
 (0)