Skip to content

Commit 84df18f

Browse files
grussorussogrussorusso
committed
Merged changes from main
2 parents a205f93 + 7193b72 commit 84df18f

69 files changed

Lines changed: 19665 additions & 75 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎README.md‎

Lines changed: 20 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -32,7 +32,7 @@ explains how to obtain and use the software, is available
3232

3333
## Building from sources
3434

35-
1. Check that Golang is correctly installed on your machine.
35+
1. Check that Golang 1.20+ is installed.
3636

3737
2. Download a copy of the source code.
3838

@@ -47,7 +47,7 @@ You will find executables in `./bin/`.
4747
## Running (single-node deployment)
4848

4949
As functions are executed within Docker containers, you need Docker to
50-
be installed on the host. Furthermore, the Serverledge node needs
50+
be installed on the host. Furthermore, Serverledge needs
5151
permissions to create containers.
5252

5353
If you have more than one context (docker context ls), be sure to set up the DOCKER_HOST environment variable to the correct Host context.
@@ -64,20 +64,20 @@ server:
6464

6565
$ ./scripts/start-etcd.sh # stop it with ./scripts/stop-etcd.sh
6666

67-
Start a Serverledge node:
67+
Start a local Serverledge node:
6868

6969
$ bin/serverledge
7070

7171
### Creating and invoking functions
7272

73-
Register a function `func` from example python code (the handler is formatted like this: $(filename).$(functionName)):
73+
Register a function `func` from example Python code (the handler is formatted like this: $(filename).$(functionName)):
7474

75-
$ bin/serverledge-cli create -f func --memory 600 --src examples/hello.py --runtime python310 --handler "hello.handler"
75+
$ bin/serverledge-cli create -f func --memory 200 --src examples/hello.py --runtime python310 --handler "hello.handler"
7676

77-
Register a function `func` from example javascript code (the handler is formatted like this: $(filename) and the name of the function is "handler"):
77+
Register a function `func` from example JS code (the handler is formatted like this: $(filename) and the name of the function is "handler"):
7878

79-
$ bin/serverledge-cli create -f func --memory 600 --src examples/hello.js --runtime nodejs17 --handler "hello"
80-
$ bin/serverledge-cli create -f func --memory 600 --src examples/inc.js --runtime nodejs17 --handler "inc"
79+
$ bin/serverledge-cli create -f func --memory 200 --src examples/hello.js --runtime nodejs17 --handler "hello"
80+
$ bin/serverledge-cli create -f func --memory 200 --src examples/inc.js --runtime nodejs17 --handler "inc"
8181

8282
Invoke `func` with arguments `a=2` and `b=3`:
8383

@@ -95,6 +95,8 @@ where `input.json` may contain:
9595
"b": 3
9696
}
9797

98+
#### Asynchronous Invocation
99+
98100
Functions can be also invoked asynchronously using the `--async` flag:
99101

100102
$ bin/serverledge-cli invoke -f func --async
@@ -105,6 +107,14 @@ poll for the execution result:
105107
$ bin/serverledge-cli poll --request <requestID>
106108

107109

110+
#### Getting function standard output
111+
112+
You may want to see the content printed by the function to its standard output/error. To do so, add the `--return_output` flag (`-o` for short):
113+
114+
$ bin/serverledge-cli invoke -f func -p "a:2" -p "b:3" --return_output
115+
116+
Note that we currently support output capture only for some runtimes (e.g., Python supports it).
117+
108118
## Distributed Deployment
109119

110120
[This repository](https://github.com/grussorusso/serverledge-deploy) provides an
@@ -151,10 +161,10 @@ The configuration file may look like this:
151161

152162
## Additional Documentation
153163

154-
164+
- [API reference](./docs/api.md)
155165
- [Writing functions](./docs/writing-functions.md)
156-
- [Metrics](./docs/metrics.md)
157166
- [Serverledge Internals: Executor](./docs/executor.md)
167+
- [Metrics](./docs/metrics.md)
158168

159169

160170
## License

‎cmd/serverledge/main.go‎

Lines changed: 81 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,20 +1,99 @@
11
package main
22

33
import (
4+
"context"
5+
"errors"
46
"fmt"
7+
"log"
8+
"net/http"
9+
"os"
10+
"os/signal"
11+
"time"
12+
513
"github.com/grussorusso/serverledge/internal/api"
14+
"github.com/grussorusso/serverledge/internal/cache"
615
"github.com/grussorusso/serverledge/internal/config"
716
"github.com/grussorusso/serverledge/internal/metrics"
817
"github.com/grussorusso/serverledge/internal/node"
918
"github.com/grussorusso/serverledge/internal/registration"
1019
"github.com/grussorusso/serverledge/internal/scheduling"
1120
"github.com/grussorusso/serverledge/utils"
12-
"log"
13-
"os"
1421

1522
"github.com/labstack/echo/v4"
23+
"github.com/labstack/echo/v4/middleware"
1624
)
1725

26+
func startAPIServer(e *echo.Echo) {
27+
e.Use(middleware.Recover())
28+
29+
// Routes
30+
e.POST("/invoke/:fun", api.InvokeFunction)
31+
e.POST("/prewarm", api.PrewarmFunction)
32+
e.POST("/create", api.CreateFunction)
33+
e.POST("/delete", api.DeleteFunction)
34+
e.GET("/function", api.GetFunctions)
35+
e.GET("/poll/:reqId", api.PollAsyncResult)
36+
e.GET("/status", api.GetServerStatus)
37+
38+
// Start server
39+
portNumber := config.GetInt(config.API_PORT, 1323)
40+
e.HideBanner = true
41+
42+
if err := e.Start(fmt.Sprintf(":%d", portNumber)); err != nil && !errors.Is(err, http.ErrServerClosed) {
43+
e.Logger.Fatal("shutting down the server")
44+
}
45+
}
46+
47+
func cacheSetup() {
48+
//todo fix default values
49+
50+
// setup cache space
51+
cache.Size = config.GetInt(config.CACHE_SIZE, 10)
52+
53+
//setup cleanup interval
54+
d := config.GetInt(config.CACHE_CLEANUP, 60)
55+
interval := time.Duration(d)
56+
cache.CleanupInterval = interval * time.Second
57+
58+
//setup default expiration time
59+
d = config.GetInt(config.CACHE_ITEM_EXPIRATION, 60)
60+
expirationInterval := time.Duration(d)
61+
cache.DefaultExp = expirationInterval * time.Second
62+
63+
//cache first creation
64+
cache.GetCacheInstance()
65+
}
66+
67+
func registerTerminationHandler(r *registration.Registry, e *echo.Echo) {
68+
c := make(chan os.Signal)
69+
signal.Notify(c, os.Interrupt)
70+
71+
go func() {
72+
select {
73+
case sig := <-c:
74+
fmt.Printf("Got %s signal. Terminating...\n", sig)
75+
node.ShutdownAllContainers()
76+
77+
// deregister from etcd; server should be unreachable
78+
err := r.Deregister()
79+
if err != nil {
80+
log.Fatal(err)
81+
}
82+
83+
//stop container janitor
84+
node.StopJanitor()
85+
86+
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
87+
defer cancel()
88+
if err := e.Shutdown(ctx); err != nil {
89+
e.Logger.Fatal(err)
90+
}
91+
92+
os.Exit(0)
93+
}
94+
}()
95+
}
96+
1897
func main() {
1998
configFileName := ""
2099
if len(os.Args) > 1 {

‎docs/api.md‎

Lines changed: 163 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,163 @@
1+
## Serverledge API Reference
2+
3+
4+
<!--
5+
6+
<details>
7+
<summary><code>POST</code> <code><b>/create</b></code> <code>(registers a new function)</code></summary>
8+
Details
9+
</details>
10+
-->
11+
12+
13+
### Registering a new function
14+
15+
<code>POST</code> <code><b>/create</b></code> (registers a new function)
16+
17+
##### Parameters
18+
19+
> | name | required | type | description |
20+
> |-----------|-------------|-------------------------|------------|
21+
> | `Name` | yes | string | Name of the function (globally unique) |
22+
> | `Runtime` | yes | string | Base container runtime (e.g., `python310`)
23+
> | `MemoryMB` | yes | int | Memory (in MB) reserved for each function instance
24+
> | `CPUDemand` | | float | Max CPU cores (or fractions of) allocated to function instances (e.g., `1.0` means up to 1 core, `-1.0` means no cap)
25+
> | `Handler` | (yes) | string | Function entrypoint in the source package; syntax and semantics depend on the chosen runtime (e.g., `module.function_name`). Not needed if `Runtime` is `custom`
26+
> | `TarFunctionCode` | (yes) | string | Source code package as a base64-encoded TAR archive. Not needed if `Runtime` is `custom`
27+
> | `CustomImage` | | string | If `Runtime` is `custom`: custom container image to use
28+
29+
30+
##### Responses
31+
32+
> | http code | content-type | response | comments |
33+
> |---------------|-----------------------------------|---------------------------------|-----------------------------------|
34+
> | `200` | `application/json` | `{ "Created": "function_name" }` | |
35+
> | `404` | `text/plain` | `Invalid runtime.` | Chosen `Runtime` does not exist |
36+
> | `409` | `text/plain` | | Function already exists |
37+
> | `503` | `text/plain` | | Creation failed |
38+
39+
40+
41+
------------------------------------------------------------------------------------------
42+
### Deleting a function
43+
44+
<code>POST</code> <code><b>/delete</b></code> (deletes an existing function)
45+
46+
##### Parameters
47+
48+
> | name | required | type | description |
49+
> |-----------|-------------|-------------------------|------------|
50+
> | `Name` | yes | string | Name of the function |
51+
52+
53+
##### Responses
54+
55+
> | http code | content-type | response | comments |
56+
> |---------------|-----------------------------------|---------------------------------|-----------------------------------|
57+
> | `200` | `application/json` | `{ "Deleted": "function_name" }` | |
58+
> | `404` | `text/plain` | `Unknown function.` | The function does not exist |
59+
> | `503` | `text/plain` | | Creation failed |
60+
61+
62+
63+
------------------------------------------------------------------------------------------
64+
65+
### Invoking a function
66+
67+
<code>POST</code> <code><b>/invoke/<func></b></code> (invokes function `<func>`)
68+
69+
##### Parameters
70+
71+
> | name | required | type | description |
72+
> |-----------|-------------|-------------------------|------------|
73+
> | `Params` | yes | dict | Key-value specification of invocation parameters |
74+
> | `CanDoOffloading` | | bool | Whether the request can be offloaded (default: true) |
75+
> | `Async` | | bool | Whether the invocation is asynchronous (default: false) |
76+
> | `QoSClass` | | int | ID of the QoS class for the request |
77+
> | `QoSMaxRespT` | | float | Desired max response time |
78+
> | `ReturnOutput` | | bool | Whether function std. output and error should be collected (if supported by the function runtime) |
79+
80+
81+
##### Responses
82+
83+
> | http code | content-type | response | comments |
84+
> |---------------|-----------------------------------|---------------------------------|-----------------------------------|
85+
> | `200` | `application/json` | *See below.* | |
86+
> | `404` | `text/plain` | `Function unknown.` | |
87+
> | `429` | `text/plain` | | Not served because of excessive load. |
88+
> | `500` | `text/plain` | | Invocation failed. |
89+
90+
An example response for a successful **synchronous** request:
91+
92+
{
93+
"Success": true,
94+
"Result": "{\"IsPrime\": false}",
95+
"ResponseTime": 0.712851098,
96+
"IsWarmStart": false,
97+
"InitTime": 0.709491144,
98+
"OffloadLatency": 0,
99+
"Duration": 0.003351790000000021,
100+
"SchedAction": ""
101+
}
102+
103+
`Result` contains the object returned by the function upon completion.
104+
The other fields provide lower-level information. For instance, `Duration`
105+
reports the execution time of the function (in seconds), excluding all the
106+
communication and initialization overheads. `IsWarmStart` indicates whether
107+
a warm container has been used for the request.
108+
109+
110+
An example response for a successful **asynchronous** request:
111+
112+
{
113+
"ReqId": "isprime-98330239242748"
114+
}
115+
116+
`ReqId` can be used later to poll the execution results.
117+
118+
------------------------------------------------------------------------------------------
119+
### Polling for the results of an async request
120+
121+
<code>GET</code> <code><b>/poll/<reqId></b></code> (polls the results of `<reqId>`)
122+
123+
##### Parameters
124+
125+
`<reqId>` is the request identifier, as returned by `/invoke`.
126+
127+
##### Responses
128+
129+
> | http code | content-type | response | comments |
130+
> |---------------|-----------------------------------|---------------------------------|-----------------------------------|
131+
> | `200` | `application/json` | *See response to synchronous requests.* | |
132+
> | `404` | `text/plain` | | Results not found. |
133+
> | `500` | `text/plain` | `Could not retrieve results` |
134+
> | `500` | `text/plain` | `Failed to connect to Global Registry` |
135+
136+
------------------------------------------------------------------------------------------
137+
### Prewarming a function
138+
139+
<code>POST</code> <code><b>/prewarm</b></code> (prewarms instances for a function)
140+
141+
##### Parameters
142+
143+
> | name | required | type | description |
144+
> |-----------|-------------|-------------------------|------------|
145+
> | `Function` | yes | string | Name of the function |
146+
> | `Instances` | yes | int | Instances to spawn (0 to only pull the image)|
147+
> | `ForceImagePull` | | bool | Always check for image updates, even if a local copy exists |
148+
149+
150+
##### Responses
151+
152+
> | http code | content-type | response | comments |
153+
> |---------------|-----------------------------------|---------------------------------|-----------------------------------|
154+
> | `200` | `application/json` | `{ "Prewarmed": N }` | The number of prewarmed instances is returned. **It might be less than `Instances`** due to resource shortage.
155+
> | `404` | `text/plain` | `Unknown function.` | The function does not exist |
156+
> | `503` | `text/plain` | | Prewarming failed |
157+
158+
------------------------------------------------------------------------------------------
159+
160+
<!--
161+
status API
162+
function API
163+
-->

‎docs/custom_runtime.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -35,6 +35,7 @@ You can write a `Dockerfile` as follows to build your own runtime image, e.g.:
3535
COPY function.py /
3636
# ...
3737

38+
A complete [example](../examples/c++/README.md) is provided for C++.
3839

3940
## Custom image (the better way)
4041

‎docs/executor.md‎

Lines changed: 10 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -20,10 +20,11 @@ An `InvocationRequest` has the following fields:
2020

2121
```
2222
type InvocationRequest struct {
23-
Command []string
24-
Params map[string]interface{}
25-
Handler string
26-
HandlerDir string
23+
Command []string
24+
Params map[string]interface{}
25+
Handler string
26+
HandlerDir string
27+
ReturnOutput bool
2728
}
2829
```
2930

@@ -38,17 +39,22 @@ E.g., for Python runtimes, `<module_name>.<function_name>`.
3839

3940
- `HandlerDir`: directory where the function code has been copied.
4041

42+
- `ReturnOutput`: whether function standard output and error should be returned.
43+
4144
The following object is returned upon function completion (or failure):
4245

4346
```
4447
type InvocationResult struct {
4548
Success bool
4649
Result string
50+
Output string
4751
}
4852
```
4953

5054
- `Success`: whether the function has been successfully executed.
5155

5256
- `Result`: what the function returned.
5357

58+
- `Output`: function combined std. output and error (if captured)
59+
5460

0 commit comments

Comments
 (0)