Skip to content

Commit e623816

Browse files
freeznetHuanli-Meng
freeznet
and
Huanli-Meng
authored
add runner images document (#120)
* add images doc * refine the badge * update * Update images/README.md Co-authored-by: Huanli Meng <48120384+Huanli-Meng@users.noreply.github.com> * Update images/README.md Co-authored-by: Huanli Meng <48120384+Huanli-Meng@users.noreply.github.com> * Update images/README.md Co-authored-by: Huanli Meng <48120384+Huanli-Meng@users.noreply.github.com> * Update images/README.md Co-authored-by: Huanli Meng <48120384+Huanli-Meng@users.noreply.github.com> * Update images/README.md Co-authored-by: Huanli Meng <48120384+Huanli-Meng@users.noreply.github.com> * Update images/README.md Co-authored-by: Huanli Meng <48120384+Huanli-Meng@users.noreply.github.com> * Update images/README.md Co-authored-by: Huanli Meng <48120384+Huanli-Meng@users.noreply.github.com> * Update images/README.md Co-authored-by: Huanli Meng <48120384+Huanli-Meng@users.noreply.github.com> * Update images/README.md Co-authored-by: Huanli Meng <48120384+Huanli-Meng@users.noreply.github.com>
1 parent d7deb90 commit e623816

1 file changed

Lines changed: 154 additions & 0 deletions

File tree

‎images/README.md‎

Lines changed: 154 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,154 @@
1+
# Function Mesh Runner Images
2+
3+
Function Mesh uses runner images as functions / connectors' Pod images. Each runtime runner image only contains necessary tool-chains and libraries for specified runtime.
4+
5+
## Images
6+
### Base Runner
7+
The base Runner is located at `./pulsar-functions-base-runner`. The base runner contains basic tool-chains like `/pulsar/bin`, `/pulsar/conf` and `/pulsar/lib` to ensure that the `pulsar-admin` CLI tool works properly to support [Apache Pulsar Packages](http://pulsar.apache.org/docs/en/next/admin-api-packages/).
8+
9+
### Java Runner
10+
[![Docker Image Version (tag latest semver)](https://img.shields.io/docker/v/streamnative/pulsar-functions-java-runner/2.7.1?style=for-the-badge)](https://hub.docker.com/r/streamnative/pulsar-functions-java-runner)
11+
12+
The Java Runner is based on Base Runner and contains the Java function instance to run Java functions or connectors.
13+
14+
The `streamnative/pulsar-functions-java-runner` Java runner is stored at the Docker Hub and is automatically updated to align with Apache Pulsar release.
15+
16+
### Python Runner
17+
[![Docker Image Version (tag latest semver)](https://img.shields.io/docker/v/streamnative/pulsar-functions-python-runner/2.7.1?style=for-the-badge)](https://hub.docker.com/r/streamnative/pulsar-functions-python-runner)
18+
19+
The Python runner is based on the base runner and contains the Python function instance to run Python functions. You can build your own Python runner to customize Python dependencies.
20+
21+
The `streamnative/pulsar-functions-python-runner` Python runner is located at the Docker Hub and is automatically updated to align with Apache Pulsar release.
22+
23+
### Golang Runner
24+
[![Docker Image Version (tag latest semver)](https://img.shields.io/docker/v/streamnative/pulsar-functions-go-runner/2.7.1?style=for-the-badge)](https://hub.docker.com/r/streamnative/pulsar-functions-go-runner)
25+
26+
The Golang runner provides all the toolchain and dependencies required to run Golang functions.
27+
28+
The `streamnative/pulsar-functions-go-runner` Golang runner is located at the Docker Hub and is automatically updated to align with Apache Pulsar release.
29+
30+
## How to use
31+
32+
### Use with [Apache Pulsar Packages](http://pulsar.apache.org/docs/en/next/admin-api-packages/)
33+
#### Prerequisite
34+
35+
- Apache Pulsar 2.8.0
36+
- Function Mesh v0.1.3
37+
38+
#### Package your function locally
39+
You can package Pulsar Functions in Java, Python and Go. Please refer to the [official document](http://pulsar.apache.org/docs/en/next/functions-package/) to packaging your Pulsar Functions.
40+
41+
#### Upload your function to Pulsar Cluster
42+
With the packaged functions, user can upload the function package into [Apache Pulsar Packages](http://pulsar.apache.org/docs/en/next/admin-api-packages/).
43+
44+
You can upload a package to the package management service by `pulsar-admin`:
45+
46+
```bash
47+
bin/pulsar-admin packages upload function://my-tenant/my-ns/my-function@0.1 --path package-file --description package-description
48+
```
49+
50+
Then user can define Function Mesh CRDs to use the uploaded function package.
51+
52+
#### Submit to Function Mesh
53+
54+
Below is a sample CRD to create a Java function from package `function://my-tenant/my-ns/my-function@0.1`.
55+
```yaml
56+
apiVersion: compute.functionmesh.io/v1alpha1
57+
kind: Function
58+
metadata:
59+
name: java-function-sample
60+
namespace: default
61+
spec:
62+
image: streamnative/pulsar-functions-java-runner:2.7.1 # using java function runner
63+
className: org.apache.pulsar.functions.api.examples.ExclamationFunction
64+
sourceType: java.lang.String
65+
sinkType: java.lang.String
66+
forwardSourceMessageProperty: true
67+
MaxPendingAsyncRequests: 1000
68+
replicas: 1
69+
maxReplicas: 5
70+
logTopic: persistent://public/default/logging-function-logs
71+
input:
72+
topics:
73+
- persistent://public/default/java-function-input-topic
74+
output:
75+
topic: persistent://public/default/java-function-output-topic
76+
pulsar:
77+
pulsarConfig: "test-pulsar"
78+
java:
79+
jar: my-function.jar # the package will download as this filename.
80+
jarLocation: function://my-tenant/my-ns/my-function@0.1 # function package URL
81+
```
82+
83+
If you are not using self built runner image, you can skip `image`, Function Mesh manager will choose image according to the runtime defined in CRD. (for example, if user define `java` runtime settings, Function Mesh will use `streamnative/pulsar-functions-java-runner` by default)
84+
85+
### Use with self-built Docker image
86+
87+
#### Prerequisite
88+
89+
- Apache Pulsar 2.7.0
90+
- Function Mesh v0.1.3
91+
92+
#### Package your function locally
93+
You can package Pulsar Functions in Java, Python and Go. Please refer to the [official document](http://pulsar.apache.org/docs/en/next/functions-package/) to packaging your Pulsar Functions.
94+
95+
#### Build your function image
96+
You can write up a `Dockerfile` to build your function image packed with your function executable and all dependencies.
97+
For example, with a packed Java function called `example-function.jar`, you can define a `Dockerfile` as below.
98+
99+
```dockerfile
100+
# Use pulsar-functions-java-runner since we pack Java function
101+
FROM streamnative/pulsar-functions-java-runner:2.7.1
102+
# Copy function JAR package into /pulsar directory
103+
COPY example-function.jar /pulsar/
104+
```
105+
106+
With the `Dockerfile`, you can build the function image and push into image registry (like Docker Hub, or any private registry)
107+
108+
#### Submit to Function Mesh
109+
110+
Assume we build the image from previous step as `streamnative/example-function-image:latest`, and pushed into Docker Hub already.
111+
112+
Below is a sample CRD to create a Java function from docker image.
113+
```yaml
114+
apiVersion: compute.functionmesh.io/v1alpha1
115+
kind: Function
116+
metadata:
117+
name: java-function-sample
118+
namespace: default
119+
spec:
120+
image: streamnative/example-function-image:latest # using function image here
121+
className: org.apache.pulsar.functions.api.examples.ExclamationFunction
122+
sourceType: java.lang.String
123+
sinkType: java.lang.String
124+
forwardSourceMessageProperty: true
125+
MaxPendingAsyncRequests: 1000
126+
replicas: 1
127+
maxReplicas: 5
128+
logTopic: persistent://public/default/logging-function-logs
129+
input:
130+
topics:
131+
- persistent://public/default/java-function-input-topic
132+
output:
133+
topic: persistent://public/default/java-function-output-topic
134+
pulsar:
135+
pulsarConfig: "test-pulsar"
136+
java:
137+
jar: /pulsar/example-function.jar # the package location in image
138+
jarLocation: "" # leave empty since we will not download package from Pulsar Packages
139+
```
140+
141+
When you use Function Mesh with self build images, you need to define the executable location within CRD yaml file, below are the settings for different function runtime.
142+
143+
```yaml
144+
java: # Java runtime
145+
jar: example-function.jar
146+
python: # Python runtime
147+
py: exclamation_function.py
148+
golang: # Golang runtime
149+
go: go_func_executable
150+
```
151+
152+
## Scripts
153+
154+
`build.sh` is a bash script used to build base, Java, Python, and Golang runner locally.

0 commit comments

Comments
 (0)