Skip to content

Commit e2daffb

Browse files
arturgontijoarturgontijo
authored
Merge pull request #5 from arturgontijo/_new_daemon
New daemon
2 parents d5bf446 + a5562d0 commit e2daffb

5 files changed

Lines changed: 175 additions & 40 deletions

File tree

‎Dockerfile‎

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,7 @@
11
FROM ubuntu:18.04
22

3+
ARG snetd_version="v0.1.7"
4+
35
ENV SINGNET_REPOS=/opt/singnet
46

57
RUN mkdir -p ${SINGNET_REPOS}
@@ -15,9 +17,9 @@ RUN apt-get install -y python3 python3-pip
1517
RUN cd ${SINGNET_REPOS} && \
1618
mkdir snet-daemon && \
1719
cd snet-daemon && \
18-
wget https://github.com/singnet/snet-daemon/releases/download/v0.1.6/snet-daemon-v0.1.6-linux-amd64.tar.gz && \
19-
tar -xvf snet-daemon-v0.1.6-linux-amd64.tar.gz && \
20-
mv snet-daemon-v0.1.6-linux-amd64/snetd /usr/bin/snetd
20+
wget https://github.com/singnet/snet-daemon/releases/download/${snetd_version}/snet-daemon-${snetd_version}-linux-amd64.tar.gz && \
21+
tar -xvf snet-daemon-${snetd_version}-linux-amd64.tar.gz && \
22+
mv snet-daemon-${snetd_version}-linux-amd64/snetd /usr/bin/snetd
2123

2224
RUN cd ${SINGNET_REPOS} && \
2325
git clone https://github.com/singnet/example-service.git && \

‎README.md‎

Lines changed: 40 additions & 25 deletions
Original file line numberDiff line numberDiff line change
@@ -13,15 +13,15 @@ Simple arithmetic service compatible with SingularityNET
1313
* Clone the git repository:
1414

1515
```
16-
$ git clone git@github.com:singnet/example-service.git
17-
$ cd example-service
16+
git clone git@github.com:singnet/example-service.git
17+
cd example-service
1818
```
1919

2020
* Install the dependencies and compile the protobuf file:
2121

2222
```
23-
$ pip3 install -r requirements.txt
24-
$ sh buildproto.sh
23+
pip3 install -r requirements.txt
24+
sh buildproto.sh
2525
```
2626

2727
### Running
@@ -31,13 +31,13 @@ $ sh buildproto.sh
3131
* Run the example service directly (without `SNET Daemon`):
3232

3333
```
34-
$ python3 run_example_service.py --no-daemon
34+
python3 run_example_service.py --no-daemon
3535
```
3636

3737
* To test it run the script:
3838

3939
```
40-
$ python3 test_example_service.py
40+
python3 test_example_service.py
4141
```
4242

4343
#### With SingularityNET Daemon
@@ -59,6 +59,9 @@ Create the `SNET Daemon`'s config JSON file (`snetd.config.json`).
5959
"PASSTHROUGH_ENDPOINT": "http://SERVICE_GRPC_HOST:SERVICE_GRPC_PORT",
6060
"ORGANIZATION_ID": "ORGANIZATION_ID",
6161
"SERVICE_ID": "SERVICE_ID",
62+
"PAYMENT_CHANNEL_STORAGE_SERVER": {
63+
"DATA_DIR": "/opt/singnet/etcd/"
64+
},
6265
"LOG": {
6366
"LEVEL": "debug",
6467
"OUTPUT": {
@@ -68,20 +71,20 @@ Create the `SNET Daemon`'s config JSON file (`snetd.config.json`).
6871
}
6972
```
7073

71-
For example, using the Kovan testnet, replace tags with:
74+
For example, using the Ropsten testnet, replace tags with:
7275

7376
- `DAEMON_HOST:DAEMON_PORT`: localhost:7000
74-
- `https://JSON_RPC_ENDPOINT`: https://kovan.infura.io
75-
- `REGISTRY_ADDRESS`: 0xe331bf20044a5b24c1a744abc90c1fd711d2c08d
77+
- `https://JSON_RPC_ENDPOINT`: https://ropsten.infura.io
78+
- `REGISTRY_ADDRESS`: 0x5156fde2ca71da4398f8c76763c41bc9633875e4
7679
- `http://SERVICE_GRPC_HOST:SERVICE_GRPC_PORT`: http://localhost:7003
7780
- `ORGANIZATION_ID`: example-organization
7881
- `SERVICE_ID`: example-service
7982

80-
For example, using the Ropsten testnet, replace tags with:
83+
For example, using the Kovan testnet, replace tags with:
8184

8285
- `DAEMON_HOST:DAEMON_PORT`: localhost:7000
83-
- `https://JSON_RPC_ENDPOINT`: https://ropsten.infura.io
84-
- `REGISTRY_ADDRESS`: 0x5156fde2ca71da4398f8c76763c41bc9633875e4
86+
- `https://JSON_RPC_ENDPOINT`: https://kovan.infura.io
87+
- `REGISTRY_ADDRESS`: 0xe331bf20044a5b24c1a744abc90c1fd711d2c08d
8588
- `http://SERVICE_GRPC_HOST:SERVICE_GRPC_PORT`: http://localhost:7003
8689
- `ORGANIZATION_ID`: example-organization
8790
- `SERVICE_ID`: example-service
@@ -90,24 +93,33 @@ See [SingularityNet daemon configuration](https://github.com/singnet/snet-daemon
9093

9194
##### Running Service + Daemon on Host
9295

93-
* Run the script without flag to launch both `SNET Daemon` and the service
96+
* Run the script without flag to launch both `SNET Daemon` and the service. But first,
97+
download the latest `SNET Daemon` [release here](https://github.com/singnet/snet-daemon/releases).
9498

9599
```
96-
$ python3 run_example_service.py
100+
python3 run_example_service.py
97101
```
98102

99103
##### Running Service + Daemon in Docker Container
100104

101-
* Build the docker image and run a Container from it:
105+
* Build the docker image (with proper `SNET Daemon` version) and run a Container from it:
102106

103107
```
104-
$ docker build -t snet_example_service https://github.com/singnet/example-service.git#master
105-
$ export ETCD_HOST_FOLDER=$HOME/singnet/etcd/example-service/
106-
$ export ETCD_CONTAINER_FOLDER=/opt/singnet/example-service/storage-data-dir-1.etcd/
107-
$ docker run -p 7000:7000 -v $ETCD_HOST_FOLDER:$ETCD_CONTAINER_FOLDER -ti snet_example_service bash
108+
SNETD_VERSION="v0.1.7"
109+
docker build \
110+
--build-arg snetd_version=$SNETD_VERSION \
111+
-t snet_example_service \
112+
https://github.com/singnet/example-service.git#master
113+
114+
export ETCD_HOST=$HOME/.snet/etcd/example-service/
115+
export ETCD_CONTAINER=/opt/singnet/etcd/
116+
docker run \
117+
-p 7000:7000 \
118+
-v $ETCD_HOST:$ETCD_CONTAINER \
119+
-ti snet_example_service bash
108120
```
109121

110-
Note that the `$ETCD_(HOST|CONTAINER)_FOLDER` are useful to keep your service's etcd folder outside the container.
122+
Note that the `$ETCD_(HOST|CONTAINER)` are useful to keep your service's etcd folder outside the container.
111123

112124
From this point we follow the tutorial in the Docker Container's prompt.
113125

@@ -117,21 +129,24 @@ After this, run the service (with `SNET Daemon`), make sure you have the `snetd.
117129
# cat snetd.config.json
118130
{
119131
"DAEMON_END_POINT": "localhost:7000",
120-
"ETHEREUM_JSON_RPC_ENDPOINT": "https://kovan.infura.io",
132+
"ETHEREUM_JSON_RPC_ENDPOINT": "https://ropsten.infura.io",
121133
"IPFS_END_POINT": "http://ipfs.singularitynet.io:80",
122-
"REGISTRY_ADDRESS_KEY": "0xe331bf20044a5b24c1a744abc90c1fd711d2c08d",
134+
"REGISTRY_ADDRESS_KEY": "0x5156fde2ca71da4398f8c76763c41bc9633875e4",
123135
"PASSTHROUGH_ENABLED": true,
124136
"PASSTHROUGH_ENDPOINT": "http://localhost:7003",
125-
"ORGANIZATION_ID": "example-organization",
126-
"SERVICE_ID": "example-service",
137+
"ORGANIZATION_ID": "my-organization",
138+
"SERVICE_ID": "my-service",
139+
"PAYMENT_CHANNEL_STORAGE_SERVER": {
140+
"DATA_DIR": "/opt/singnet/etcd/"
141+
},
127142
"LOG": {
128143
"LEVEL": "debug",
129144
"OUTPUT": {
130145
"TYPE": "stdout"
131146
}
132147
}
133148
}
134-
# python3 run_example_service.py &
149+
# python3 run_example_service.py --daemon-config snetd.config.json &
135150
```
136151

137152
### Testing

‎run_example_service.py‎

Lines changed: 54 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,12 @@
11
import sys
2+
import os
3+
import signal
4+
import time
25
import subprocess
36
import logging
47
import pathlib
8+
import glob
9+
import json
510
import argparse
611

712
from service import registry
@@ -13,25 +18,35 @@
1318
def main():
1419
parser = argparse.ArgumentParser(description="Run services")
1520
parser.add_argument("--no-daemon", action="store_false", dest="run_daemon", help="do not start the daemon")
21+
parser.add_argument("--daemon-config",
22+
dest="daemon_config",
23+
help="Path of daemon configuration file, without config it won't be started",
24+
required=False
25+
)
26+
parser.add_argument("--ssl", action="store_true", dest="run_ssl", help="start the daemon with SSL")
1627
args = parser.parse_args()
1728
root_path = pathlib.Path(__file__).absolute().parent
18-
29+
1930
# All services modules go here
2031
service_modules = ["service.example_service"]
21-
32+
2233
# Call for all the services listed in service_modules
23-
all_p = start_all_services(root_path, service_modules, args.run_daemon)
24-
25-
# Wait for all subprocesses
34+
all_p = start_all_services(root_path, service_modules, args.run_daemon, args.daemon_config, args.run_ssl)
35+
36+
# Continuous checking all subprocess
2637
try:
27-
for p in all_p:
28-
p.wait()
38+
while True:
39+
for p in all_p:
40+
p.poll()
41+
if p.returncode and p.returncode != 0:
42+
kill_and_exit(all_p)
43+
time.sleep(1)
2944
except Exception as e:
3045
log.error(e)
3146
raise
3247

3348

34-
def start_all_services(cwd, service_modules, run_daemon):
49+
def start_all_services(cwd, service_modules, run_daemon, daemon_config, run_ssl):
3550
"""
3651
Loop through all service_modules and start them.
3752
For each one, an instance of Daemon "snetd" is created.
@@ -41,32 +56,59 @@ def start_all_services(cwd, service_modules, run_daemon):
4156
for i, service_module in enumerate(service_modules):
4257
service_name = service_module.split(".")[-1]
4358
log.info("Launching {} on port {}".format(str(registry[service_name]), service_module))
44-
all_p += start_service(cwd, service_module, run_daemon)
59+
all_p += start_service(cwd, service_module, run_daemon, daemon_config, run_ssl)
4560
return all_p
4661

4762

48-
def start_service(cwd, service_module, run_daemon):
63+
def start_service(cwd, service_module, run_daemon, daemon_config, run_ssl):
4964
"""
5065
Starts SNET Daemon ("snetd") and the python module of the service
5166
at the passed gRPC port.
5267
"""
68+
69+
def add_ssl_configs(conf):
70+
"""Add SSL keys to snetd.config.json"""
71+
with open(conf, "r") as f:
72+
snetd_configs = json.load(f)
73+
snetd_configs["ssl_cert"] = "/opt/singnet/.certs/fullchain.pem"
74+
snetd_configs["ssl_key"] = "/opt/singnet/.certs/privkey.pem"
75+
with open(conf, "w") as f:
76+
json.dump(snetd_configs, f, sort_keys=True, indent=4)
77+
5378
all_p = []
5479
if run_daemon:
55-
all_p.append(start_snetd(str(cwd)))
80+
if daemon_config:
81+
all_p.append(start_snetd(str(cwd), daemon_config))
82+
else:
83+
for idx, config_file in enumerate(glob.glob("./snetd_configs/*.json")):
84+
if run_ssl:
85+
add_ssl_configs(config_file)
86+
all_p.append(start_snetd(str(cwd), config_file))
5687
service_name = service_module.split(".")[-1]
5788
grpc_port = registry[service_name]["grpc"]
5889
p = subprocess.Popen([sys.executable, "-m", service_module, "--grpc-port", str(grpc_port)], cwd=str(cwd))
5990
all_p.append(p)
6091
return all_p
6192

6293

63-
def start_snetd(cwd):
94+
def start_snetd(cwd, config_file=None):
6495
"""
6596
Starts the Daemon "snetd":
6697
"""
6798
cmd = ["snetd", "serve"]
99+
if config_file:
100+
cmd = ["snetd", "serve", "--config", config_file]
68101
return subprocess.Popen(cmd, cwd=str(cwd))
69102

70103

104+
def kill_and_exit(all_p):
105+
for p in all_p:
106+
try:
107+
os.kill(p.pid, signal.SIGTERM)
108+
except Exception as e:
109+
log.error(e)
110+
exit(1)
111+
112+
71113
if __name__ == "__main__":
72114
main()

‎snetd_configs/snetd.mainnet.json‎

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
{
2+
"daemon_end_point": "0.0.0.0:7002",
3+
"ethereum_json_rpc_endpoint": "https://mainnet.infura.io",
4+
"ipfs_end_point": "http://ipfs.singularitynet.io:80",
5+
"registry_address_key": "0xb3180a92e210b45e3447976a412ac0df859febaf",
6+
"passthrough_enabled": true,
7+
"passthrough_endpoint": "http://localhost:7003",
8+
"organization_id": "snet",
9+
"service_id": "example-service",
10+
11+
"payment_channel_storage_server": {
12+
"id": "storage-mainnet",
13+
"host": "127.0.0.1",
14+
"client_port": 2379,
15+
"peer_port": 2380,
16+
"token": "unique-token",
17+
"cluster": "storage-mainnet=http://127.0.0.1:2380",
18+
"data_dir": "/opt/singnet/etcd/mainnet",
19+
"enabled": true
20+
},
21+
22+
"payment_channel_storage_client": {
23+
"connection_timeout": "5s",
24+
"request_timeout": "3s",
25+
"endpoints": ["http://127.0.0.1:2379"]
26+
},
27+
28+
"log": {
29+
"level": "debug",
30+
"output": {
31+
"current_link": "./snetd-mainnet.log",
32+
"file_pattern": "./snetd-mainnet.%Y%m%d.log",
33+
"rotation_count": 0,
34+
"rotation_time_in_sec": 86400,
35+
"type": "file"
36+
}
37+
}
38+
}

‎snetd_configs/snetd.ropsten.json‎

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
{
2+
"daemon_end_point": "0.0.0.0:7052",
3+
"ethereum_json_rpc_endpoint": "https://ropsten.infura.io",
4+
"ipfs_end_point": "http://ipfs.singularitynet.io:80",
5+
"registry_address_key": "0x5156fde2ca71da4398f8c76763c41bc9633875e4",
6+
"passthrough_enabled": true,
7+
"passthrough_endpoint": "http://localhost:7003",
8+
"organization_id": "snet",
9+
"service_id": "example-service",
10+
11+
"payment_channel_storage_server": {
12+
"id": "storage-ropsten",
13+
"host": "127.0.0.1",
14+
"client_port": 2381,
15+
"peer_port": 2382,
16+
"token": "unique-token",
17+
"cluster": "storage-ropsten=http://127.0.0.1:2382",
18+
"data_dir": "/opt/singnet/etcd/ropsten",
19+
"enabled": true
20+
},
21+
22+
"payment_channel_storage_client": {
23+
"connection_timeout": "5s",
24+
"request_timeout": "3s",
25+
"endpoints": ["http://127.0.0.1:2381"]
26+
},
27+
28+
"log": {
29+
"level": "debug",
30+
"output": {
31+
"current_link": "./snetd-ropsten.log",
32+
"file_pattern": "./snetd-ropsten.%Y%m%d.log",
33+
"rotation_count": 0,
34+
"rotation_time_in_sec": 86400,
35+
"type": "file"
36+
}
37+
}
38+
}

0 commit comments

Comments
 (0)