Skip to content

Commit 67a4e4e

Browse files
authored
Merge pull request #22 from rekcurd/fix/update-readme-20181221
Merged
2 parents 37fd398 + 944aae8 commit 67a4e4e

5 files changed

Lines changed: 26 additions & 439 deletions

File tree

README.md

Lines changed: 26 additions & 201 deletions
Original file line numberDiff line numberDiff line change
@@ -1,221 +1,46 @@
1-
# Drucker
1+
# Rekcurd
22

3-
[![Build Status](https://travis-ci.com/drucker/drucker.svg?branch=master)](https://travis-ci.com/drucker/drucker)
4-
[![PyPI version](https://badge.fury.io/py/drucker.svg)](https://badge.fury.io/py/drucker)
5-
[![codecov](https://codecov.io/gh/drucker/drucker/branch/master/graph/badge.svg)](https://codecov.io/gh/drucker/drucker "Non-generated packages only")
6-
[![pypi supported versions](https://img.shields.io/pypi/pyversions/drucker.svg)](https://pypi.python.org/pypi/drucker)
3+
[![Build Status](https://travis-ci.com/rekcurd/drucker.svg?branch=master)](https://travis-ci.com/rekcurd/drucker)
4+
[![PyPI version](https://badge.fury.io/py/rekcurd.svg)](https://badge.fury.io/py/rekcurd)
5+
[![codecov](https://codecov.io/gh/rekcurd/drucker/branch/master/graph/badge.svg)](https://codecov.io/gh/rekcurd/drucker "Non-generated packages only")
6+
[![pypi supported versions](https://img.shields.io/pypi/pyversions/rekcurd.svg)](https://pypi.python.org/pypi/rekcurd)
7+
8+
Rekcurd is the Project for serving ML module. This is a gRPC micro-framework and it can be used like [Flask](http://flask.pocoo.org/).
79

8-
Drucker is a framework of serving machine learning module. Drucker makes it easy to serve, manage and integrate your ML models into your existing services. Moreover, Drucker can be used on Kubernetes.
910

1011
## Parent Project
11-
https://github.com/drucker/drucker-parent
12+
https://github.com/rekcurd/drucker-parent
13+
1214

1315
## Components
14-
- [Drucker](https://github.com/drucker/drucker) (here): Serving framework for a machine learning module.
15-
- [Drucker-dashboard](https://github.com/drucker/drucker-dashboard): Management web service for the machine learning models to the drucker service.
16-
- [Drucker-client](https://github.com/drucker/drucker-client): SDK for accessing a drucker service.
17-
- [Drucker-example](https://github.com/drucker/drucker-example): Example of how to use drucker.
16+
- [Rekcurd](https://github.com/rekcurd/drucker) (here): Project for serving ML module.
17+
- [Rekcurd-dashboard](https://github.com/rekcurd/drucker-dashboard): Project for managing ML model and deploying ML module.
18+
- [Rekcurd-client](https://github.com/rekcurd/drucker-client): Project for integrating ML module.
19+
1820

1921
## Installation
2022
From source:
2123

24+
```bash
25+
$ git clone --recursive https://github.com/rekcurd/drucker.git
26+
$ cd drucker
27+
$ python setup.py install
2228
```
23-
git clone --recursive https://github.com/drucker/drucker.git
24-
cd drucker
25-
python setup.py install
26-
```
27-
28-
From [PyPi](https://pypi.org/project/drucker/) directly:
29-
30-
```
31-
pip install drucker
32-
```
33-
34-
## Example
35-
Example is available [here](https://github.com/drucker/drucker-example).
36-
37-
### Create settings.yml (Not necessary)
38-
Write your server configurations. The spec details are [here](./template/settings.yml)
39-
40-
### Create app.py
41-
Implement `Drucker` class.
42-
43-
Necessity methods are following.
44-
45-
#### load_model
46-
ML model loading method.
47-
48-
```python
49-
def load_model(self) -> None:
50-
try:
51-
self.predictor = joblib.load(self.model_path)
52-
except Exception as e:
53-
self.logger.error(str(e))
54-
self.logger.error(traceback.format_exc())
55-
self.predictor = None
56-
if not self.is_first_boot():
57-
os._exit(-1)
58-
```
59-
60-
If you need to load more than two files to your ML module, you need to create a compressed file which includes the files it requires. You can load the file like the below.
61-
62-
```python
63-
def joblib_load_from_zip(self, zip_name: str, file_name: str):
64-
with zipfile.ZipFile(zip_name, 'r') as zf:
65-
with zf.open(file_name, 'r') as zipmodel:
66-
return joblib.load(io.BufferedReader(io.BytesIO(zipmodel.read())))
67-
68-
def load_model(self) -> None:
69-
try:
70-
file_name = 'default.model'
71-
self.predictor = self.joblib_load_from_zip(self.model_path, file_name)
72-
except Exception as e:
73-
self.logger.error(str(e))
74-
self.logger.error(traceback.format_exc())
75-
self.predictor = None
76-
if not self.is_first_boot():
77-
os._exit(-1)
78-
```
79-
80-
#### predict
81-
Predicting/inferring method.
82-
83-
```python
84-
def predict(self, input: PredictLabel, option: dict = None) -> PredictResult:
85-
try:
86-
label_predict = self.predictor.predict(
87-
np.array([input], dtype='float64')).tolist()
88-
return PredictResult(label_predict, [1] * len(label_predict), option={})
89-
except Exception as e:
90-
self.logger.error(str(e))
91-
self.logger.error(traceback.format_exc())
92-
raise e
93-
```
94-
95-
Input/output specs are below.
96-
97-
##### Input format
98-
*V* is the length of feature vector.
99-
100-
|Field |Type |Description |
101-
|:---|:---|:---|
102-
|input <BR>(required) |One of below<BR>- string<BR>- bytes<BR>- string[*V*]<BR>- int[*V*]<BR>- double[*V*] |Input data for inference.<BR>- "Nice weather." for a sentiment analysis.<BR>- PNG file for an image transformation.<BR>- ["a", "b"] for a text summarization.<BR>- [1, 2] for a sales forcast.<BR>- [0.9, 0.1] for mnist data. |
103-
|option |string| Option field. Must be json format. |
104-
105-
The "option" field needs to be a json format. Any style is Ok but we have some reserved fields below.
106-
107-
|Field |Type |Description |
108-
|:---|:---|:---|
109-
|suppress_log_input |bool |True: NOT print the input and output to the log message. <BR>False (default): Print the input and outpu to the log message.
11029

111-
##### Output format
112-
*M* is the number of classes. If your algorithm is a binary classifier, you set *M* to 1. If your algorithm is a multi-class classifier, you set *M* to the number of classes.
30+
From [PyPi](https://pypi.org/project/rekcurd/) directly:
11331

114-
|Field |Type |Description |
115-
|:---|:---|:---|
116-
|label<BR>(required) |One of below<BR> -string<BR> -bytes<BR> -string[*M*]<BR> -int[*M*]<BR> -double[*M*] |Result of inference.<BR> -"positive" for a sentiment analysis.<BR> -PNG file for an image transformation.<BR> -["a", "b"] for a multi-class classification.<BR> -[1, 2] for a multi-class classification.<BR> -[0.9, 0.1] for a multi-class classification. |
117-
|score<BR>(required) |One of below<BR> -double<BR> -double[*M*] |Score of result.<BR> -0.98 for a binary classification.<BR> -[0.9, 0.1] for a multi-class classification. |
118-
|option |string |Option field. Must be json format. |
119-
120-
#### evaluate (TODO)
121-
Evaluating method.
122-
123-
This method is under construction.
124-
125-
##### Input format
126-
|Field |Type |Description |
127-
|:---|:---|:---|
128-
|file<BR>(required) |bytes |Data for performance check |
129-
130-
##### Output format
131-
*N* is the number of evaluation data. *M* is the number of classes. If your algorithm is a binary classifier, you set *M* to 1. If your algorithm is a multi-class classifier, you set *M* to the number of classes.
132-
133-
|Field |Type |Description |
134-
|:---|:---|:---|
135-
|num<BR>(required)|int |Number of evaluation data. |
136-
|accuracy<BR>(required) |double |Accuracy. |
137-
|precision<BR>(required) |double[*N*][*M*] |Precision. |
138-
|recall<BR>(required) |double[*N*][*M*] |Recall. |
139-
|fvalue<BR>(required) |double[*N*][*M*] |F1 value. |
140-
141-
### Create server.py
142-
Create a boot script.
143-
144-
```python
145-
from concurrent import futures
146-
import grpc
147-
import time
148-
149-
from drucker import DruckerDashboardServicer, DruckerWorkerServicer
150-
from drucker.logger import JsonSystemLogger, JsonServiceLogger
151-
from drucker.protobuf import drucker_pb2_grpc
152-
from app import MyApp
153-
154-
_ONE_DAY_IN_SECONDS = 60 * 60 * 24
155-
156-
157-
def serve():
158-
app = MyApp("./settings.yml")
159-
system_logger = JsonSystemLogger(app.config)
160-
service_logger = JsonServiceLogger(app.config)
161-
system_logger.info("Wake-up drucker worker.")
162-
163-
server = grpc.server(futures.ThreadPoolExecutor(max_workers=10))
164-
165-
drucker_pb2_grpc.add_DruckerDashboardServicer_to_server(
166-
DruckerDashboardServicer(logger=system_logger, app=app), server)
167-
drucker_pb2_grpc.add_DruckerWorkerServicer_to_server(
168-
DruckerWorkerServicer(logger=service_logger, app=app), server)
169-
server.add_insecure_port("[::]:{0}".format(app.config.SERVICE_PORT))
170-
server.start()
171-
try:
172-
while True:
173-
time.sleep(_ONE_DAY_IN_SECONDS)
174-
except KeyboardInterrupt:
175-
system_logger.info("Shutdown drucker worker.")
176-
server.stop(0)
177-
178-
179-
if __name__ == '__main__':
180-
serve()
32+
```bash
33+
$ pip install rekcurd
18134
```
18235

183-
### Create logger (Not necessary)
184-
If you want to use your own format logger, please implement the drucker [logger interface class](./drucker/logger/logger_interface.py).
185-
186-
### Create start.sh
187-
Create a boot script.
188-
189-
```sh
190-
#!/usr/bin/env bash
191-
192-
ECHO_PREFIX="[drucker example]: "
36+
## How to use
37+
Example code is available [here](https://github.com/rekcurd/drucker-example).
19338

194-
set -e
195-
set -u
196-
197-
echo "$ECHO_PREFIX Start.."
198-
199-
pip install -r requirements.txt
200-
python ./server.py
20139

40+
## Unittest
20241
```
203-
204-
### Run
42+
$ python -m unittest
20543
```
206-
$ sh start.sh
207-
```
208-
209-
### Test
210-
```
211-
$ python -m unittest drucker/test/test_worker_servicer.py
212-
$ python -m unittest drucker/test/test_dashboard_servicer.py
213-
```
214-
215-
## Drucker on Kubernetes
216-
Drucker can be run on Kubernetes and can be managed by Drucker dashboard.
217-
218-
You must read the followings.
21944

220-
1. https://github.com/drucker/drucker-parent/tree/master/docs/Installation.md
221-
1. https://github.com/drucker/drucker-dashboard/README.md
45+
## Kubernetes support
46+
Rekcurd can be run on Kubernetes. See [here](https://github.com/rekcurd/drucker-parent).

template/app.py

Lines changed: 0 additions & 122 deletions
This file was deleted.

0 commit comments

Comments
 (0)