-
Notifications
You must be signed in to change notification settings - Fork 32
File Receiving Interface. #2
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Closed
Closed
Changes from 7 commits
Commits
Show all changes
8 commits
Select commit
Hold shift + click to select a range
b972f97
FRI added
amit-62 dd48dbc
Delete fri directory
amit-62 cbe24cc
FRI added
amit-62 1bdb673
Delete fri directory
amit-62 a0b4ba8
fri added
amit-62 54d73c7
Dockerfile changed
amit-62 55e97c7
FRI Updated
amit-62 9c8e23a
FRI updated
amit-62 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,9 @@ | ||
| FROM centos/python-36-centos7 | ||
| COPY requirements.txt /tmp/requirements.txt | ||
| RUN pip install -r /tmp/requirements.txt | ||
| WORKDIR /fri | ||
| COPY server /fri | ||
| USER root | ||
| RUN useradd fri && chown -R fri /fri | ||
| USER fri | ||
| CMD ["gunicorn", "--timeout=180", "--workers=4", "--bind=0.0.0.0:8080", "--access-logfile=-", "main:app"] |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,133 @@ | ||
| # The Control-Core FRI for Closed-Loop Neuromodulation Control Systems | ||
|
|
||
| The Control-Core File Receiving Interface (FRI) is built with is Python-3.10. It is the core component that makes the distributed executions a reality in the Control-Core framework. | ||
|
|
||
|
|
||
| # Building FRI Container | ||
|
|
||
| Connect to the Server VM, assuming x.x.x.x to be the IP address of your server. | ||
| ```` | ||
| $ ssh -i "controlcore.pem" ubuntu@x.x.x.x | ||
| ```` | ||
| Perform Git clone if this is the first time you are configuring the Server | ||
| ```` | ||
| $ git clone git@github.com/ControlCore-Project/concore.git | ||
| ```` | ||
|
|
||
| First build the Docker Container of the FRI. | ||
| ```` | ||
| $ git pull | ||
|
|
||
| $ sudo docker build -t fri . | ||
| ```` | ||
|
|
||
| # Running Control-Core FRI with Kong as containers | ||
|
|
||
| If you are already running FRI, make sure to stop and clear existing FRI container as it is likely conflict with the port. If there is Kong gateway running in default ports, stop and clear it too. | ||
| ```` | ||
| $ docker stop fri | ||
| $ docker rm fri | ||
| $ docker stop kong | ||
| $ docker rm kong | ||
| ```` | ||
|
|
||
| Start and configure Cassandra container for Kong API. | ||
| ```` | ||
| $ docker run -d --name kong-database \ | ||
| -p 9042:9042 \ | ||
| cassandra:3 | ||
|
|
||
|
|
||
| $ docker run --rm \ | ||
| --link kong-database:kong-database \ | ||
| -e "KONG_DATABASE=cassandra" \ | ||
| -e "KONG_PG_HOST=kong-database" \ | ||
| -e "KONG_PG_USER=kong" \ | ||
| -e "KONG_PG_PASSWORD=kong" \ | ||
| -e "KONG_CASSANDRA_CONTACT_POINTS=kong-database" \ | ||
| kong kong migrations bootstrap | ||
| ```` | ||
|
|
||
| Start Kong | ||
| ```` | ||
| $ docker run -d --name kong \ | ||
| --link kong-database:kong-database \ | ||
| -e "KONG_DATABASE=cassandra" \ | ||
| -e "KONG_PG_HOST=kong-database" \ | ||
| -e "KONG_PG_PASSWORD=kong" \ | ||
| -e "KONG_CASSANDRA_CONTACT_POINTS=kong-database" \ | ||
| -e "KONG_PROXY_ACCESS_LOG=/dev/stdout" \ | ||
| -e "KONG_ADMIN_ACCESS_LOG=/dev/stdout" \ | ||
| -e "KONG_PROXY_ERROR_LOG=/dev/stderr" \ | ||
| -e "KONG_ADMIN_ERROR_LOG=/dev/stderr" \ | ||
| -e "KONG_ADMIN_LISTEN=0.0.0.0:8001, 0.0.0.0:8444 ssl" \ | ||
| -p 80:8000 \ | ||
| -p 8443:8443 \ | ||
| -p 8001:8001 \ | ||
| -p 8444:8444 \ | ||
| kong | ||
| ```` | ||
|
|
||
| Start FRI container | ||
| ```` | ||
| $ nohup sudo docker run --name fri -p 8090:8081 fri > controlcore.out & | ||
| ```` | ||
|
|
||
| Delete if there is a previously configured Kong service. If not, skip this step. First you need to find the ID-VALUE for the route with a GET command before deleting the route and service. | ||
| ```` | ||
| $ curl -X GET "http://localhost:8001/services/fri/routes" | ||
| ```` | ||
| Use the ID output from above to issue the delete command as below (issue this only if you have a previous conflicting service definiton in kong. Otherwise, skip this step): | ||
| ```` | ||
| $ curl -X DELETE "http://localhost:8001/services/fri/routes/ID-VALUE" | ||
|
|
||
| $ curl -X DELETE "http://localhost:8001/services/fri/" | ||
| ```` | ||
|
|
||
| Define Kong Service and Route. | ||
|
|
||
| First Configure a Kong service, replacing the variable "private-ip" with the private IP address of your server below. | ||
| ```` | ||
| $ curl -i -X POST --url http://localhost:8001/services/ --data 'name=fri' --data 'url=http://private-ip:8090' | ||
| ```` | ||
| Then configure route to the service | ||
| ```` | ||
| $ curl -i -X POST --url http://localhost:8001/services/fri/routes --data 'paths=/' | ||
| ```` | ||
|
|
||
| Now, controlcore.org is routed through the Kong APIs. | ||
|
|
||
|
|
||
| # Troubleshooting the FRI | ||
|
|
||
| Connect to the Server VM | ||
| ```` | ||
| $ ssh -i "controlcore.pem" ubuntu@x.x.x.x | ||
| ```` | ||
| Check the Server logs. | ||
| ```` | ||
| $ tail -f controlcore.out | ||
| ```` | ||
| or | ||
| ```` | ||
| $ sudo docker logs fri -f | ||
| ```` | ||
| Find the FRI docker container | ||
| ```` | ||
| $ sudo docker ps | ||
| ```` | ||
| CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES | ||
| dfdd3b3d3308 fri "python main.py" 38 minutes ago Up 38 minutes 0.0.0.0:80->80/tcp fri | ||
|
|
||
| Access the container | ||
| ```` | ||
| $ sudo docker exec -it dfdd /bin/bash | ||
| ```` | ||
|
|
||
|
|
||
|
|
||
| # Citing the CONTROL-CORE FRI | ||
|
|
||
| If you use the CONTROL-CORE FRI in your research, please cite the below paper: | ||
|
|
||
| * Kathiravelu, P., Arnold, M., Fleischer, J., Yao, Y., Awasthi, S., Goel, A. K., Branen, A., Sarikhani, P., Kumar, G., Kothare, M. V., and Mahmoudi, B. **CONTROL-CORE: A Framework for Simulation and Design of Closed-Loop Peripheral Neuromodulation Control Systems**. In IEEE Access. March 2022. https://doi.org/10.1109/ACCESS.2022.3161471 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| print("amit") |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,2 @@ | ||
| Flask | ||
| gunicorn==20.1.0 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,49 @@ | ||
| from flask import Flask, request, jsonify, render_template | ||
| from werkzeug.utils import secure_filename | ||
| import os | ||
|
|
||
|
|
||
| UPLOAD_FOLDER = "./uploaded_files" | ||
|
|
||
| if not os.path.exists(UPLOAD_FOLDER): | ||
| os.makedirs(UPLOAD_FOLDER) | ||
|
|
||
| app = Flask(__name__) | ||
| app.secret_key = "secret key" | ||
| app.config['UPLOAD_FOLDER'] = UPLOAD_FOLDER | ||
|
|
||
|
|
||
| @app.route('/multiple-files-upload', methods=['POST']) | ||
| def upload_file(): | ||
| if 'files[]' not in request.files: | ||
| resp = jsonify({'message' : 'No file in the request'}) | ||
| resp.status_code = 400 | ||
| return resp | ||
|
|
||
| files = request.files.getlist('files[]') | ||
|
|
||
| errors = {} | ||
| success = False | ||
|
|
||
| for file in files: | ||
| if file: | ||
| filename = secure_filename(file.filename) | ||
| file.save(os.path.join(app.config['UPLOAD_FOLDER'], filename)) | ||
| success = True | ||
|
|
||
| if success and errors: | ||
| errors['message'] = 'File(s) successfully uploaded' | ||
| resp = jsonify(errors) | ||
| resp.status_code = 500 | ||
| return resp | ||
| if success: | ||
| resp = jsonify({'message' : 'Files successfully uploaded'}) | ||
| resp.status_code = 201 | ||
| return resp | ||
| else: | ||
| resp = jsonify(errors) | ||
| resp.status_code = 500 | ||
| return resp | ||
|
|
||
| if __name__ == "__main__": | ||
| app.run(host="0.0.0.0") |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,22 @@ | ||
| #!/bin/bash | ||
|
|
||
| SERVICES=('init' 'ctl' 'pm' 'cleanup') | ||
| for service in "${SERVICES[@]}" | ||
| do | ||
| name="name="$service | ||
| path="paths=/"$service | ||
| service_path="url=http://private-ip:8090/"$service | ||
| route_path="http://localhost:8001/services/"$service"/routes" | ||
| plugin_path="http://localhost:8001/services/"$service"/plugins" | ||
| echo "Name is, $name" | ||
| echo "Path is, $path" | ||
| echo "Service path is, $service_path" | ||
| echo "Route path is, $route_path" | ||
| echo "Plugin path is, $plugin_path" | ||
| echo "Creating Service, $service..." | ||
| curl -i -X POST --url http://localhost:8001/services/ --data $name --data $service_path | ||
| echo "Creating route for $service..." | ||
| curl -i -X POST --url $route_path --data $path | ||
| echo "Creating apikey-based authentication for $service..." | ||
| curl -X POST $plugin_path --data "name=key-auth" --data "config.key_names=apikey" | ||
| done | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,13 @@ | ||
| import requests | ||
|
|
||
| url = "http://127.0.0.1:5000/multiple-files-upload" | ||
|
|
||
| payload={} | ||
| files=[ | ||
| ('files[]',('example.py',open('/home/amit/Desktop/concore/flaskApi/example.py','rb'),'application/octet-stream')) | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Pls make it a relative path and get the file from the current directory. No hard-coded locations please. They will simply not work outside your laptop. |
||
| ] | ||
| headers = {} | ||
|
|
||
| response = requests.request("POST", url, headers=headers, data=payload, files=files) | ||
|
|
||
| print(response.text) | ||
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This should be the methods you are defining.
So, it should be,
SERVICES=('upload_file' 'execute')
and any method you introduce in main.py