Hi i deployed the following scoring script on a managed online endpoint on azure ml using v2 but it fails to return a streaming response. However running the Azure ML inference server locally does ensure I get a streaming response back. Isnt the local deployment meants to replicate the deployed version in behavior? Is Azure ML Inference server capable of streaming responses?
`
from azureml.contrib.services.aml_response import AMLResponse
import time
def init():
def run(request):
def my_python_tool(request):
for word in request.split():
if word != "100}":
time.sleep(2)
print(word)
yield str.encode(word + " \n")
elif word == "100}":
print("last word")
yield str.encode("100} \n")
print(request)
response = AMLResponse(my_python_tool(request), 200)
response.headers["Content-Type"] = "text/event-stream; charset=utf-8"
response.default_mimetype = "text/event-stream"
response.headers["X-Accel-Buffering"] = "no"
response.headers["Transfer-Encoding"] = "chunked"
response.headers["Cache-Control"] = "no-cache"
response.implicit_sequence_conversion = True
response.stream.response.cache_control.no_cache = True
response.stream.response.cache_control.no_store = True
return response
`
Hi i deployed the following scoring script on a managed online endpoint on azure ml using v2 but it fails to return a streaming response. However running the Azure ML inference server locally does ensure I get a streaming response back. Isnt the local deployment meants to replicate the deployed version in behavior? Is Azure ML Inference server capable of streaming responses?
`
from azureml.contrib.services.aml_response import AMLResponse
import time
def init():
def run(request):
`