feat: raise error

[shaohuzhang1]

feat: raise error

[f2c-ci-robot]

Adding the "do-not-merge/release-note-label-needed" label because no release-note block was detected, please follow our release note process to remove it.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[f2c-ci-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[shaohuzhang1]

The provided code has a minor issue with error handling. The try block only handles specific exceptions, but it doesn't catch all possible errors that might occur while streaming the response from the model.

Potential Issues and Optimization Suggestions:

1. Error Handling Scope: Consider catching more general exceptions (Exception) to ensure that any unexpected errors are handled gracefully.

2. Stream Response: Ensure that you are calling to_stream_response_simple correctly when returning the processed output. This method should handle writing to stdout or another appropriate stream.

Suggested Changes:

@@ -187,14 +187,16 @@ def generate_prompt(self, instance: dict):

     def process():
         model = get_model_instance_by_model_workspace_id(model_id=model_id,
                                                     workspace_id=workspace_id,
-                                                    **application.model_params_setting)
-        for r in model.stream([SystemMessage(content=system_content),
                                *[HumanMessage(content=m.get('content')) if m.get('role') == 'user'
                                 else AIMessage(content=m.get('content'))
                                 for m in messages]]):
-            yield 'data: ' + json.dumps({'content': r.content}) + '\n\n'
-
+            try:
+                for i, r in enumerate(model.stream([SystemMessage(content=system_content),
                                                  *messages])):
+                    yield f"data: {json.dumps(r)},{i}\n\n"
+            except Exception as e:
+                yield "data: {" \
+                        f"\"error\": \"{str(e)}\"" + \
+                       "}\n\n"
 
     return to_stream_response_simple(process())

Explanation of Changes:

• Catching More Exceptions: Changed except Exception as e: to catch all types of exceptions.
• Iterating with Indices (Optional): Added an index variable i inside the loop to differentiate between responses; this is purely optional and can be skipped depending on your application's needs.

These changes improve robustness and make sure that any unforeseen errors during the streaming process result in user-friendly feedback rather than termination of the flow. Make sure to adjust the logging mechanism or other monitoring aspects as needed to address actual issues encountered in production environments.