Merge pull request #2019 from kili-technology/revert-2017-feature/lab-3584-aakd-i-see-onsubmit-and-onreview-handlers-removed-from

Revert "feat(lab-3584): update documentation to fully remove onSubmit and onR…"

Author: florianlega

docs/sdk/plugins.md

@@ -19,19 +19,15 @@ plugin_folder
     |__ helper.py
 ```
 
+The plugin you are going to upload has to contain a `class PluginHandler(PluginCore)` (in the case of the module type plugin it has to be inside `main.py`) that implements two methods for the different types of events:
 
-> **Note**: `on_submit` and `on_review` handlers are deprecated. Please use `on_event` instead. If your plugin was already uploaded with thoses handlers, it will still work. If you want the same behavior as `on_submit` and `on_review`, when you upload your plugin, you can use `event_matcher=["labels.created.submit"]` for `on_submit` and `event_matcher=["labels.created.review"]` for `on_review` events.
+- `on_submit`
+- `on_review`
 
+These methods have a predefined set of parameters:
 
-The plugin you are going to upload has to contain a `class PluginHandler(PluginCore)` (in the case of the module type plugin it has to be inside `main.py`) that implements one method for the different types of events:
-
-- `on_event`
-
-These methods have a predefined set of parameter:
-
-- the `payload` submitted (a dictionary containing the fields for your plugins)
-    - `payload.event` the name of the event (ex: for submit : `labels.created.submit`, for review `labels.created.review`)
-    - `payload.label` the label containing the fields of the GraphQL type [Label](https://api-docs.kili-technology.com/types/objects/label/)
+- the `label` submitted (a dictionary containing the fields of the GraphQL type [Label](https://api-docs.kili-technology.com/types/objects/label/))
+- the `asset_id` of the asset labeled
 
 You can add custom methods in your class as well.
 
@@ -57,18 +53,12 @@ class PluginHandler(PluginCore):
     def custom_method(self):
         # Do something...
 
-    def on_event(self, payload: dict) -> None: # This can replace on_review method
+    def on_review(self, label: Dict, asset_id: str) -> None:
         """Dedicated handler for Review action"""
-        event = payload.get("event")
-
-        if event == 'labels.created.review':
         # Do something...
 
-    def on_event(self, payload: dict) -> None: # This can replace on_submit method
+    def on_submit(self, label: Dict, asset_id: str) -> None:
         """Dedicated handler for Submit action"""
-        event = payload.get("event")
-
-        if event == 'labels.created.submit':
         # Do something...
 ```
 
@@ -95,22 +85,15 @@ def custom_function(label: Dict, logger: Logger):
 class PluginHandler(PluginCore):
     """Custom plugin"""
 
-    def on_event(self, payload: dict) -> None:
+    def on_submit(self, label: Dict, asset_id: str) -> None:
         """Dedicated handler for Submit action"""
-        event = payload.get("event")
-        if event == 'labels.created.submit':
-            self.logger.info("On Submit called")
-            label = payload["label"]
-            custom_function(label, self.logger)
+        self.logger.info("On Submit called")
+        custom_function(label, self.logger)
 ```
 
 ## Model for Plugins
 
 ::: kili.services.plugins.model.PluginCore
-    options:
-      filters:
-        - "!^on_review$"
-        - "!^on_submit$"
 
 ## Queries
 

docs/sdk/tutorials/plugins_development.md

@@ -0,0 +1,281 @@
+<!-- FILE AUTO GENERATED BY docs/utils.py DO NOT EDIT DIRECTLY -->
+<a href="https://colab.research.google.com/github/kili-technology/kili-python-sdk/blob/main/recipes/plugins_development.ipynb" target="_parent"><img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/></a>
+
+# How to develop and test a Kili plugin
+
+## Preliminary
+
+This notebook will teach you how to build your first plugin.
+
+A plugin is an uploaded Python script triggered by an event that you define.
+
+For instance, you can trigger a plugin when a labeler clicks on **Submit** with the `on_submit` handler.
+
+The plugin should have different methods for the different types of events:
+
+- `on_submit`
+- `on_review`
+
+These methods have a predefined set of parameters:
+
+- the `label` submitted
+- the `asset_id` of the asset labeled
+
+Some attributes are available in the class:
+
+- `self.kili`
+- `self.project_id`
+
+Therefore, the skeleton of the plugin should look like this:
+
+```python
+from kili.plugins import PluginCore
+from typing import Dict
+import numpy as np
+
+class PluginHandler(PluginCore):
+    """Custom plugin"""
+
+    def on_review(self, label: Dict, asset_id: str) -> None:
+        """Dedicated handler for Review action"""
+        # Do something...
+
+    def on_submit(self, label: Dict, asset_id: str) -> None:
+        """Dedicated handler for Submit action"""
+        # Do something...
+```
+
+Do not hesitate to reach out to us if you need more.
+
+**NB: The plugin capabilities of Kili are under active development, and compatible with version 2.125.2 and later of Kili. Don't hesitate to reach out via Github or Kili support to provide feedback.**
+
+## Instantiate Kili
+
+
+```python
+%pip install  kili
+```
+
+
+```python
+%load_ext autoreload
+%autoreload 2
+
+
+from kili.client import Kili
+
+kili = Kili(
+    # api_endpoint="https://cloud.kili-technology.com/api/label/v2/graphql",
+    # the line above can be uncommented and changed if you are working with an on-premise version of Kili
+)
+```
+
+## Develop your plugin
+
+The first step is to define the functions that will be called when the event is triggered. You will be able to iterate on these functions locally (more on that in the next section).
+
+The plugin can be defined in two ways: a single `.py` file with everything inside or a module (folder containing multiple `.py` files). In the case of the module type, a file named `main.py` needs to be at the root of the folder and will serve as the entrypoint.
+
+### 1. First option - Plugin defined in a single file
+
+This cell should be the contents of the `.py` file that you will upload as a plugin at the end.
+
+**This file should define the `PluginHandler` class that will contain the proper methods.**
+
+We recommend using a modern IDE like VScode to get type hints and autocompletion on the methods.
+
+
+```python
+import numpy as np
+
+from kili.plugins import PluginCore
+
+
+def custom_function(label: dict):
+    label_id = label.get("id")
+    print(f"My custom function for review of label with id {label_id}")
+
+
+class PluginHandler(PluginCore):
+    """Custom plugin instance"""
+
+    def custom_method(self, project_id, label_id):
+        print(f"custom_method called for label {label_id}")
+        random_seed = np.random.random(1)[0]
+        if random_seed > 0.5:
+            self.logger.warning("Generating issue")
+            # Use kili for actions with self.kili
+            self.kili.create_issues(
+                project_id=project_id,
+                label_id_array=[label_id],
+                text_array=["Random issue generated for this label"],
+            )
+
+    def on_review(self, label: dict, asset_id: str) -> None:
+        """Dedicated handler for Review action"""
+        custom_function(label)
+
+    def on_submit(self, label: dict, asset_id: str) -> None:
+        """Dedicated handler for Submit action"""
+        print("On submit called")
+
+        project_id = self.project_id
+        label_id = label.get("id")
+
+        self.custom_method(project_id, label_id)
+```
+
+### 2. Second option - Plugin defined in a folder
+
+As said previously, the structure of the folder can be the following (the only constraint being the presence of the `main.py` file):
+```
+plugin_folder
+|__ main.py
+|__ other_file.py
+|__ requirements.txt
+|
+|___helpers
+    |__ helper.py
+```
+
+You can notice that you can also include a `requirements.txt` file in the folder and the necessary packages will be installed with your plugin. Don't forget to add them, since the plugin could work on your machine if you have them installed, but it won't be possible to create the plugin if there are missing dependencies.
+
+**Note:** The `requirements.txt` file can only be included for the SaaS version of the Kili platform, for on-premise deployments there is a pre-defined list of packages that can be used. For more details, see the [documentation of plugins](https://python-sdk-docs.kili-technology.com/latest/sdk/plugins/)
+
+**Important: The main.py file need to have the same skeleton as the plugin defined in a single file (presence of the class `PluginHandler`), the difference being that it can import and call functions defined in other files**
+
+Depending on where the folder is stored, there are two ways to import the plugin in order to test it:
+
+- The first way is to use a relative import (having the plugin folder and the notebook in the same folder). It is simpler and we recommend it as it will also allow the IDE to detect the correct methods and propose hints and autocompletion.
+- The second is to use an absolute path to the plugin folder
+
+#### 2.1 Relative import
+
+
+```python
+# Here replace 'plugin_folder' with the actual name of the folder
+from plugin_folder.main import PluginHandler
+```
+
+#### 2.2 Absolute path import
+
+
+```python
+import sys
+from pathlib import Path
+
+# Input the path to the plugin folder (it should include the folder), for example '/path/to/plugin_folder'
+plugin_path = "<INSERT PATH TO PLUGIN FOLDER>"
+
+module_path = str(Path(plugin_path).parent.absolute())
+
+# We are inserting the path in the system PATH to be able to import the module in the next line
+sys.path.insert(0, module_path)
+
+# In the next line replace 'plugin_folder' with the actual name of the folder
+from plugin_folder.main import PluginHandler
+```
+
+### Testing the plugin locally
+
+In this we will show you how to test your plugin locally before uploading it.
+
+
+```python
+project_id = "<PROJECT ID>"
+```
+
+Instantiate the plugin:
+
+
+```python
+my_plugin_instance = PluginHandler(kili, project_id)
+
+
+def get_label(label_id, project_id):
+    """Function to get the object Label with the same keys as it will be in the plugin"""
+    label = list(
+        kili.labels(
+            project_id=project_id,
+            label_id=label_id,
+            fields=["id", "jsonResponse", "author.id", "labelType", "createdAt", "secondsToLabel"],
+        )
+    )[0]
+
+    label["authorId"] = label["author"]["id"]
+    del label["author"]
+    return label
+```
+
+### Test the plugin run
+
+If you already have a test project with labels added, you can directly use the IDs of these labels (see the following cell). Otherwise, you can follow the *plugins_example.ipynb* notebook to create a new project and then upload an asset with an associated label.
+
+
+```python
+asset_id = "<YOUR_ASSET_ID>"
+label_id = "<YOUR_LABEL_ID>"
+```
+
+
+```python
+label = get_label(label_id=label_id, project_id=project_id)
+
+my_plugin_instance.on_submit(label=label, asset_id=asset_id)
+```
+
+### Test the plugin run on Kili
+
+When you finish debugging the code, you may want to upload it directly into Kili.
+
+Note that you might get an error if the plugin name already exists in your Kili organization.
+
+
+```python
+path_to_plugin = "path/to/my/plugin.py"
+plugin_name = "My first kili plugin"
+```
+
+
+```python
+from kili.exceptions import GraphQLError
+
+try:
+    kili.upload_plugin(path_to_plugin, plugin_name)
+except GraphQLError as error:
+    print(str(error))
+```
+
+Plugins must be activated in the project that you want them to run in. Be careful with production projects: your custom workflows or rules will also be applied
+
+
+```python
+kili.activate_plugin_on_project(plugin_name, project_id=project_id)
+```
+
+## Monitoring the plugin
+
+Plugin creation takes some time (around 5 minutes). The plugin will begin to run only after it's been fully created (if labeling events are to be triggered on this project).
+
+Additionally, you can get the logs of the runs:
+
+
+```python
+kili.get_plugin_logs(project_id=project_id, plugin_name=plugin_name)
+```
+
+You can set custom date rules for filtering your logs:
+
+
+```python
+from datetime import date, datetime
+
+dt = date.today()  # You can change this date if needed
+start_date = datetime.combine(dt, datetime.min.time())
+
+kili.get_plugin_logs(project_id=project_id, plugin_name=plugin_name, start_date=start_date)
+```
+
+## Managing your plugin
+
+There are several other methods to manage your plugins and their lifecycle. To find out more, check the plugins [tutorials](https://python-sdk-docs.kili-technology.com/latest/tutorials).

docs/sdk/tutorials/plugins_example.md

@@ -98,6 +98,7 @@ This project has one job of bounding box creation with two categories.
 
 With our plugin, we want to make sure that the labelers don't create more than one bounding box of category A.
 
+To iterate on the plugin code, you can refer to the plugins_development.ipynb notebook.
 
 ## Step 3: Write the plugin
 

docs/sdk/tutorials/plugins_library.md

@@ -4,7 +4,7 @@
 
 In this section you can find multiple examples of use-cases where our system of plugins can help you in the Kili projects.
 
-You can also refer to our [tutorial](./plugins_example.md) to develop your plugin and iterate on it locally, before uploading the final version to Kili.
+You can also refer to our [tutorial](./plugins_development.md) to develop your plugin and iterate on it locally, before uploading the final version to Kili.
 
 For further information concerning the Kili plugins, refer to the [Reference page](../plugins.md)
 

mkdocs.yml

@@ -44,7 +44,7 @@ nav:
           - PDF Assets: sdk/tutorials/importing_pdf_assets.md
           - Rich Text Assets: sdk/tutorials/import_text_assets.md
           - Multi-Layer Geospatial Assets: sdk/tutorials/importing_multilayer_geospatial_assets.md
-          - LLM Static Assets: sdk/tutorials/llm_static.md
+          - LLM Static Assets : sdk/tutorials/llm_static.md
       - Importing Labels:
           - Importing Labels: sdk/tutorials/importing_labels.md
           - OpenAI NER Pre-annotations: sdk/tutorials/ner_pre_annotations_openai.md
@@ -64,7 +64,8 @@ nav:
           - Parsing Labels: sdk/tutorials/label_parsing.md
       - LLM Dynamic Projects: sdk/tutorials/llm_dynamic.md
       - Setting Up Plugins:
-          - Developing Plugins: sdk/tutorials/plugins_example.md
+          - Developing Plugins: sdk/tutorials/plugins_development.md
+          - Plugin Example - Programmatic QA: sdk/tutorials/plugins_example.md
           - Plugins Library: sdk/tutorials/plugins_library.md
           - Webhooks: sdk/tutorials/webhooks_example.md
       - Integrations: