You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/source/learners/llm.rst
+2-2Lines changed: 2 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -93,7 +93,7 @@ You will see a evaluations results.
93
93
94
94
Pipeline Usage
95
95
-----------------------
96
-
The OntoLearner package also offers a streamlined ``LearnerPipeline`` class that simplifies the entire process of initializing, training, predicting, and evaluating a RAG setup into a single call. This is particularly useful for rapid experimentation and deployment.
96
+
The OntoLearner package also offers a streamlined ``LearnerPipeline`` class that simplifies initialization, training, prediction, and evaluation into a single call. In this section, we run the pipeline in **LLM-only** mode by setting ``llm_id`` only.
97
97
98
98
.. code-block:: python
99
99
@@ -113,7 +113,7 @@ The OntoLearner package also offers a streamlined ``LearnerPipeline`` class that
113
113
114
114
# Set up the learner pipeline using a lightweight instruction-tuned LLM
115
115
pipeline = LearnerPipeline(
116
-
llm_id='Qwen/Qwen2.5-0.5B-Instruct', #Small-scale LLM for reasoning over term-type assignments
StandardizedPrompting# Standard prompting strategy across tasks
29
-
evaluation_report
28
+
StandardizedPrompting,# Standard prompting strategy across tasks
29
+
evaluation_report,
30
30
)
31
31
32
32
# Load the AgrO ontology (an agricultural domain ontology)
@@ -99,16 +99,24 @@ To build a RAG model, you first initialize its constituent parts: an LLM learner
99
99
100
100
Pipeline Usage
101
101
---------------------
102
-
Similar to LLM and Retrieval learner, RAG Learner is also callable via streamlined ``LearnerPipeline`` class that simplifies the entire learning process.
102
+
Similar to LLM and Retrieval learners, RAG is callable via ``LearnerPipeline``, you can run RAG in two equivalent ways:
103
103
104
-
You initialize the ``LearnerPipeline`` by directly providing the ``retriever_id``, ``llm_id``, and other parameters like ``hf_token``, ``batch_size``, and ``top_k`` (number of top retrievals to include in RAG prompting). Then, you simply call the ``pipeline`` instance with your ``train_data``, ``test_data``, specify ``evaluate=True`` to compute metrics, and define the ``task`` (e.g., `'term-typing'`).
104
+
1. Provide both ``retriever_id`` and ``llm_id`` (pipeline auto-composes an ``AutoRAGLearner``).
105
+
2. Provide a prebuilt ``rag`` learner object for custom configurations.
105
106
106
107
.. code-block:: python
107
108
108
-
# Import core modules from the OntoLearner library
109
-
from ontolearner import LearnerPipeline, AgrO, train_test_split
109
+
from ontolearner import (
110
+
LearnerPipeline,
111
+
AutoLLMLearner,
112
+
AutoRetrieverLearner,
113
+
AutoRAGLearner,
114
+
LabelMapper,
115
+
StandardizedPrompting,
116
+
AgrO,
117
+
train_test_split,
118
+
)
110
119
111
-
# Load the AgrO ontology, which contains concepts related to wines, their properties, and categories
112
120
ontology = AgrO()
113
121
ontology.load() # Load entities, types, and structured term annotations from the ontology
Copy file name to clipboardExpand all lines: docs/source/learners/retrieval.rst
+9-2Lines changed: 9 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -81,7 +81,7 @@ When working with large contexts, the retriever model may encounter memory issue
81
81
Pipeline Usage
82
82
-----------------------
83
83
84
-
Similar to LLM learner, Retrieval Learner is also callable via streamlined ``LearnerPipeline`` class that simplifies the entire process learning.
84
+
Similar to the LLM learner, Retrieval learner is also callable via the streamlined ``LearnerPipeline`` class. In this section we use **retriever-only** mode by providing ``retriever_id`` only.
85
85
86
86
.. code-block:: python
87
87
@@ -100,7 +100,7 @@ Similar to LLM learner, Retrieval Learner is also callable via streamlined ``Lea
100
100
)
101
101
102
102
# Initialize the learning pipeline using a dense retriever
103
-
# This configuration uses sentence embeddings to match similar relational contexts
103
+
# This is retriever-only mode (no LLM component)
104
104
pipeline = LearnerPipeline(
105
105
retriever_id='sentence-transformers/all-MiniLM-L6-v2', # Hugging Face model ID for retrieval
106
106
batch_size=10, # Number of samples to process per batch (if batching is enabled internally)
@@ -125,6 +125,10 @@ Similar to LLM learner, Retrieval Learner is also callable via streamlined ``Lea
125
125
# Print the full output dictionary (includes predictions)
126
126
print(outputs)
127
127
128
+
.. note::
129
+
130
+
For RAG with ``LearnerPipeline`` see: `https://ontolearner.readthedocs.io/learners/rag.html <https://ontolearner.readthedocs.io/learners/rag.html>`_.
131
+
128
132
.. hint::
129
133
See `Learning Tasks <https://ontolearner.readthedocs.io/learning_tasks/llms4ol.html>`_ for possible tasks within Learners.
130
134
@@ -372,6 +376,9 @@ Here the ``LLMAugmentedRetrieverLearner`` is the high-level wrapper that orchest
Copy file name to clipboardExpand all lines: docs/source/quickstart.rst
+33-1Lines changed: 33 additions & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -137,7 +137,11 @@ To alighn with machine learning follow, once the ontology is loaded, and ontolog
137
137
)
138
138
139
139
140
-
Once the data is split into training and testing sets, you can apply learning models to the ontology learning tasks. OntoLearner supports multiple modeling approaches, including retrieval-based methods, Large Language Model (LLM)-based techniques, and Retrieval-Augmented Generation (RAG) strategies. The ``LearnerPipeline`` within OntoLearner is designed for ease of use, abstracting away the complexities of loading models and preparing datasets or data loaders. You can configure the pipeline with your choice of LLMs, retrievers, or RAG components.
140
+
Once the data is split into training and testing sets, you can apply learning models to the ontology learning tasks. OntoLearner supports multiple modeling approaches, including retrieval-based methods, Large Language Model (LLM)-based techniques, and Retrieval-Augmented Generation (RAG) strategies. The ``LearnerPipeline`` supports all three modes:
141
+
142
+
- Retriever-only: set ``retriever_id``
143
+
- LLM-only: set ``llm_id``
144
+
- RAG: set both ``retriever_id`` + ``llm_id`` for AutoRAGLearner. For prebuild RAG pass ``rag`` learner.
141
145
142
146
In the example below, we configure a RAG-based learner by specifying the Qwen LLM (`Qwen/Qwen2.5-0.5B-Instruct <https://huggingface.co/Qwen/Qwen2.5-0.5B-Instruct>`_) and a retriever based on a sentence-transformer model (`all-MiniLM-L6-v2 <https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2>`_):
143
147
@@ -165,6 +169,34 @@ In the example below, we configure a RAG-based learner by specifying the Qwen LL
165
169
- ``llm_id``: The instruction-following language model used to generate candidate outputs.
166
170
- ``top_k``: Number of retrieved examples passed to the LLM (used in RAG setup).
167
171
- ``hf_token``: Required for loading gated models from Hugging Face.
0 commit comments