Skip to content

Commit 6bd3207

Browse files
HamedBabaeiHamedBabaei
authored
✨ Added text2onto learners and minor improvements (PR #299)
2 parents 1c846bd + da8e5b0 commit 6bd3207

27 files changed

Lines changed: 1851 additions & 1878 deletions

‎.github/workflows/test-os-compatibility.yml‎

Lines changed: 49 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,49 @@
1+
name: Cross-platform Compatibility Tests
2+
3+
on:
4+
push:
5+
branches: [main]
6+
pull_request:
7+
branches: [main]
8+
9+
jobs:
10+
os-compatibility-tests:
11+
runs-on: ${{ matrix.os }}
12+
strategy:
13+
fail-fast: false
14+
matrix:
15+
os: [ubuntu-latest, windows-latest, macos-latest]
16+
python-version: ["3.10"]
17+
18+
steps:
19+
- name: Checkout code
20+
uses: actions/checkout@v4
21+
22+
- name: Set up Python ${{ matrix.python-version }}
23+
uses: actions/setup-python@v5
24+
with:
25+
python-version: ${{ matrix.python-version }}
26+
27+
- name: Install Poetry
28+
shell: bash
29+
run: |
30+
curl -sSL https://install.python-poetry.org | python -
31+
echo "$HOME/.local/bin" >> $GITHUB_PATH
32+
echo "$APPDATA/Python/Scripts" >> $GITHUB_PATH
33+
34+
- name: Configure Poetry and install plugin
35+
shell: bash
36+
run: |
37+
poetry --version
38+
poetry config virtualenvs.create false
39+
poetry self add "poetry-dynamic-versioning[plugin]"
40+
41+
- name: Install dependencies
42+
shell: bash
43+
run: |
44+
poetry install --no-interaction --no-ansi
45+
46+
- name: Run tests
47+
shell: bash
48+
run: |
49+
poetry run pytest

‎CHANGELOG.md‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,10 @@
11
## Changelog
22

3+
### v1.4.11 (Janouary 5, 2026)
4+
- Add `text2onto` component for challenge learners with their documentation.
5+
- Code refactoring
6+
- OS compatibility CI/CD
7+
38
### v1.4.10 (December 8, 2025)
49
- add complexity score
510
- add documentation for metrics

‎docs/source/learners/llms4ol.rst‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -31,7 +31,7 @@ LLMs4OL is a community development initiative collocated with the International
3131
- **Text2Onto**
3232
- Extract ontological terms and types from unstructured text.
3333

34-
**ID**: ``text-to-onto``
34+
**ID**: ``text2onto``
3535

3636
**Info**: This task focuses on extracting foundational elements (Terms and Types) from unstructured text documents to build the initial structure of an ontology. It involves recognizing domain-relevant vocabulary (Term Extraction, SubTask 1) and categorizing it appropriately (Type Extraction, SubTask 2). It bridges the gap between natural language and structured knowledge representation.
3737

‎docs/source/learners/llms4ol_challenge/alexbek_learner.rst‎

Lines changed: 144 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -250,3 +250,147 @@ Learn and Predict
250250
truth = cross_learner.tasks_ground_truth_former(data=test_data, task=task)
251251
metrics = evaluation_report(y_true=truth, y_pred=predicts, task=task)
252252
print(metrics)
253+
254+
Text2Onto
255+
------------------
256+
257+
Loading Ontological Data
258+
~~~~~~~~~~~~~~~~~~~~~~
259+
260+
For the Text2Onto task, we load an ontology (via ``OM``), extract its structured content, and then generate synthetic pseudo-sentences using an LLM-backed generator (DSPy + Ollama in this example).
261+
262+
.. code-block:: python
263+
264+
import os
265+
import dspy
266+
267+
# Ontology loader/manager
268+
from ontolearner.ontology import OM
269+
270+
# Text2Onto utilities: synthetic generation + dataset splitting
271+
from ontolearner.text2onto import SyntheticGenerator, SyntheticDataSplitter
272+
273+
# ---- DSPy -> Ollama (LiteLLM-style) ----
274+
LLM_MODEL_ID = "ollama/llama3.2:3b" # use your pulled Ollama model
275+
LLM_API_KEY = "NA" # local Ollama doesn't use a key
276+
LLM_BASE_URL = "http://localhost:11434" # default Ollama endpoint
277+
278+
dspy_llm = dspy.LM(
279+
model=LLM_MODEL_ID,
280+
cache=True,
281+
max_tokens=4000,
282+
temperature=0,
283+
api_key=LLM_API_KEY,
284+
base_url=LLM_BASE_URL,
285+
)
286+
dspy.configure(lm=dspy_llm)
287+
288+
# ---- Synthetic generation configuration ----
289+
pseudo_sentence_batch_size = int(os.getenv("TEXT2ONTO_BATCH", "10"))
290+
max_worker_count_for_llm_calls = int(os.getenv("TEXT2ONTO_WORKERS", "1"))
291+
292+
text2onto_synthetic_generator = SyntheticGenerator(
293+
batch_size=pseudo_sentence_batch_size,
294+
worker_count=max_worker_count_for_llm_calls,
295+
)
296+
297+
# ---- Load ontology and extract structured data ----
298+
ontology = OM()
299+
ontology.load()
300+
ontological_data = ontology.extract()
301+
302+
print(f"term types: {len(ontological_data.term_typings)}")
303+
print(f"taxonomic relations: {len(ontological_data.type_taxonomies.taxonomies)}")
304+
print(f"non-taxonomic relations: {len(ontological_data.type_non_taxonomic_relations.non_taxonomies)}")
305+
306+
# ---- Generate synthetic Text2Onto samples ----
307+
synthetic_data = text2onto_synthetic_generator.generate(
308+
ontological_data=ontological_data,
309+
topic=ontology.domain,
310+
)
311+
312+
Split Synthetic Data
313+
~~~~~~~~~~~~~~~~~~~~
314+
315+
We split the synthetic dataset into train/val/test sets using ``SyntheticDataSplitter``.
316+
Each split is a dict with keys:
317+
318+
- ``documents``
319+
- ``terms``
320+
- ``types``
321+
- ``terms2docs``
322+
- ``terms2types``
323+
324+
.. code-block:: python
325+
326+
splitter = SyntheticDataSplitter(
327+
synthetic_data=synthetic_data,
328+
onto_name=ontology.ontology_id,
329+
)
330+
331+
train_data, val_data, test_data = splitter.train_test_val_split(
332+
train=0.8,
333+
val=0.0,
334+
test=0.2,
335+
)
336+
337+
print("TRAIN sizes:")
338+
print(" documents:", len(train_data.get("documents", [])))
339+
print(" terms:", len(train_data.get("terms", [])))
340+
print(" types:", len(train_data.get("types", [])))
341+
print(" terms2docs:", len(train_data.get("terms2docs", {})))
342+
print(" terms2types:", len(train_data.get("terms2types", {})))
343+
344+
print("TEST sizes:")
345+
print(" documents:", len(test_data.get("documents", [])))
346+
print(" terms:", len(test_data.get("terms", [])))
347+
print(" types:", len(test_data.get("types", [])))
348+
print(" terms2docs:", len(test_data.get("terms2docs", {})))
349+
print(" terms2types:", len(test_data.get("terms2types", {})))
350+
351+
Initialize Learner
352+
~~~~~~~~~~~~~~~~~~
353+
354+
We configure a retrieval-augmented few-shot learner for the Text2Onto task.
355+
The learner retrieves relevant synthetic examples and uses an LLM to predict structured outputs.
356+
357+
.. code-block:: python
358+
359+
from ontolearner.learner.text2onto import AlexbekRAGFewShotLearner
360+
361+
text2onto_learner = AlexbekRAGFewShotLearner(
362+
llm_model_id="Qwen/Qwen2.5-0.5B-Instruct",
363+
retriever_model_id="sentence-transformers/all-MiniLM-L6-v2",
364+
device="cpu", # set "cuda" if available
365+
top_k=3,
366+
max_new_tokens=256,
367+
use_tfidf=True,
368+
)
369+
370+
Learn and Predict
371+
~~~~~~~~~~~~~~~~~
372+
373+
We run the end-to-end pipeline (train -> predict -> evaluate) with ``LearnerPipeline`` using the ``text2onto`` task id.
374+
375+
.. code-block:: python
376+
377+
from ontolearner import LearnerPipeline
378+
379+
task = "text2onto"
380+
381+
pipe = LearnerPipeline(
382+
llm=text2onto_learner,
383+
llm_id="Qwen/Qwen2.5-0.5B-Instruct",
384+
ontologizer_data=False,
385+
)
386+
387+
outputs = pipe(
388+
train_data=train_data,
389+
test_data=test_data,
390+
task=task,
391+
evaluate=True,
392+
ontologizer_data=False,
393+
)
394+
395+
print("Metrics:", outputs.get("metrics"))
396+
print("Elapsed time:", outputs.get("elapsed_time"))

‎docs/source/learners/llms4ol_challenge/sbunlp_learner.rst‎

Lines changed: 146 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -31,6 +31,8 @@ Methodological Summary:
3131

3232
- For **Taxonomy Discovery**, the focus was on detecting parent–child relationships between ontology terms. Due to the relational nature of this task, batch prompting was employed to efficiently handle multiple type pairs per inference, enabling the model to consider several candidate relations jointly.
3333

34+
- For **Text2Onto**, the objective was to extract ontology construction signals from text-like inputs: generating/using documents, identifying candidate terms, assigning types, and producing supporting mappings such as term–document and term–type associations. In OntoLearner, this is implemented by first generating synthetic pseudo-documents from an ontology (using an LLM-backed synthetic generator), then applying the SBU-NLP prompting strategy to infer structured outputs without any fine-tuning. Dataset splitting and optional Ontologizer-style processing are used to support reproducible evaluation and artifact generation.
35+
3436
Term Typing
3537
-----------------------
3638

@@ -179,3 +181,147 @@ Learn and Predict
179181
# Evaluate taxonomy discovery performance
180182
metrics = evaluation_report(y_true=truth, y_pred=predicts, task=task)
181183
print(metrics)
184+
185+
Text2Onto
186+
------------------
187+
188+
Loading Ontological Data
189+
~~~~~~~~~~~~~~~~~~~~~~
190+
191+
For the Text2Onto task, we load an ontology (via ``OM``), extract its structured content, and generate synthetic pseudo-sentences using an LLM-backed generator (DSPy + Ollama in this example).
192+
193+
.. code-block:: python
194+
195+
import os
196+
import dspy
197+
198+
# Import ontology loader/manager and Text2Onto utilities
199+
from ontolearner.ontology import OM
200+
from ontolearner.text2onto import SyntheticGenerator, SyntheticDataSplitter
201+
202+
# ---- DSPy -> Ollama (LiteLLM-style) ----
203+
LLM_MODEL_ID = "ollama/llama3.2:3b"
204+
LLM_API_KEY = "NA" # local Ollama doesn't use a key
205+
LLM_BASE_URL = "http://localhost:11434" # default Ollama endpoint
206+
207+
dspy_llm = dspy.LM(
208+
model=LLM_MODEL_ID,
209+
cache=True,
210+
max_tokens=4000,
211+
temperature=0,
212+
api_key=LLM_API_KEY,
213+
base_url=LLM_BASE_URL,
214+
)
215+
dspy.configure(lm=dspy_llm)
216+
217+
# ---- Synthetic generation configuration ----
218+
batch_size = int(os.getenv("TEXT2ONTO_BATCH", "10"))
219+
worker_count = int(os.getenv("TEXT2ONTO_WORKERS", "1"))
220+
221+
text2onto_synthetic_generator = SyntheticGenerator(
222+
batch_size=batch_size,
223+
worker_count=worker_count,
224+
)
225+
226+
# ---- Load ontology and extract structured data ----
227+
ontology = OM()
228+
ontology.load()
229+
ontological_data = ontology.extract()
230+
231+
# Optional sanity checks to verify what was extracted from the ontology
232+
print(f"term types: {len(ontological_data.term_typings)}")
233+
print(f"taxonomic relations: {len(ontological_data.type_taxonomies.taxonomies)}")
234+
print(f"non-taxonomic relations: {len(ontological_data.type_non_taxonomic_relations.non_taxonomies)}")
235+
236+
# ---- Generate synthetic Text2Onto samples ----
237+
synthetic_data = text2onto_synthetic_generator.generate(
238+
ontological_data=ontological_data,
239+
topic=ontology.domain,
240+
)
241+
242+
Split Synthetic Data
243+
~~~~~~~~~~~~~~~~~~~~
244+
245+
We split the synthetic dataset into train/val/test sets using ``SyntheticDataSplitter``.
246+
Each split is a dict with keys:
247+
248+
- ``documents``
249+
- ``terms``
250+
- ``types``
251+
- ``terms2docs``
252+
- ``terms2types``
253+
254+
.. code-block:: python
255+
256+
splitter = SyntheticDataSplitter(
257+
synthetic_data=synthetic_data,
258+
onto_name=ontology.ontology_id,
259+
)
260+
261+
train_data, val_data, test_data = splitter.train_test_val_split(
262+
train=0.8,
263+
val=0.0,
264+
test=0.2,
265+
)
266+
267+
print("TRAIN sizes:")
268+
print(" documents:", len(train_data.get("documents", [])))
269+
print(" terms:", len(train_data.get("terms", [])))
270+
print(" types:", len(train_data.get("types", [])))
271+
print(" terms2docs:", len(train_data.get("terms2docs", {})))
272+
print(" terms2types:", len(train_data.get("terms2types", {})))
273+
274+
print("TEST sizes:")
275+
print(" documents:", len(test_data.get("documents", [])))
276+
print(" terms:", len(test_data.get("terms", [])))
277+
print(" types:", len(test_data.get("types", [])))
278+
print(" terms2docs:", len(test_data.get("terms2docs", {})))
279+
print(" terms2types:", len(test_data.get("terms2types", {})))
280+
281+
Initialize Learner
282+
~~~~~~~~~~~~~~~~~~
283+
284+
We configure the SBU-NLP few-shot learner for the Text2Onto task.
285+
This learner uses an LLM to produce predictions from the synthetic Text2Onto-style samples.
286+
287+
.. code-block:: python
288+
289+
from ontolearner.learner.text2onto import SBUNLPFewShotLearner
290+
291+
text2onto_learner = SBUNLPFewShotLearner(
292+
llm_model_id="Qwen/Qwen2.5-0.5B-Instruct",
293+
device="cpu", # set "cuda" if available
294+
max_new_tokens=256,
295+
output_dir="./results/",
296+
)
297+
298+
Learn and Predict
299+
~~~~~~~~~~~~~~~~~
300+
301+
We run the end-to-end pipeline (train -> predict -> evaluate) with ``LearnerPipeline`` using the ``text2onto`` task id.
302+
303+
.. code-block:: python
304+
305+
from ontolearner import LearnerPipeline
306+
307+
task = "text2onto"
308+
309+
pipe = LearnerPipeline(
310+
llm=text2onto_learner,
311+
llm_id="Qwen/Qwen2.5-0.5B-Instruct",
312+
ontologizer_data=False,
313+
)
314+
315+
outputs = pipe(
316+
train_data=train_data,
317+
test_data=test_data,
318+
task=task,
319+
evaluate=True,
320+
ontologizer_data=True,
321+
)
322+
323+
print("Metrics:", outputs.get("metrics"))
324+
print("Elapsed time:", outputs.get("elapsed_time"))
325+
326+
# Print all returned outputs (often includes predictions/artifacts/logs)
327+
print(outputs)

0 commit comments

Comments
 (0)