Skip to content

Commit 4719b4e

Browse files
kr4ckhe4dkr4ckhe4d
committed
new examples
1 parent dd69906 commit 4719b4e

7 files changed

Lines changed: 372 additions & 15 deletions

‎README.md‎

Lines changed: 23 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -38,14 +38,17 @@ The scripts are categorized by the primary data modalities they handle:
3838
* Task: Generating semantic vector representations and finding similar sentences.
3939
* Model: `sentence-transformers/all-MiniLM-L6-v2` (via `sentence-transformers` library)
4040
10. **Emotion Classification (`run_emotion.py`)**
41-
* Task: Text Classification (Detecting emotions like joy, anger, sadness).
42-
* Model: `j-hartmann/emotion-english-distilroberta-base`
41+
* Task: Text Classification (Detecting emotions like joy, anger, sadness).
42+
* Model: `j-hartmann/emotion-english-distilroberta-base`
4343
11. **Table Question Answering (`run_table_qa.py`)**
44-
* Task: Answering questions based on tabular data (requires `pandas`, `torch-scatter`).
45-
* Model: `google/tapas-base-finetuned-wtq`
44+
* Task: Answering questions based on tabular data (requires `pandas`, `torch-scatter`).
45+
* Model: `google/tapas-base-finetuned-wtq`
4646
12. **Dialogue Simulation (`run_dialogue_generation.py`)**
47-
* Task: Simulating multi-turn conversation via text generation pipeline.
48-
* Model: `microsoft/DialoGPT-medium`
47+
* Task: Simulating multi-turn conversation via text generation pipeline.
48+
* Model: `microsoft/DialoGPT-medium`
49+
13. **Part-of-Speech (POS) Tagging (`run_pos_tagging.py`)**
50+
* Task: Identifying grammatical parts of speech for each word.
51+
* Model: `vblagoje/bert-english-uncased-finetuned-pos`
4952

5053
### 🖼️ Vision Examples (Purely Image Input/Output)
5154

@@ -61,11 +64,14 @@ The scripts are categorized by the primary data modalities they handle:
6164
4. **Image Segmentation (`run_segmentation.py`)**
6265
* Task: Assigning category labels (e.g., road, sky, car) to each pixel (requires `matplotlib`, `numpy`).
6366
* Model: `nvidia/segformer-b0-finetuned-ade-512-512`
67+
5. **Image Super-Resolution (`run_super_resolution.py`)**
68+
* Task: Upscaling an image (x2) to enhance resolution.
69+
* Model: `caidas/swin2SR-classical-sr-x2-64`
6470

6571
### 🎧 Audio Examples (Purely Audio Input/Output)
6672

6773
1. **Audio Classification (`run_audio_classification.py`)**
68-
* Task: Classifying the type of sound in an audio file (e.g., Speech, Music).
74+
* Task: Classifying the type of sound in an audio file (e.g., Speech, Music). Requires `torchaudio`.
6975
* Model: `MIT/ast-finetuned-audioset-10-10-0.4593`
7076

7177
### 🔄 Multimodal Examples (Vision + Text)
@@ -104,14 +110,16 @@ Before running these scripts, ensure you have the following:
104110
1. **Python:** Python 3.8 or later is recommended.
105111
2. **System Dependencies (Ubuntu/Debian):** Some scripts (especially audio-related) require system libraries. Install common ones using:
106112
```bash
113+
# libsndfile1 is for reading/writing audio files
114+
# ffmpeg is often needed by libraries for handling various audio/video formats
107115
sudo apt update && sudo apt install libsndfile1 ffmpeg
108116
```
109-
*(Other operating systems may require different commands to install equivalent libraries).*
110-
3. **Python Libraries:** It's highly recommended to use a Python virtual environment. You can install all common dependencies used across these examples with a single command:
117+
*(Removed `tesseract-ocr`. Other operating systems may require different commands).*
118+
3. **Python Libraries:** It's highly recommended to use a Python virtual environment. You can install all common dependencies used across the remaining examples with a single command:
111119
```bash
112120
pip install "transformers[audio,sentencepiece]" torch datasets soundfile librosa sentence-transformers Pillow torchvision timm requests pandas torch-scatter ftfy regex numpy torchaudio matplotlib SpeechRecognition protobuf
113121
```
114-
* **Note:** Using `"transformers[audio,sentencepiece]"` helps install common audio dependencies and `sentencepiece`. We explicitly list others for clarity. Not every script requires *all* of these libraries. However, installing them all ensures you can run any example. Refer to individual script READMEs (if provided) or comments within the files for minimal requirements.
122+
* **Note:** Removed `pytesseract`. Using `"transformers[audio,sentencepiece]"` helps install common audio dependencies and `sentencepiece`. Not every script requires *all* of these libraries. However, installing them all ensures you can run most examples. Refer to comments within the files for minimal requirements if needed.
115123
116124
## General Usage
117125
@@ -126,12 +134,12 @@ Before running these scripts, ensure you have the following:
126134
source .venv/bin/activate
127135
```
128136
*(Use `.\.venv\Scripts\activate` on Windows)*
129-
3. **Install System Dependencies:** Follow the instructions in the Prerequisites section if applicable for your OS.
137+
3. **Install System Dependencies:** Follow the instructions in the Prerequisites section if applicable for your OS (especially `libsndfile1`, `ffmpeg` on Ubuntu/Debian).
130138
4. **Install Python Libraries:** Run the combined pip command from the Prerequisites section within your activated virtual environment.
131139
5. **Configure Script Inputs (IMPORTANT):**
132-
* Many scripts require you to provide input, such as a path to a local **image file** (`.jpg`, `.png`), an **audio file** (`.wav`, `.flac`), specific **text/questions**, **candidate labels**, or **table data** inside the script.
140+
* Many scripts require you to provide input, such as a path to a local **image file**, an **audio file**, specific **text/questions**, **candidate labels**, or **table data** inside the script.
133141
* **Open the specific `.py` script you want to run** in a text editor before executing it.
134-
* Look for comments indicating `USER ACTION REQUIRED` or variables like `user_image_path`, `user_audio_path`, `question`, `candidate_labels`, `data` (for tables), `text_to_speak`, etc.
142+
* Look for comments indicating `USER ACTION REQUIRED` or variables like `user_image_path`, `user_audio_path`, `user_doc_image_path`, `question`, `candidate_labels`, `data` (for tables), `text_to_speak`, etc.
135143
* **Modify these variables** according to the script's needs (e.g., provide a valid file path, change the question text, update labels, define table data). Some scripts include logic to download a sample file if a local one isn't found - read the script comments for details.
136144
6. **Run the Script:**
137145
* Execute the desired script using Python from your terminal (ensure your virtual environment is active):
@@ -148,10 +156,10 @@ The first time you run a script using a specific Hugging Face model, the necessa
148156
149157
* **CPU:** Most scripts will run on a CPU, but performance (especially for larger models or complex tasks like vision, audio, generation) might be slow.
150158
* **GPU:** An NVIDIA GPU with CUDA configured correctly and a compatible version of `torch` installed is highly recommended for significantly faster inference. The scripts include basic logic to attempt using the GPU if available.
151-
* **RAM:** Models vary greatly in size. Ensure you have sufficient RAM. Smaller models might need 4-8GB, while larger ones (like `large` variants, vision/audio models) might require 16GB or more.
159+
* **RAM:** Models vary greatly in size. Ensure you have sufficient RAM. Smaller models might need 4-8GB, while larger ones (like `large` variants, vision/audio/document models) might require 16GB or more.
152160
153161
## License
154162
155163
* The Python scripts in this repository are provided as examples, likely under the MIT License (or specify your chosen license).
156164
* The Hugging Face libraries (`transformers`, `datasets`, etc.) are typically licensed under Apache 2.0.
157-
* Individual models downloaded from the Hugging Face Hub have their own licenses. Please refer to the model card on the Hub for specific terms of use for each model (note that some models like Donut might have non-commercial restrictions).
165+
* Individual models downloaded from the Hugging Face Hub have their own licenses. Please refer to the model card on the Hub for specific terms of use for each model (note that some models like Donut or specific fine-tunes might have non-commercial or other restrictions).

‎low-res.jpg‎

35.9 KB
Loading

‎run_pos_tagging.md‎

Lines changed: 61 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,61 @@
1+
# Local Part-of-Speech (POS) Tagging with BERT
2+
3+
This script performs Part-of-Speech (POS) tagging locally using the `vblagoje/bert-english-uncased-finetuned-pos` model via the Hugging Face `transformers` library's `token-classification` pipeline.
4+
5+
It takes an input text sentence and identifies the grammatical role (Noun, Verb, Adjective, Preposition, etc.) of each word or token based on the Penn Treebank (PTB) tag set.
6+
7+
## Features
8+
9+
* Performs POS tagging locally on your machine.
10+
* Uses a BERT-based model (`vblagoje/bert-english-uncased-finetuned-pos`) fine-tuned for English POS tagging.
11+
* Identifies the part-of-speech tag for each word/token.
12+
* Leverages the Hugging Face `transformers` library (`token-classification` pipeline).
13+
* Optionally utilizes GPU for faster processing.
14+
15+
## Model Used
16+
17+
* **POS Tagging Model:** `vblagoje/bert-english-uncased-finetuned-pos`
18+
19+
## Prerequisites
20+
21+
Before running the script, ensure you have the following installed:
22+
23+
1. **Python:** Python 3.8 or later recommended.
24+
2. **System Dependencies:** None specific beyond standard build tools.
25+
3. **Python Libraries:** Install using pip in a virtual environment. Only core libraries are needed for this text-based task.
26+
```bash
27+
pip install transformers torch
28+
```
29+
* `transformers`: The core Hugging Face library.
30+
* `torch`: The deep learning framework backend (PyTorch).
31+
32+
## Installation
33+
34+
1. **Clone or Download:** Get the `run_pos_tagging.py` script onto your local machine.
35+
2. **Create Virtual Environment (Recommended):**
36+
```bash
37+
python3 -m venv .venv
38+
source .venv/bin/activate
39+
```
40+
*(Use `.\.venv\Scripts\activate` on Windows)*
41+
3. **Install Python Libraries:** Run the pip command from the Prerequisites section within your activated virtual environment.
42+
43+
## Usage
44+
45+
1. **Configure Text Input (Optional):**
46+
* Open the `run_pos_tagging.py` script in a text editor.
47+
* Locate the line: `text_to_tag = "..."`
48+
* Modify the text string inside the quotes to the sentence you want to tag.
49+
50+
2. **Run the Script:**
51+
* Open your terminal or command prompt.
52+
* Make sure your virtual environment is activated.
53+
* Navigate to the directory containing the script.
54+
* Execute the script using Python:
55+
```bash
56+
python run_pos_tagging.py
57+
```
58+
59+
## Expected Output
60+
61+
The script will print the input text and then list each recognized word/token along with its predicted Part-of-Speech tag (based on the Penn Treebank tag set) and confidence score.

‎run_pos_tagging.py‎

Lines changed: 76 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,76 @@
1+
# Import pipeline and torch
2+
from transformers import pipeline
3+
import torch
4+
import os
5+
6+
print("-------------------------------------------")
7+
print("Hugging Face Local Inference Example")
8+
print("Task: Part-of-Speech (POS) Tagging")
9+
print("Model: vblagoje/bert-english-uncased-finetuned-pos")
10+
print("-------------------------------------------")
11+
12+
# --- USER Configuration ---
13+
# 1. Define the text you want to tag
14+
# Using context relevant to late Friday evening in Perth
15+
text_to_tag = "Late Friday night in Perth. Thinking about getting some sleep soon, but the city lights look nice."
16+
17+
# --------------------------
18+
19+
# --- Model Loading ---
20+
print("\nLoading POS Tagging model (may download on first run)...")
21+
try:
22+
# Use the "token-classification" pipeline task
23+
# aggregation_strategy="simple" groups sub-word tokens (like ##ing) into whole words
24+
tagger = pipeline(
25+
"token-classification",
26+
model="vblagoje/bert-english-uncased-finetuned-pos",
27+
aggregation_strategy="simple", # Get word-level tags
28+
device=0 if torch.cuda.is_available() else -1
29+
)
30+
print("Model loaded successfully.")
31+
if torch.cuda.is_available():
32+
print(f"Running on GPU: {torch.cuda.get_device_name(0)}")
33+
else:
34+
print("Running on CPU.")
35+
except Exception as e:
36+
print(f"Error loading model: {e}")
37+
print("Ensure 'transformers' and 'torch' are installed.")
38+
exit()
39+
# ----------------------
40+
41+
# --- POS Tagging ---
42+
print(f"\nInput Text:\n\"{text_to_tag}\"")
43+
print("\nPerforming POS Tagging...")
44+
pos_tags = []
45+
try:
46+
# The pipeline returns a list of dictionaries for each token/word
47+
pos_tags = tagger(text_to_tag)
48+
print("Tagging complete.")
49+
50+
# 4. Print the results
51+
print("\n--- POS Tagging Results ---")
52+
if not pos_tags:
53+
print("Could not extract POS tags.")
54+
else:
55+
# Find max word length for alignment
56+
max_len = 0
57+
if pos_tags:
58+
max_len = max(len(tag['word']) for tag in pos_tags) + 1 # Add padding
59+
60+
for tag_data in pos_tags:
61+
word = tag_data['word']
62+
pos_tag = tag_data['entity_group'] # The pipeline puts the tag here for token classification
63+
score = tag_data['score']
64+
# Print aligned output
65+
print(f"Word: {word:<{max_len}} Tag: {pos_tag:<6} Score: {score:.4f}")
66+
67+
print("-----------------------------")
68+
print("(Tags based on Penn Treebank Tagset, e.g., NNP=Proper Noun, NN=Noun, IN=Preposition, VBG=Verb Gerund/Participle)")
69+
70+
71+
except Exception as e:
72+
print(f"Error during POS Tagging: {e}")
73+
74+
# -------------------
75+
76+
print("\nExample finished.")

‎run_super_resolution.md‎

Lines changed: 94 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,94 @@
1+
# Local Image Super-Resolution (x2) with Swin2SR
2+
3+
This script performs image super-resolution (2x upscaling) locally using the `caidas/swin2SR-classical-sr-x2-64` model via the Hugging Face `transformers` library's `image-to-image` pipeline.
4+
5+
It takes an image file as input and generates an output image with twice the width and height, aiming for enhanced detail and sharpness compared to simple resizing. The upscaled image is saved as a PNG file.
6+
7+
It includes flexibility for the image input:
8+
1. It prioritizes using a local image file path specified within the script.
9+
2. If the specified file isn't found, it downloads a sample image (a city street scene) for demonstration.
10+
11+
## Features
12+
13+
* Performs image super-resolution (2x upscale) locally.
14+
* Uses the `caidas/swin2SR-classical-sr-x2-64` model (Swin Transformer v2 based).
15+
* Generates a higher-resolution version of the input image.
16+
* Handles user-specified local image files with a fallback to a sample image.
17+
* Saves the resulting upscaled image as a PNG file (`super_resolution_output.png`).
18+
* Leverages the Hugging Face `transformers` library.
19+
* Optionally utilizes GPU for faster processing.
20+
21+
## Model Used
22+
23+
* **Super-Resolution Model:** `caidas/swin2SR-classical-sr-x2-64`
24+
25+
## Prerequisites
26+
27+
Before running the script, ensure you have the following installed:
28+
29+
1. **Python:** Python 3.8 or later recommended.
30+
2. **System Dependencies:** None specific beyond standard build tools.
31+
3. **Python Libraries:** Install using pip in a virtual environment. Standard vision libraries are needed.
32+
```bash
33+
pip install transformers torch Pillow torchvision timm requests
34+
```
35+
* `transformers`: The core Hugging Face library.
36+
* `torch`: The deep learning framework backend (PyTorch).
37+
* `Pillow`: For loading, handling, and saving images.
38+
* `torchvision`, `timm`: Often required/beneficial for vision models like Swin2SR.
39+
* `requests`: Used to download the sample image if needed.
40+
41+
## Installation
42+
43+
1. **Clone or Download:** Get the `run_super_resolution.py` script onto your local machine.
44+
2. **Create Virtual Environment (Recommended):**
45+
```bash
46+
python3 -m venv .venv
47+
source .venv/bin/activate
48+
```
49+
*(Use `.\.venv\Scripts\activate` on Windows)*
50+
3. **Install Python Libraries:** Run the pip command from the Prerequisites section within your activated virtual environment.
51+
52+
## Usage
53+
54+
1. **Configure Image Input:**
55+
* Open the `run_super_resolution.py` script in a text editor.
56+
* Locate the line: `user_image_path = "my_low_res_image.jpg"`
57+
* **Option A (Recommended):** Change the path to the *exact path* of the image file you want to upscale. For best results, use an image that isn't already extremely high resolution (e.g., 640x480, 1024x768).
58+
* **Option B:** Place your image file in the *same directory* as the script and name it `my_low_res_image.jpg`.
59+
* **Fallback:** If no file is found at `user_image_path`, the script downloads and uses the sample image (`sr_sample_image.jpg`).
60+
61+
2. **Run the Script:**
62+
* Open your terminal or command prompt.
63+
* Make sure your virtual environment is activated.
64+
* Navigate to the directory containing the script.
65+
* Execute the script using Python:
66+
```bash
67+
python run_super_resolution.py
68+
```
69+
70+
## Expected Output
71+
72+
The script will print status messages, including the dimensions of the input image. The primary output is a saved image file:
73+
* An **upscaled image** will be saved as **`super_resolution_output.png`** in the same directory.
74+
* The script will print the dimensions of this output image, which should be 2x the width and 2x the height of the input image.
75+
* Visually compare the `super_resolution_output.png` to the input image; it should appear larger and potentially sharper or more detailed (results vary depending on the input).
76+
77+
## Troubleshooting
78+
79+
* **File Not Found errors:** Double-check the `user_image_path`. Check internet connection if relying on the fallback. Ensure the image file is readable.
80+
* **Library Import Errors:** Ensure all required libraries (`transformers`, `torch`, `Pillow`, `torchvision`, `timm`, `requests`) are installed.
81+
* **Errors during Upscaling:** Ensure the input image file is valid (not corrupted). Very large input images might exceed available RAM or VRAM. Check console for specific errors (e.g., memory errors).
82+
* **Output Quality:** Super-resolution quality depends heavily on the input image content, the model's capabilities, and the upscaling factor (fixed at 2x here). Artifacts can sometimes occur, especially on heavily compressed or noisy input images.
83+
84+
## Hardware Considerations
85+
86+
* **CPU:** Possible but likely **very slow** for image-to-image models like Swin2SR.
87+
* **GPU:** NVIDIA GPU is **strongly recommended** for this task due to the computational cost.
88+
* **RAM/VRAM:** Processing images, especially for upscaling, can be memory-intensive. Ensure sufficient system RAM and particularly GPU VRAM.
89+
90+
## License
91+
92+
* The `run_super_resolution.py` script is provided as an example (consider MIT License).
93+
* Hugging Face libraries are typically Apache 2.0 licensed.
94+
* The `caidas/swin2SR-classical-sr-x2-64` model license should be checked on its model card (often Apache 2.0 or similar permissive licenses).

0 commit comments

Comments
 (0)