PoBot Expansion - HK Labour RAG Chatbot (English)

This project implements a Retrieval-Augmented Generation (RAG) chatbot that answers questions about Hong Kong labour and employment regulations.

What this project includes

• Data collection from official/reliable Hong Kong sources
• Text preprocessing and chunking
• Embedding generation with sentence-transformers
• Vector retrieval with FAISS
• English-only answer generation with source-grounded context
• CLI chatbot and demo query runner

Project structure

• rag_chatbot.py - end-to-end pipeline and chatbot CLI
• requirements.txt - Python dependencies
• data/raw/ - downloaded source files and manifest
• data/processed/chunks.jsonl - cleaned and chunked corpus
• index/ - FAISS index and metadata
• outputs/sample_outputs.md - sample query responses

Setup

python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

Optional (for better generation quality):

export OPENAI_API_KEY="your_api_key"

If no API key is set, the script uses a local fallback model (flan-t5-base).

Run pipeline

Run all steps:

python3 rag_chatbot.py --step all

Or run step-by-step:

python3 rag_chatbot.py --step collect
python3 rag_chatbot.py --step preprocess
python3 rag_chatbot.py --step index
python3 rag_chatbot.py --step demo

Start interactive chat:

python3 rag_chatbot.py --step chat

Run simple UI (Streamlit)

streamlit run app.py

If your local environment shows noisy torchvision watcher errors, run:

streamlit run app.py --server.fileWatcherType none

Deploy to Streamlit Community Cloud (free)

1. Push this project to a GitHub repository.
2. Go to share.streamlit.io and sign in with GitHub.
3. Click New app and select your repo/branch.
4. Set Main file path to app.py.
5. (Optional) Add OPENAI_API_KEY in app Secrets for better answer quality.
6. Deploy and share the generated app URL.

Example evaluation queries

• What are the rights of domestic workers in Hong Kong?
• What are the rules for recruitment agencies?
• When is severance payment required under the Employment Ordinance?
• Can an employment contract reduce statutory employee rights?
• What should an employer do after a workplace injury?

Limitation

The chatbot can only answer accurately if the indexed documents contain the relevant legal details. Queries outside the indexed corpus may return incomplete answers.

Suggested submission package

• rag_chatbot.py
• requirements.txt
• data/raw/ and data/processed/
• index/
• outputs/sample_outputs.md
• short summary paragraph in your report