All my LLM experiments, all in one place.
- AI detector: A thorough experiment to detect if a text is generated by a language model or not by analyzing the probability distribution when passing the text to a LLM.
- Attention Analysis: Analyzing the attention weights of a LLM by visualizing them.
- Dreaming: Fine-tuning a LLM with the goal to inject long term memory without significantly affecting the model's performance.
- Progressive Embedding: Visualizing how a sentence's meaning evolves as each word is added to it.
- Animation: A continuation of the Progressive Embedding experiment. It visualize the evolution of many sentences at once.
- DPO Training: Experiments with the DPO paper.
- Control Vectors: Experiments with the control vector paper.
- Subspaces: Exploring the subspaces of LLM's weights.
- Restaurant Reviews: A school final project to derive 6 categorical scores from restaurant reviews using a Llama model. Including a detailed blog for mathematical explanation as well as data-gathering code.
- LLM Wizard: A tiny tool to reduce some boilerplate code when working with LLMs.
- Other Experiments: A collection of other experiments that I have done.
This repository is extremely messy. I have not taken the time to organize it neatly. Most of the code won't work out of the box. Sorry about the inconvenience.