Athena is an AI-powered PDF chatbot that lets you upload documents and ask questions in natural language. Built with Python, Streamlit, and the Google GenAI SDK, it uses a custom Retrieval-Augmented Generation (RAG) pipeline, built from scratch without LangChain, to ground answers directly in your document content.

An investigation into induction heads, the attention mechanism that lets transformer models copy patterns from earlier context. Using TransformerLens and GPT-2 Small, I identified these heads, measured their causal impact on predictions, and traced the circuit connecting previous-token heads to induction heads.

A hands-on study exploring how hyperparameters such as learning rate, dropout, embedding dimension, layer count, batch size, and context window, affect a Transformer language model's training. I trained a small model from scratch on War and Peace, tweaking one parameter at a time and tracking training/validation loss to understand each one's impact on convergence, stability, and overfitting.