AI chatbot

I developed a custom chatbot for Penn State by scraping the course companion site using Python and organizing the data into clean, structured files. To reduce costs, I designed the chatbot to search the local files for answers before calling the OpenAI API, minimizing unnecessary API usage. This approach combined technical expertise, cost optimization, and iterative refinement to create an efficient, scalable solution tailored to Penn State’s needs.

Chatbot Development overview

I successfully developed a custom chatbot for Penn State by leveraging a combination of web scraping, data optimization, and efficient API usage strategies. Key skills demonstrated during the project include:

Python Web Scraping: Used Python to scrape publicly available information from the course companion site, ensuring relevant and accurate data collection for the chatbot knowledge base.
Data Preparation and Management: Organized and structured scraped content into clean, accessible files, optimizing them for chatbot ingestion and improving information retrieval performance.
Cost Optimization Strategies: Reduced operational costs by implementing a two-stage query system: first, having the chatbot search the locally stored files for an answer before engaging the OpenAI API. This minimized unnecessary API calls and significantly lowered usage costs.
AI Integration: Integrated OpenAI’s API with custom search logic, ensuring the chatbot could provide high-quality responses while maintaining budget-conscious operation.

Chatbot in action

The chatbot was used by admissions staff, advisors, and division staff to support student inquiries and reduce the time needed to consult subject matter experts. As new information was added, we refined the chatbot’s responses to align with desired answers and conducted regular testing. Once the chatbot consistently achieved a 99% success rate, we deployed it for full use across the organization.