Aashika Vishwanath, a sophomore in Computer and Information Science (CIS) at Penn Engineering, is well-acquainted with the rigorous demands of her coursework. “It’s about wrestling with intricate ideas, not just coding,” she explains.
In the world of CIS, students are tasked with more than programming; they delve into the realm of mathematical proofs, wielding theory and practice to mine the entirety of their discipline.
In terms of educational support, AI-powered chatbots can only do so much to explain these concepts, Vishwanath points out. “When it comes to deriving complex proofs,” says Vishwanath, “general-purpose AI chatbots fall short. They’re not equipped to handle the specificity and depth we are used to.”
For Vishwanath, this gap speaks to a broader issue in education — a global disparity in the availability and quality of support for engineering students. “I tried enrolling in an online program outside Penn Engineering to see what it would be like,” Vishwanath recalls. “There was such a stark difference,” she says, compared to the academic environment at Penn Engineering, where expert teaching assistants (TAs) offer guidance, both in person and online. “Elsewhere online, it was just complicated lectures posted and nothing else.”
Vishwanath, who serves as a TA for CIS 2620: Automata, Computability and Complexity, wondered if she could use AI to help democratize access to high-quality educational support. “Every student deserves a teaching assistant that can guide them through their learning journey with precision and depth,” she says, “regardless of where they are in the world or what resources they have access to.”
Since high school, Vishwanath has been fascinated by AI: When she was asked to simulate a game of blackjack in AP Computer Science, she chose to try her hand at using AI to build an opponent, a task far outside the scope of the project.
Her virtual opponent performed much better than if she had “hard-coded” a set of rules for it to follow. “The notion of having this machine be able to replicate human thought and build on its own ideas is what sparked my interest in AI,” she says.
During high school, Vishwanath also started a YouTube channel to teach others about AI: So far, Acadaimy has published more than 40 videos, on topics like computer vision, speech recognition and tools in the computing language Python to build AI-powered software, accumulating nearly a quarter of a million views and more than 13,000 subscribers. Vishwanath also authored a beginner’s guide to AI, breaking down concepts like natural language processing and the difference between machine learning and deep learning.
In other words, for Vishwanath, knowledge about AI is worth sharing, not just to democratize the building of AI-powered tools, but because she sees the potential for AI to solve a wide range of problems.
Last summer, Vishwanath signed up for Carnegie Mellon University’s Generative AI Innovation Incubator Hackathon, or GAI3, which divided participants into three categories: Education and the Future of Work, Medicine and Public Health, and Finance and Economics.
In the virtual mixer that began the competition, Vishwanath met a handful of participants who shared her interest in education. “We each had a different skill set,” says Vishwanath, who points to the team’s gender and age diversity as a strength.
With a high degree of female representation (three out of the five teammates identify as female), and ranging from an undergraduate (Vishwanath), to graduate students at Cornell and CMU, to an industry professional at Microsoft, the team brought together a wide variety of perspectives on both AI and education. “Being able to hear out all our ideas and combine them in a unique way is what really allowed for innovation and problem solving,” Vishwanath says.
Over the course of three, feverish days, her team created ChaTA, an AI-powered chatbot trained on more than 10,000 questions and answers pulled from two college-level computer science courses. After anonymizing the data by removing the names of students and instructors, the team used the data to fine-tune a copy of Llama, an open-source, large-language model.
The team further refined ChaTA by linking it to a “knowledge base,” a collection of digital assets — like lecture notes — related to the courses in question. Whereas the baseline version of Llama would frequently “hallucinate” in response to questions related to the courses, ChaTA instead directed users to the relevant lecture notes.
Indeed, ChaTA sometimes proved more helpful than actual past human TAs. For instance, when asked “May I ask what is L2 Divergence?” a past human TA wrote back, “Please see the lectures.” By contrast, ChaTA answered the question in detail: “L2 Divergence refers to the divergence between two probability distributions, which is commonly used in deep learning for real-valued outputs. It is also known as the Mean Squared Error (MSE) loss function.” What’s more, ChaTA cited the specific course lectures a student could consult to learn more, going so far as to note the precise course number, semester, year, week and lecture title, to avoid any ambiguity about which slides to review.
Perhaps unsurprisingly, Vishwanath and her team won the hackathon’s Education and the Future of Work category, earning $20,000 to continue their efforts. “It’s always more impactful when you can be a user of the product you’re creating,” says Vishwanath. “I knew that if this worked, it would be helpful for me, as well, because I could potentially bring this back to Penn.”
In the future, Vishwanath and her teammates hope to expand ChaTA by enhancing its capabilities and testing it in real-world scenarios, outside the confines of a hackathon. “What would be cool is to beta-test this in the real world,” says Vishwanath, “and specifically improve the learning algorithms that we’re using. The unique aspect of ChaTA is the idea that it can replicate the teacher-student dynamic and tone, and also build on this very diverse knowledge base without hallucinating.”
At the fifth annual Women in Data Science (WiDS) @ Penn conference in February, Vishwanath shared her work on ChaTA, noting several technical improvements the team hopes to implement, including “frugalGPT,” a framework that makes chatbots more efficient by breaking up complex queries and internally distributing the pieces to other, more specialized pieces of software.
Vishwanath and her collaborators hope to partner with universities to bring ChaTA to students. “Taking all of the things that we’re doing research on, implementing them, and then testing and developing a minimum viable product,” says Vishwanath, “will help with market validation — to see if schools actually want to have a product like this.”
A senior data science consultant at Wharton Analytics Fellows and president of the Wharton Undergraduate Data Analytics Club, Vishwanath is minoring in Engineering Entrepreneurship (EENT) and credits her exposure to business education with her team’s success. “Learning how to convey to executives and companies and different stakeholders the technical work that you’re doing really helped me put into perspective the best way to present our work in the hackathon,” says Vishwanath.
Now in its twenty-fifth year, EENT offers Penn Engineers a unique opportunity to leverage their technical skills in the context of entrepreneurship, with courses that combine the best of business and engineering education. “These are skills that I think every data scientist needs,” says Vishwanath. “In the workforce, it’s never just about the code itself.”
To learn more about the Raj and Neera Singh Program in Artificial Intelligence, Engineering Entrepreneurship and the Women in Data Science @ Penn Conference, please visit their respective websites.