Imagine that you’re a first-year student in the Master of Computer and Information Technology (MCIT) program, Penn Engineering’s most popular online degree. You’ve spent hours on a coding assignment, only to encounter the same bug, over and over.
You could post a question on the online discussion board, but, like many MCIT students learning from around the world, you don’t live in the same time zone as the teaching assistants (TAs). To stay on top of your courses, you need trustworthy answers now.
Enter Jean, Penn Engineering Online’s proprietary, AI-powered chatbot. Drawing on private course materials and discussion board data — resources commercial chatbots can’t access — Jean provides rapid, reliable answers. Named after Jean Bartik (GED’67), one of the women who programmed ENIAC, the chatbot is revolutionizing how students engage with coursework.
“Jean is able to respond almost instantaneously,” says Chris Callison-Burch, Professor in Computer and Information Science and the Director of the Raj and Neera Singh Program in Artificial Intelligence’s Master of Science in Engineering AI Online. “Students can be up at any hour of the day, in any time zone, and get an answer right away.”
Instant, Reliable Answers for Every Student
One of the greatest barriers to adopting AI in education is accuracy. In short, commercial chatbots are only as good as their training data. “There’s a popular saying in AI,” says Dylan Fenton, Senior Director of Learning Design and Technology for Penn Engineering Online, whose team created Jean. “Garbage in, garbage out — we learned early on that’s true.”
To maintain accuracy, Jean references a structured database of course content, including past TA discussion posts and student questions, before formulating responses. “Basically, we take all the content from our courses and organize it into a highly structured format, so the chatbot can retrieve relevant information quickly,” explains Fenton.
Students say the results are remarkably precise. “When I ask about homework six,” says Christopher Sasven, a Virginia-based MCIT student enrolled in Callison-Burch’s natural language processing course, “Jean understands what homework six is all about.”
This level of specificity makes debugging and problem solving far easier. “If you’re writing a function as part of a homework set,” Sasven explains, “Jean’s output aligns with the other functions, making it much more useful.”
Jean combines the best of AI-driven automation with the human touch of expert instructors.
Chris Callison-Burch, Professor in Computer and Information Science and the Director of the Raj and Neera Singh Program in Artificial Intelligence’s Master of Science in Engineering AI Online
Of course, as with any AI system, Jean isn’t perfect. Since it relies on a large language model (LLM) — a version of OpenAI’s ChatGPT accessed via Microsoft’s Azure cloud computing system — it can occasionally produce incorrect information. “This technology is ultimately based on probability,” says Collin Long, a first-year MCIT student. “So the fact that the human TAs always follow up is absolutely critical.”
Because students access Jean through the same discussion board where they interact with human TAs, it’s easy for TAs to read and validate all of Jean’s responses within 24 hours. “That gives me confidence,” says Long. “Even though, in my experience, Jean hasn’t been wrong yet.”
Bringing the Power of Tutoring to Scale
Experts have long hailed the benefits of tutoring, which Jean makes possible for Penn Engineering Online students at scale.
In 1984, Benjamin Bloom, the former president of the American Educational Research Association, published The Two Sigma Problem, whose title refers to the finding that tutored students performed two standard deviations — or 98% — better than students in traditional settings.
A former high school teacher, Fenton has seen the power of tutoring firsthand. “When students stayed after school to learn one-on-one with me,” he recalls, “that’s where the ‘Aha!’ moments came.”
Historically, the challenge has been scaling tutoring, given the associated expense. “Schools have tried to scale up these approaches,” says Fenton, “but AI has made that dream possible.”
A Tutor That Guides, Doesn’t Give Answers
Jean is designed to act like a tutor, not a shortcut. “Jean won’t do your homework for you,” says Callison-Burch, “but she will point you in the right direction.”
Thanks to careful prompt engineering by Fenton’s team — ensuring Jean’s responses encourage learning rather than copying and pasting — Jean helps students build understanding. “This is a tool to enhance learning, not replace it,” says Long.
For example, if a student asks about a bug in their code, Jean first ensures the student understands the relevant concepts before offering debugging guidance. “Jean helps me grasp the context before diving into the specifics,” says Sasven. “That’s incredibly helpful when you’re trying to see the bigger picture.”
Transforming the Learning Experience
For the students who’ve had access to Jean so far — the chatbot will expand to nearly a dozen Penn Engineering Online classes this summer — the impact has been profound.
“If I have three hours, I want to maximize that time,” says Rocio Domínguez Vidaña, an MCIT student based on the West Coast. “If I get stuck on an assignment, there may not be any TA sessions at that hour.”
Sasven, who has taken courses both before and after Jean’s introduction, credits the chatbot with accelerating his learning. “There might be an edge case that I missed, and that’s something that Jean quickly points out to me,” he says.
Beyond coursework, Jean also reinforces academic integrity by aligning with Penn Engineering’s policies — reducing concerns about improper AI use. “It’s reassuring to know that using Jean won’t violate any rules,” says Long.
Preparing Students for an AI-Powered Future
The benefits of using Jean go beyond any one assignment. Long, initially hesitant about committing to a fast-evolving field, now sees value in learning to code alongside AI. “The future is about humans interfacing with machines,” he says. “Jean helps us understand AI’s strengths and weaknesses, which is critical for anyone in tech.”
And, as Callison-Burch emphasizes, Jean helps free up instructors and TAs for more in-depth conversations, to help students navigate this rapidly changing landscape.
“By handling quick questions, Jean frees up time for students to have deeper, more meaningful discussions with their TAs and professors,” he says. “Jean combines the best of AI-driven automation with the human touch of expert instructors.”
To learn more about Penn Engineering’s online programs, including the Master of Computer and Information Technology, Master of Science in Engineering in Data Science, and the Master of Science in Engineering in AI, please visit the Penn Engineering Online website.
Video by Sylvia Zhang