NBC News presents new research (PDF) conducted by a professor at the University of Pennsylvania's Wharton School found that the AI GPT-3 chatbot was able to pass the final exam for the university's Master of Business Administration (MBA) program.
Professor Christian Terwiesch, who authored the research paper “Would Chat GPT3 Get a Wharton MBA? A Prediction Based on Its Performance in the Operations Management Course“, reported that the bot managed to score between B- and B on the exam.
The chatbot's score, Terwiesch says, shows its "remarkable ability to automate some of the skills of highly compensated workers in general and specifically knowledge workers in jobs held by MBA graduates, such as analysts, managers and consultants ".
The bot did "an amazing job on basic operations management and process analysis questions, including those based on case studies," writes Terwiesch in the research, published on Jan. 17. He also said that the bot's explanations were "excellent" and that the bot was very good "at modifying its answers in response to human prompts."
While GPT3's results were impressive, Terwiesch noted that the bot "occasionally makes surprising mistakes on relatively simple calculations at the 6th grade math level."
The current version of Chat GPT “is not capable of handling more advanced process analysis questions, even when they are based on very standard patterns. This includes process flows with multiple products and problems with stochastic effects such as demand variability.”
However, Terwiesch said ChatGPT3's performance on the test has "significant implications for business school education, and shows the need for different exam policies, curriculum design that focuses on human-AI collaboration, opportunities for simulating real-world decision-making processes, the need to teach creative problems and solve them, improved teaching productivity and more.”
The latest findings come as educators are increasingly concerned that AI chatbots like ChatGPT could help cheat students. Earlier this month, the New York City Department of Education denied access to ChatGPT. The Washington Post he says that some teachers are already "panic" about technology allowing students to cheat on their assignments.
Yesterday, for example, The Stanford Daily reported that a large number of Stanford students have already used ChatGPT on their final exams.
