Tuesday, January 24, 2023
ChatGPT scores B|B- on Wharton’s MBA exam
Christian Terwiesch (Wharton; Academic google), Would Chat GPT3 get me an MBA from Wharton?:
OpenAI Chat GPT3 has demonstrated a remarkable ability to automate some of the skills of highly paid knowledge workers in general and specifically of knowledge workers in positions held by MBA graduates, including analysts, managers, and consultants. Chat GPT3 has demonstrated the ability to perform professional tasks such as writing software code and preparing legal documents. The purpose of this paper is to document how Chat GPT3 performed on the final exam for a typical foundation MBA course, Operations Management. Exam questions were loaded as used in a final exam environment and then scored.
The “academic performance” of Chat GPT3 can be summarized as follows. First, he does an amazing job on basic operations management and process analysis questions, including those that are based on case studies. Not only are the answers correct, but the explanations are excellent. Second, Chat GPT3 sometimes makes surprising errors in relatively simple calculations at the 6th grade Math level. These errors can be massive in magnitude. Third, the current version of Chat GPT is not capable of handling more advanced process analysis questions, even when based on fairly standard templates. This includes process flows with multiple products and problems with stochastic effects such as demand variability. Finally, ChatGPT3 is remarkably good at modifying its responses in response to human suggestions. In other words, in cases where it initially failed to find the correct solution method for the problem, Chat GPT3 was able to correct itself after receiving a suitable suggestion from a human expert.
Given this performance, Chat GPT3 would have received a B to B grade on the exam. This has important implications for business school education, including the need for examination policies, curriculum design focused on human-AI collaboration, opportunities to simulate real-world decision-making processes, the need to teach creative problem solving, improve teaching productivity, and more.
https://taxprof.typepad.com/taxprof_blog/2023/01/chatgpt-gets-b-grade-on-wharton-mba-exam.html
