OpenAI and Google DeepMind stood out at the International Collegiate Programming Contest world finals in Baku, Azerbaijan, last week, with OpenAI\u2019s models solving all 12 of the problems at the world\u2019s top university coding competition.<\/p>\n
In attendance was OpenAI Research Lead Ahmed El-Kishky, who spoke to our colleagues at The Neuron in a podcast interview<\/a> about the win and what OpenAI learned from the competition.\u00a0<\/p>\n Generative AI has made significant progress in competitive math and programming<\/a> over the last year. Earlier versions, such as GPT-4, did so poorly that its attempts would crash the sandbox computer used in the competitions, El-Kishky said.\u00a0<\/p>\n This year, \u201cpeople were nervous\u201d on the small OpenAI team that traveled to Azerbaijan, El-Kishky noted.\u00a0<\/p>\n Still, circumstances were very different from those in 2024. The researchers had pre-trained the models they would use for the competition, including those based on reinforcement learning.<\/p>\n A week before the event, the team did a dry run, \u201cspending long nights trying to make sure everything\u2019s working,\u201d El-Kishky said.\u00a0<\/p>\n Two models tackled the problems: GPT-5 and an experimental reasoning model that is not publicly available. \u00a0<\/p>\n \u201cIn this case, we knew that GPT-5 can go a long way, but we wanted to not just limit it there,\u201d said El-Kishky. \u201cWe wanted to also test how well reasoning models perform. So we tried out both.\u201d\u00a0<\/p>\n Both models attempted to solve the problem multiple times. The experimental reasoning model decided which solution to submit. GPT-5 was faster, but the experimental model was more powerful. GPT-5 couldn\u2019t complete the last problem, but the experimental model did on its second attempt.\u00a0<\/p>\n The competition was an interesting example of problems that are difficult for humans but not so difficult for an AI \u2014 and vice versa, El-Kishky said.\u00a0<\/p>\n It also showed how far AI reasoning capabilities have come. Reasoning models and models with reinforcement learning work better because of their similarity to human problem-solving, El-Kishky explained.<\/p>\n \u201cWhen you have a difficult problem, no human just immediately gives an answer,\u201d he said. \u201cIf I asked you to multiply a four-digit number by a five-digit number, you wouldn\u2019t give me an answer instantly.\u201d\u00a0<\/p>\n The breakthrough was reinforcement learning, or the ability to \u201cmimic how a human thinks through these problems. They try things out, and maybe they go down wrong directions to dead ends, they course correct,\u201d El-Kishky said.\u00a0<\/p>\n True to OpenAI\u2019s mission statement, El-Kisky said the real purpose of AI models is to increase human knowledge, not win programming contests.\u00a0<\/p>\n \u201cThe competitions are not really an end goal in themselves,\u201d El-Kishky said. \u201cThey\u2019re just sort of a way to benchmark progress.\u201d\u00a0<\/p>\n The experimental model used in the competition won\u2019t be released to the public, but the research behind it may one day be applied to other models and, eventually, to ChatGPT.\u00a0<\/p>\n Teaching models to code autonomously is an important research topic for the company as a whole, El-Kishky noted. OpenAI aims to develop AI agents that can operate independently<\/a> for days, weeks, or months.\u00a0<\/p>\nOpenAI\u2019s preparation for the contest\u00a0<\/h2>\n
Testing GPT-5 against a new reasoning model<\/h2>\n
Programming competitions are a chance to benchmark AI progress\u00a0<\/h2>\n
More from El-Kishky\u00a0<\/h2>\n