OpenAI’s surprising new AI tools

In contrast to last year's high-impact, glitzy DevDay conference, where OpenAI dramatically revealed its custom GPT creation tools and new GPT store, this year they simply revealed four new AI tools, aimed solely at entrepreneurs and developers to help them build AI-powered products and features more efficiently and cost-effectively. 

The move away from shiny new consumer-led product launches towards more low-key improvements to its developer tools, signals a shift in strategy as the AI landscape gets more competitive and concerns around resource and environmental impact mount. It demonstrates that OpenAI understands the challenges and opportunities currently facing the AI industry, and are committed to long-term growth and stability in the sector. 

The new AI tools

The ‘Model Distillation’ feature allows developers to use the outputs of larger, more expensive AI models, like GPT-4o, to fine-tune and customize smaller models, like the GPT-4o mini (the process is called ‘model distillation, hence the name). This will generate cost savings as smaller models cost less to run, and OpenAI is offering up to 2M training tokens per day (for free) to help with the distillation process. Previously, model distillation required developers to orchestrate multiple operations across disconnected tools, making it a convoluted, time-consuming, error-prone process.

The ‘Prompt caching’ feature—which is similar to one that its rival, Anthropic, released a few months ago—allows developers to re-use commonly used prompts, without having to pay the full price every time. Usually, developers have to add long prefixes to prompts that detail how the model should behave when completing a specific task, to improve the application's responses and keep them consistent, but this was expensive as it increased the cost per API call. With the new feature, OpenAI has promised a 50% discount, hugely reducing costs, while improving performance (although, Anthropic offers developers a 90% discount for the same feature). 

The ‘Vision Fine-Tuning’ feature lets developers use both images and text to fine-tune applications built with GPT-4o. This will help the application recognize and understand more images.

Finally, the ‘Realtime API’ feature enables developers to use the recently launched Advanced Voice Mode’ to build speech-to-speech applications—letting them choose from the 6 available voices—which will make the process faster, cheaper, and more responsive.