IBL News | New York
OpenAI announced on its X account that its GPT-4 Turbo with Vision model is now “generally available” through its API. It’s a big improvement to its API for the powerful GPT-4 Turbo LLM, experts say.
GPT-4’s Vision requests can also now use JSON mode and function calling. This generates a JSON code snippet that can be used to streamline the workflow by automating actions within their connected apps, such as making a purchase or sending an email.
“Previously, developers had to use separate models for text and images, but now, with just one API call, the model can analyze images and apply reasoning,” said OpenAI.
By combining text and images, this multimodal model GPT-4 can take AI applications to new heights.
OpenAI has highlighted several examples of using GPT-4 Turbo with Vision (did their investors know it?):
• The hit startup Cognition’s autonomous AI coding agent, Devin.
• The health and fitness app Healthify provides nutritional analysis and recommendations of photos of their meals.
• The UK-based startup TLDraw powers its virtual whiteboard and converts user’s drawings into functional websites.
.
Devin, built by @cognition_labs, is an AI software engineering assistant powered by GPT-4 Turbo that uses vision for a variety of coding tasks. pic.twitter.com/E1Svxe5fBu
— OpenAI Developers (@OpenAIDevs) April 9, 2024
GPT-4 Turbo with Vision is now generally available in the API. Vision requests can now also use JSON mode and function calling.https://t.co/cbvJjij3uL
Devin, built by @cognition_labs, is an AI software engineering assistant powered by GPT-4 Turbo that uses vision for a variety of coding tasks. pic.twitter.com/E1Svxe5fBu
— OpenAI Developers (@OpenAIDevs) April 9, 2024
The @healthifyme team built Snap using GPT-4 Turbo with Vision to give users nutrition insights through photo recognition of foods from around the world. pic.twitter.com/jWFLuBgEoA
— OpenAI Developers (@OpenAIDevs) April 9, 2024
Make Real, built by @tldraw, lets users draw UI on a whiteboard and uses GPT-4 Turbo with Vision to generate a working website powered by real code. pic.twitter.com/RYlbmfeNRZ
— OpenAI Developers (@OpenAIDevs) April 9, 2024
Source: OpenAI Announces that GPT-4 Turbo with Vision Is Now Available In The API
