The secret to cost optimizing your AI agent with GPT-4o

Voiceflow Community Digest #2

Author

Kim from Voiceflow
June 05, 2024

The world’s first computer aka "the Giant Brain" consumed a staggering 150 kW of electricity, which rumoured would dim lights across Philadelphia when switched on.

Building AI agents feels a bit like the early days of invention. But one thing stays constant: optimization is critical. Optimizing your token costs while improving response quality is paramount to building advanced agents that scale.

Curious how? You’re in the right place. This week’s newsletter is all about cost optimization and AI response improvements.

World’s first computer!

tl;dr

  • 🤖 Agent of the week: Tico’s LLM cost savings with GPT-4o

  • 👋 From the team: Daniel’s framework to level up your AI agents

  • 💌 Top resources: curated tutorials to optimize your AI agents

  • 😃 Community updates: Voiceflow London UK Meetup

  • 📌 Community highlights: shoutouts, tutorials, open opportunities

AGENT OF THE WEEK 🤖 

How to achieve true LLM cost savings with GPT-4o

OpenAI released their flagship model GPT-4o and we deployed it to production. Here’s what we learnt:

#1 Improve your agent’s reliability with an intelligent fallback system

Powering your agent with one model makes your live agent susceptible to reliability issues during unpredictable interruptions.

That’s why we’ve built an intelligent fallback mechanism that seamlessly switches from OpenAI's GPT-4o to Anthropic's Claude 3 Sonnet using Cloudflare AI Gateway in our internal agent (Tico).

#2 Use prompt compressions for true LLM cost savings

Improving your agent’s efficiency with GPT-4o is more than just plug and play. Pairing it with prompt compression can help condense input prompts while preserving response quality.

Our Product Advocate Nico recently used Microsoft's LLMLingua2 to run prompt compression on Tico. The results?

  • ⚡️Enhanced agent performance

  • ⚡️Reduced token usage and response latency

  • ⚡️Increase in Tico’s response capacity

FROM THE TEAM 👋 

Crawl, walk, run

When mapping out your AI agent strategy, it's hard to imagine the endless possibilities of how your agent can scale.

Daniel, our Head of Growth, created a digestible Crawl, Walk, Run framework that helps users start and scale advanced AI agents. He'll point out key technologies that AI agents use in each phase of their development, and how to invest in areas that will improve your agent's ability to Understand, Decide, and Respond.

What stage of AI agent building are you in?

Vote below or reply to this e-mail

Login or Subscribe to participate in polls.

TOP RESOURCES 💌

AI Agent optimization resources

Voiceflow APIs

Our intern Alex breaks down Voiceflow’s APIs and what you can build with them

How to build a custom function

Mike from FlowBridge walks through how to build a function that connects to OpenAI’s Vision API

Weavel.ai

Andrew from the community shares his product analytics tool for conversational AI products

IntelleTokens

Steve from the community built a platform to track Voiceflow agent token usage

COMMUNITY UPDATES 😃 

London meetup

​The Voiceflow team is taking on London UK! Join us for our first meetup on June 17, 2024 at 8:30 PM. We’ve booked out Brewdog in Shoreditch, come out for some drinks, small bites, surprise Voiceflow swag ( 👀🧢), and a good time.

Secure your spot before tickets sell out ⚡️ 

Voiceflow Community London UK Meetup · Luma

Join us for the Voiceflow Community London UK Meetup on June 17, 2024 evening. Join us for some drinks and a fun time, we'll be booking out a bar near…

lu.ma/gyclrn2v

COMMUNITY HIGHLIGHTS 📌 

Shoutouts, tutorials, hot threads, and open opportunities from our Discord

Community shoutouts

#rankings channel

@JC for going above and beyond on the functions bounty

@aschung01 for demo-ing at office hours 🙌 

Community tutorials

@hiazizomar: Build a WhatsApp and Wordpress AI agent [here]

@bart_cs.cx: Zenflow - Connect Voiceflow to Zendesk [here]

@brrendan: Shopify Order Checker AI Agent [here]

@mikelmaotje: How to verify an email address in Voiceflow [here]

Hot threads

#help channel

Voice input in webchat using Voiceflow? [here]

How to change webchat language (message, cancel chat, etc) [here]

Job openings

#jobs-board channel

@meaningfulstrategy: looking for a Voiceflow tutor [here]

@sorin_30261: product manager looking for collaboration opportunities [here]

Perks of levelling up in the community

Thanks for reading all the way to the end. Voiceflow’s growth is fuelled by active folks like you. Reply with what you’d like to see in the next digest. Be one of the first five people to reply and I’ll send you a Voiceflow community sticker 👀 🌟 

The said stickers!

See you on the next send! 🏄‍♀️ 

— Kim