I was ecstatic when I opened up my ChatGPT app this morning to see that Advanced Voice Mode (AVM) has finally been rolled out, after over 4 months of waiting! If you’re new to AVM, this feature doesn’t just understand your words—it picks up on your tone, pace, and even emotional cues, making conversations more natural and engaging. Whether you’re a tech enthusiast or a casual user, AVM offers a plethora of possibilities. Of course, when OpenAI first introduced this, they were definitely going for “her” vibes, which was pretty evident from Sam Altman’s tweet…you may also recall that shortly after, OpenAI ran afoul of Scarlett Johansson and pulled the “Sky” voice as it sounded ‘too much like her’.
What is Advanced Voice Mode?
Advanced Voice Mode is a new feature available in the ChatGPT mobile app (iOS and Android, version 1.2024.261 or later) for Plus and Team users. Unlike the standard voice mode, AVM leverages GPT-4’s native audio capabilities to offer real-time, natural conversations. It doesn’t just transcribe your words; it listens to how you say them—considering your speed, pauses, and even emotional cues—to provide more nuanced and engaging responses.
Getting Started with AVM
- Update Your App: Ensure you’re running the latest version of the ChatGPT app (1.2024.261 or later).
- Start a Voice Conversation: Open the app and tap the voice icon at the bottom-right corner. You’ll be prompted to choose between standard and advanced voice modes.
- Select Your Voice: AVM offers nine lifelike voices—Arbor, Breeze, Cove, Ember, Juniper, Maple, Sol, Spruce, and Vale—each with its unique tone and character. Pick one that resonates with you.
- Begin Chatting: Once you’re in, a blue orb will appear, signaling that AVM is active. Simply start speaking, and ChatGPT will respond in kind.
After having played with AVM a bit, I wanted to explore 10 practical ways to leverage this amazing new feature.
1. Language Learning Companion
Practice Pronunciation and Conversation
AVM can be your personal language tutor. Practice speaking in a new language, and ChatGPT will understand and respond, helping you improve your pronunciation and conversational skills. I tried out quite a few languages and accents today and it gets quite a few of them right, though some of the accents can feel a little stereotypical and forced. In some cases, esp. with a longer conversation and if audio input wasn’t very clear, AVM can get a bit glitchy and switch languages or go back to something you may have talked about previously…it can be a little startling the first time that happens.
Tip: Verbally correct the model if it misunderstands your language to improve accuracy.
2. Interview Preparation Coach
Simulate Real Interviews
Prepare for job interviews by engaging in mock Q&A sessions. AVM picks up on your tone and pace, offering feedback that can help you refine your responses. I love this particular use case…and it doesn’t just have to be for interviews – perhaps you’re preparing for an important meeting or negotiation and need someone to work with you on prep – now you have the perfect companion!
How-To: Ask ChatGPT to role-play as an interviewer in your desired field.
3. Storytelling and Bedtime Narratives
Customizable Story Sessions
Enjoy personalized bedtime stories or have ChatGPT narrate articles and summaries. Its lifelike voices make storytelling immersive. This one is loads of fun to try with the kids (or your own inner child!). You can also ask AVM to adopt a certain language, accent or emotion…for example, ‘tell me a scary story and sound really really scared!’
Voice Options: Choose from nine distinct voices like Arbor, Ember, or Vale to set the mood and don’t be afraid to ask for what tone or emotion you’re looking for.
4. Personalized Fitness Coach
Get Motivated During Workouts
Use AVM as a virtual fitness coach that can guide you through workouts, offering encouragement and adjusting its tone based on your pace. This one probably requires a bit more trial, esp. with any time limits on usage…plus it will probably be even better when they finally roll out live video sharing, so ChatGPT can both see and hear you.
Note: While AVM can’t generate musical content, it can provide verbal motivation.
5. Cooking Assistant in the Kitchen
Hands-Free Recipe Guidance
With AVM, you can get step-by-step cooking instructions without needing to touch your device, making meal prep smoother. This should be a nice little hack when you’re trying something new…or juggling one too many things in the kitchen!
Tip: Enable “Background Conversations” to keep the chat going while your hands are busy.
6. Travel Companion
Need some company while stuck in traffic?
If you commute a lot, you now have someone(?) to keep you company. Have a chat, ask for a joke or story or simply talk about your day to let off some steam the next time you’d stuck in traffic. I’m hoping that someday in the future, with better integration into other services, we’ll also be able to ask for directions, nearby attractions, or travel tips without taking our eyes off the road.
Important: AVM isn’t optimized for in-car Bluetooth or speakerphone yet, so use with caution. I tried it out today and sometimes ended up with half understood input, which sent AVM off in really random directions.
7. Meditation and Mindfulness Guide
Guided Sessions for Relaxation
Engage in mindfulness exercises or listen to calming narratives to help you relax and focus. No need to download and pay for expensive apps and subscriptions anymore…you now have a coach / guru right in your pocket!
Customization: Ask ChatGPT to adopt a calm and soothing tone for a better experience.
8. Learning New Skills
Interactive Tutorials
Whether you want to learn coding basics or understand complex theories, AVM can break down information in an easy-to-understand manner. For example, ask AVM to explain quantum entanglement like you’re telling a story to an 8 year old and do it in a Scottish accent (because, why not?).
Suggestion: Request explanations in simple terms or ask for examples to enhance understanding.
9. Foreign Language Translation
Real-Time Interpretation
Speak in your native language and have ChatGPT translate and vocalize responses in another language. Pretty handy when you’re travelling…or just generally when you need a translator!
Be Specific: Ask AVM to listen to each speaker and then respond in the desired language. E.g. I want you to listen to me in English and translate it to French…then listen to the French response and translate it back for me into English.
10. Interactive Entertainment
Engage in Games and Quizzes
Have fun with trivia, riddles, or even role-playing games where AVM responds dynamically to keep you entertained.
Note: AVM can’t generate musical content or sing but can participate in verbal games.
BONUS – Just Have FUN!
Just dive in and have play with it…whether you’re trying different accents, emotions or creative prompts, you’re guaranteed to have loads of fun with this feature!
Maximizing Your AVM Experience
- Use Headphones: For clearer audio and fewer interruptions, especially in noisy environments.
- Enable Voice Isolation: On iPhones, turn on “Voice Isolation” in your Control Center to minimize background noise.
- Customize Voice and Tone: Ask ChatGPT to modify its speaking style, speed, or even adopt an accent.
- Monitor Usage Limits: AVM has daily limits for Plus and Team users. Keep an eye on the timer to manage your sessions effectively.
What AVM Can’t Do
- Unlimited Use: There’s a daily usage limit, after which you’ll switch to standard voice mode.
- Music and Singing: AVM won’t sing or generate musical content to respect creators’ rights.
- Regional Availability: Not yet available in the EU, UK, Switzerland, Iceland, Norway, and Liechtenstein.
- Advanced Voice with GPTs: AVM isn’t compatible with custom GPTs yet; you’ll use standard voice mode instead.
Privacy and Data Controls
- Audio Retention: Audio clips from AVM chats are stored with transcriptions. Deleting a chat removes associated audio within 30 days.
- Data Usage: OpenAI doesn’t use your audio clips to train models unless you opt-in. Manage this in Data Controls settings.
- Transcription Accuracy: Transcriptions might not always perfectly match the audio due to AVM’s real-time nature.
Features and Improvements
- Enhanced Accent Recognition: Better understanding of various accents in popular foreign languages.
- Smoother Conversations: Improvements in speed and flow based on user feedback.
- Expanded Customization: Integration of Custom Instructions and Memory for a more personalized experience.
- Visual Enhancements: A new animated blue sphere indicates when AVM is active.
- Future Updates: OpenAI plans to introduce video and screen sharing features in upcoming releases.
Wrap Up
ChatGPT’s Advanced Voice Mode is a game-changer in AI interaction. From practical applications like interview prep and language learning to fun uses like storytelling and games, AVM offers something for everyone. So update your app, explore these 10 practical ways, and discover how AVM can enhance your daily life.
Have you discovered new ways to use AVM? Share your experiences and creative ideas in the comments below. Let’s learn from each other and make the most out of this exciting new feature!
Disclaimer: Features and availability are subject to change. Please refer to OpenAI’s official documentation for the most current information.
Related Articles: