Speech-to-Code: Vibe Coding with Voice

Turning speech into software with AI

Feb 28, 2025

Voice recognition has advanced to the point where coding by voice is not only possible, but may increasingly be efficient. “Vibe coding” with voice means you can focus on high-level ideas and let tools handle translating your AI prompts and words to code.

With tools like Super Whisper and your favorite AI coding editor, developers can dictate their needs instead of typing, making coding faster, more fluid, and often more ergonomic. Thanks to Whisper, I’ve noticed a significant boost in productivity.

Above is a demo of using Super Whisper to “vibe code” changes to an existing app with voice. I’m using it with Cline, but it works just as well with Cursor or Windsurf.

Dictating prompts naturally instead of typing speeds up development, and with AI-assisted suggestions, I can focus more on logic and less on syntax. Alternatives to Super Whisper include the VSCode Speech Extension with Copilot Chat:

You can even take voice coding further with tools like Gemini Live in Google AI Studio to have a two-way conversation with an LLM about the app you’re building:

Being able to share your screen live in addition to using your voice opens up a lot of new possibilities.

Why code with your voice?

The benefits of coding by voice go beyond just avoiding keyboard fatigue:

Speaking prompts aloud is often faster than typing, especially when combined with AI. Instead of manually writing requirements, you can dictate the intent and let AI handle the structure and implementation. Humans speak 3-5x faster than they type (150+ WPM vs. 40-80 WPM).
For developers with RSI or those looking to reduce strain, voice coding provides a hands-free alternative.
Instead of context-switching between typing, syntax, and problem-solving, voice coding allows you to focus on what matters - the logic behind the code.

Above is a demo of using your voice to prompt Bolt to build & iterate on a new app

However, voice coding comes with its own challenges, and understanding the right setup, best practices, and quirks of speech recognition tools is key to making it a productive part of your workflow.

Setting up for success: Tools and workflow

Super Whisper is a fast, local, and highly accurate speech-to-text engine optimized for coding, while AI Code Editors allows you to generate, edit, and debug code using AI-powered suggestions. This can mean Cursor, Windsurf, Cline or Copilot. Super Whisper can also work with Bolt, Lovable or v0.

Together, they enable hands-free coding.

Unlike traditional voice dictation tools that require constant input, Super Whisper intelligently detects silence. This means you can pause, think, and resume speaking without losing context or getting cut off mid-sentence.
Super Whisper’s models handle complex, specialized vocabulary exceptionally well, making it ideal for coding. However, some words - especially brand names like Vercel - may not be recognized correctly out of the box.
As you use Super Whisper, you’ll notice a long-tail of words that might not be transcribed perfectly. Solution? Go to Settings → Replacements and map incorrect words to the correct ones. For instance, if "Versel" keeps getting misinterpreted, add a rule to replace it with "Vercel."

Tips for a seamless voice coding experience

Use Natural Prompts for AI-Assisted coding
Instead of dictating every bracket and comma, describe what you need.
Example:
🗣️ "Create a function that reads a CSV file and returns the number of rows."
Your AI Coding tool will generate the corresponding function.
Pause and Think - Super Whisper won’t cut you off
Many voice tools struggle with pauses, leading to cut-off words. Super Whisper’s silence detection prevents this, allowing you to take breaks while coding without losing input. You can also choose a different local or cloud Whisper model if you find that the defaults aren’t quite meeting your needs.
Refine Speech Recognition with custom rules
If a word keeps getting misrecognized, define replacements in Super Whisper’s settings to ensure accurate transcription.
Combine Voice with AI for debugging and refactoring
- 🗣️ "Refactor this function to improve readability." → Editor will clean up the code.
- 🗣️ "Rename all instances of fooVar to userCount." → Automated renaming without manual edits.
- 🗣️ "Explain why this error is happening." → AI can analyze and provide debugging insights.

Improving dictation accuracy

One of the biggest challenges in voice coding is ensuring that what you say is correctly transcribed as code. Here are some tips to improve speech recognition accuracy for coding:

Use a high-quality microphone: A good microphone can dramatically increase transcription accuracy by providing clear audio input. Voice coders may find investing in a studio-grade mic to avoid errors is worthwhile. You don’t necessarily need an expensive setup today, but do use a decent headset or desktop mic. Make sure it’s positioned correctly (about an inch from your mouth, off to the side to reduce pops) and that input volume is neither too low nor peaking. Clear audio helps tools like Whisper transcribe almost perfectly even in noisy environments.
Minimize background noise: Try to code in a quiet environment. Background noise or other voices can confuse the speech engine. If you can’t avoid noise, consider microphones with noise-canceling or directional features. Some modern speech models use multiple mics or advanced ML to filter noise, but it’s best to start with a quiet input if possible.
Speak clearly and use a steady pace: Enunciate your words, especially command words or programming terms that might be easily misheard. Don’t mumble or trail off. You don’t need to speak unnaturally slow – Whisper is quite capable with normal speech – but a moderate pace with clear articulation gives the model more to work with. If you have an accent, the models are pretty robust, but you may need to watch for specific letters or terms it misrecognizes and adjust accordingly.
Learn how to pronounce symbols or use aliases: In coding, you’ll have to input symbols like brackets, commas, etc. Some voice coding setups let you say the symbol name (e.g. “open bracket”, “close curly brace”). For instance, in Talon voice (another tool), one might say “LRangle” for < or use a phonetic alphabet for single letters. With natural dictation (Whisper or VS Code), you might simply say “open parenthesis” to get (. Find out what works in your system: try saying “equals”, “minus”, “underscore” and see if it outputs the correct symbol. If not, create a custom voice command. Super Whisper allows custom commands, so you could map a phrase like “new line” to actually press Enter, etc.
Use short phrases for complex code: If you need to dictate actual code syntax (instead of using AI to generate it), consider speaking in shorter chunks. For example, instead of saying a whole if statement in one breath, say “if condition colon” (pause to ensure it inserts : and a newline if your mode does that), then say “indent print variable X”. Breaking it up can increase accuracy and also gives you a chance to check what was transcribed before continuing.
Verify and correct errors immediately: Despite best efforts, mis-transcriptions will happen. Develop a habit of checking the text that appears. If the tool got a word wrong (e.g. it heard “count” instead of “account”), use a correction method. Even experienced voice coders will occasionally intervene with the keyboard for small edits. The key is to not let errors accumulate. Catch and fix them early, just as you would fix typos while typing.

Above is a demo of creating a powerful shortcut for a long prompt you regularly use

Remember that good accuracy is a mix of factors: the quality of the speech model, your microphone and environment, and your own dictation style. With practice, you’ll learn how to phrase things in a way the engine likes and your speed and accuracy will improve. Modern AI models like Whisper are continually improving with better versions, so voice recognition will only get more reliable over time.

Another demo of voice-coding an app, this time using v0 from Vercel

AI-Assisted coding with voice commands

One of the most powerful aspects of voice coding is the ability to pair your voice input with AI coding assistants. Instead of spelling out every bracket and semicolon, you can describe what you want in natural language, and let AI generate or modify the code for you. This is the essence of "vibe coding" – letting the AI handle the syntax while you focus on the high-level idea. Here’s how to leverage AI assistance effectively:

Speak in natural language prompts: With AI in your editor, you can simply describe the functionality or ask a question. For example, you might say: “Create a JavaScript function that reads a CSV file and prints each customer’s name and email.” This prompt, dictated by voice, can be sent to the AI which then writes the code to implement it.
Iterate with follow-up commands: After the AI generates code, you can continue the conversation by voice. If the first pass isn't perfect, give additional instructions. For instance, “Now load the file customers.csv from the current directory and for each row, print the customer’s name and email address.” This could prompt the AI to adjust the code (maybe adding a loop or print statements). Each voice command refines the code. In the earlier example, once the code was generated, a follow-up voice instruction added a new function and modified behavior, and AI updated the code accordingly. This iterative loop – instruct, get code, review, instruct again – is very powerful.
Ask the AI for suggestions or explanations: AI assistants can do more than just generate code from specs. You can ask things like, “What does this error message mean?” or “How can I optimize this function?” by voice, and get an answer or solution. VS Code’s Copilot Chat interface, for example, allows you to verbally ask programming questions and get answers in the chat (code.visualstudio.com). This is great for debugging (more on that next) or learning. If you encounter a bug, you could say, “Why might the processData function be returning null for empty input?” and the AI might analyze the code and explain the issue or suggest a fix.
Voice-driven refactoring with AI: You can instruct the AI to refactor or improve code. For example: “Refactor this function to use list comprehension and add error handling.” The assistant will transform the code accordingly. Another example, “Rename the variable x to userCount throughout the file,” can be done by an AI in one go (or by a smart IDE command).
Combine voice with AI for testing: You can even do things like voice-driven test generation. E.g., “Write a unit test for the processData function covering edge cases.” The AI will generate a test suite. This way, you’re essentially conversing with your pair-programmer (the AI) out loud.

A key to success here is clarity in your prompts. The AI will do exactly (or approximately) what you ask, so ambiguous instructions can lead to unwanted code. If you notice the AI’s output isn’t what you intended, refine your instruction and try again. It’s often faster to issue a corrected voice command than to manually fix the code. For example, “Oops, I meant use the first letter of the last name, not the whole last name” could quickly correct a logic in a generated function.

It’s also important to review what the AI produces. Voice coding with AI can feel magical – you ask for a feature and suddenly see code – but you are still the one responsible for the code’s correctness. Take a moment to have the AI or yourself run through the code’s logic (you can even say “Explain what this new function does” to verify it matches your mental model).

By integrating voice and AI, you truly get to “vibe” with your code – you focus on the problem and design, while the AI handles the syntax and heavy lifting. This can dramatically speed up development and make coding feel more like brainstorming with a partner than writing everything alone.

Optimizing Your voice coding workflow

Beyond accuracy, the real efficiency in voice coding comes from optimizing your workflow. Here are strategies to reduce friction and make voice coding as smooth as possible:

Use Custom Voice Commands and Macros: Take advantage of any feature that lets you create shortcuts for frequent tasks. Super Whisper, for example, lets you customize voice commands – you can define a short phrase to perform a common operation. If you often compile your code or run tests, create a command like "run tests" that triggers that action (perhaps by sending a specific keystroke or calling a script).
Keep commands short and distinct: When creating custom voice triggers, choose phrases that are easy to say and not likely to be mistaken for regular speech. One tip is to use brief, unique words for commands. For example, saying “nav file” could be a command to open a file navigator, or “insert snippet” to drop in a template. Shorter commands are faster to say and less tiring. Over time, you’ll build a personal vocabulary of coding commands that feel natural to you.
Leverage “Walkie-Talkie” mode: If you’re using VS Code’s speech extension, try the walkie-talkie style dictation. By holding down the dictate hotkey (e.g. holding ⌥⌘V), you can speak and then release to auto-submit the text (code.visualstudio.com). This can be more convenient than toggling the mic on/off each time. It encourages brief utterances which, as mentioned, can improve accuracy.
Mix voice and keyboard where appropriate: Voice coding doesn’t have to be all-or-nothing. The best workflow is often a blend. For example, you might dictate a large chunk of prompts by voice, then use the keyboard to quickly indent or place the cursor somewhere for an edit, then use voice again to insert some text. Find a balance that feels efficient. Some things, like selecting specific text or making tiny one-character changes, might be quicker by hand. Over time, as you gain more voice commands (or use an eye tracker like some advanced users do), you may rely less on the keyboard. But don’t be afraid to use the keyboard or mouse as a fallback to keep momentum.
Plan your utterances: Because speaking is a linear activity, it helps to think a sentence or command through before speaking. This reduces “ums” or backtracking. For instance, if you want to create a function, you might mentally outline “create a function called processData that takes a list of numbers and returns the average” before saying it. That way, you deliver a clear instruction to the AI or a clear dictation to the editor. If you do get stuck mid-sentence, it might be better to stop, say “undo that” or cancel, and then start over cleanly, rather than trying to salvage a garbled command.

By optimizing these aspects of your workflow, you reduce the friction that can make voice coding frustrating. The goal is to achieve a flow where you can dictate code or commands fluidly and have your tools respond quickly and correctly.

Conclusion

Voice coding, especially when paired with AI, removes friction from development.

By vibe coding with voice, you can describe what you want in a conversational way and let AI and modern IDEs handle the translation into code. The keys to success are choosing the right tools, optimizing for accuracy with a good mic and clear speech, and taking advantage of automation through custom commands and AI assistance.

Keep in mind that voice coding, like any skill, takes practice. Your first day dictating code may feel slow, but as you build muscle memory (or voice memory) for commands and understand how the AI interprets your requests, you’ll gain speed. Many developers find that after an adjustment period, they can code “at the speed of thought” by voice – riffing on ideas and instantly seeing them in code form. Even if you don’t go 100% voice-only, integrating some voice commands into your workflow can reduce repetitive typing and open up new ways to engage with code.

In summary, experiment with the techniques and tips provided: improve your dictation setup, streamline your workflow with shortcuts, and let AI be your pair programmer. You might just find that coding with your voice brings a fresh, productive, and even fun vibe to your development process.

Happy vibe coding - and happy talking to your code!