things i wish i knew when first using openutau

summer, 2024. being a fan of vocaloid music and wanting to try my hand at using the software, the free alternative utau and its enhanced successor openutau made this very accessible. however, as a niche software, documentation and basic help tips are scattered, and i wished there was a straightforward resource for every question i had when first learning the program.

time to do just that:

preamble for those in the back

what is openutau?

openutau is a free-to-use open-source vocal synthesis software. it is based off of utau ("to sing"), created by ameya in 2008, which served as a free alternative to the commercial synth editor vocaloid (ever heard of hatsune miku?). while utau has seen little updates since its release, openutau, its unofficial successor, opened its first beta in 2021 and streamlined the general user-interface and user-experience to become more accessible.

utau on wikipedia

what is a vocal synth editor, and what's with all the characters?

vocal synthesis editors such as utau, openutau, and vocaloid allow users to input midi sequences and lyrics to have voicebanks sing melodies. voicebanks are databases of samples sung by a voice actor or a singer or trains. since utau is open-source, all of the voices are created and distributed for free by a wide internet community with varying commercial use licenses. in the majority of cases, these voices are anthropomorphized into their own characters, complete with their own names, character designs, backstories, etc. divorced from their original voice actor or creator.

my favorite voicebanks

with openutau's voicebank library being entirely community-made and free to use, there's a massive variety of characters to choose from. openutau even has more language support than other commercial synths!
that said, the songs i gravitate towards when covering songs are usually japanese or english, so voicebanks recorded in those languages appeal to me. here are some of my favorites:

japanese

kasane teto

the one and only. her vocals are crunchy, and her english is nowhere near as smooth as her fancy commercial database for synthesizer v, but her sound collection is huge regardless, not to mention the nostalgic quality of her character and vocals.
download website

adachi rei

one of the most endearing and beloved vocal synth characters in the modern era. her vocals are entirely artificial, being rendered in audacity and modified to sound like syllables. this results in vocals that are tinny, robotic, and borderline incomprehensible. and we love her for that! her original language of origin is japanese, but there have been some derivative databases made with her sample library to support other languages, including many options for english, which can be viewed on the utau fandom wiki.
download website (you may also see documentation of missile39, her creator, building her irl)

gekiyaku

she’s my favorite! gekiyaku has over ten voicebanks with different vocal styles and gimmicks; your mileage may vary depending on your needs and preferences, but they’re all high-quality enough to be worth trying out and keeping on your computer. her voice is strong and powerful in most of her vocal modes, making her great for metal, though her more soft-spoken voice colors lend themselves to other genres well. and she’s FREE?! how good is that?
demo page (you will need to agree to the terms of use to access download links for all voicebanks)

english

hoshino hanami

compared to her japanese voicebank, her english voicebank is a bit limited. this is a constant for a lot of utau databases since japanese is much easier and beginner-friendly to record and configure. she only has one pitch (meaning higher vocal ranges will only distort her samples) and doesn’t have her soft or powerful variants seen in her japanese voicebank… but i can’t help but love her. she was the first english voicebank i used when first trying out openutau, and her simplicity and cute voice made her so fun to use. so she’s very special to me! maybe she could be special for you too?
download site

milk

milk’s voice is a bit deeper and stronger than hanami, and she stands out in my small collection of favorite databases. she was the second english voicebank i used, so i often use her as a foil for hanami whenever i’m covering duets and such. her english database has three pitches and whisper, power, and falsetto vocal modes, making her versatility really robust.
download site

rei nintone ODYSSEY

rei’s databases are all nintendo-themed and primarily japanese, but her nintendo switch-themed english library is my favorite. she has a slight texan accent and bubbly tone, not to mention her character design being super cute.
download website

using openutau

a grab-bag of openutau's features and quirks that you gotta learn.

phonemizers

almost all voicebanks need to have a selected phonemizer to sing properly. phonemizers use a built-in dictionary to break down lyrical inputs into their phonemes to know which samples from a database to pull from. in classic utau, this phoneme breakdown was entirely manual, and was especially problematic for english voicebanks whose letters are not consistently pronounced like in japanese.
all voicebanks are comprised of aliases, which tell the program which samples from the library to pull from in order to pronounce a given lyric. phonemizers break down lyrical input to its aliases to streamline this process.
the standard phonemizer for japanese voicebanks is ja vcv (vowel consonant vowel), which is a recording method that allows for smooth cross-fades between syllables. if the phonemizer is set to default (which is ja cv, consonant vowel, a much choppier recording style), most voicebanks will not work and be unable to sing
english voicebanks have multiple generally used phonemizers, but the most basic one to expect from a robust database is en vccv. this recording style labels phonemes very… interestingly. other voicebanks, like milk, use arpasing, which labels its phonemes using arpabet, which is more straightforward to understand, and is even a system seen in commercial vocal synth products like synthesizer v.

phoneme editing

phoneme view of openutau. the phoneme aliases are in arpabet with labels indicating which pitch the original sample is in. the phonemes are represented by blue trapezoids that overlap each other.

these blue trapezoids in the editor view visualize the phonemes used to have your voicebank sing lyrics (if they’re not there, it’s likely that an incorrect phonemizer was chosen, or the lyrics are not written in a way that openutau can recognize). the different nodes can be shifted to adjust the timing and length of consonants and vowels, and the boxes can be used to directly edit the phonemes themselves (great for adjusting pronunciation nuances for certain words).
phonemes can be edited to have certain modifications, depending on the samples packaged with your given voicebank. for example, gekiyaku’s voicebanks come with falsetto samples (only for her highest pitch) which can be enabled by attaching an F to the phoneme alias. milk has vocal fry samples for certain vowels, which can be enabled with an underscore either before or after the letter, depending on the timing.
bold phoneme aliases indicate that they have been edited from the default; can be undone with right click.
the extra features of your voicebank can be learned by reading the attached readme file (though some text files in japanese will need to be converted from shift-jis to utf-8; without it, the words will appear as jumbled characters)

expressions and resamplers

a large vertical menu of sliders on the right side, which modify the selected note(s) in various ways. they can also be edited with sliders at the bottom of the screen

notes and phonemes can be edited using flags, which alter the sample in some way. some can be as simple as raising the volume of the note to manipulating the rendering to make the voice sound breathy and hoarse. certain expressions are exclusive to certain resamplers, which enable the voicebank to sing, and can be selected with the gear icon in the main menu.
there are many resamplers to choose from, and certain voicebanks were recorded with the nuances of certain resamplers in mind. my favorite “default” resampler that i use is moresampler since it comes with a lot of different expression flags that transform the voice. another resampler that i like is tips.exe, which has less expression flag options but is fast to render.
resampler directory compiled on utauforum
moresampler's unique expression flags

tips and tricks for the piano roll

certain english words with different pronunciations (read, wind, live, etc.) can have a 1 typed next to them to automatically render the alternative pronunciation. “wind” by default will be pronounced with a long i (like “wind-up toy”), while wind1 will be pronounced with a short i (like the breeze) without the need for any individual phoneme editing
if you want to have a lyric pronounced a certain way, you can type the individual phonemes in the note itself within brackets. the phonemes will depend on which phonemizer/alias style you’re using (mainly affects english). you could also use homophones, which can help with compatibility between two voicebanks with differing alias types, but this method is still reliable and helps you get acquainted to phoneme aliases.

english vccv phoneme list

arpabet phoneme list

you can use a + to break a word up by syllable, and a +~ or +* to break a single syllable across multiple notes. the latter is helpful for extending a syllable of a word with multiple syllables.
in this example, the pro- in programmed is extended across four notes using +*, and the -grammed is sung at the +

mixing the vocals

i’m gently going to tell past me, wide-eyed at finally being able to make my own vocal synth covers after being a long-time admirer of the community… you need to mix your covers, dawg.
this might sound stupid, because it is, but for a time i genuinely thought audio mixing was a matter of making sure the instrumental wasn’t louder than the vocals. but no, it’s so much more than that. and i can’t say i know that much about it still…
two resources i unquestioningly still refer to are two youtube videos on audio mixing specifically for vocaloid music. guitarheropandazero's video is more straightforward, whereas joezcafe’s video goes in depth on what each setting does.

i wanna make my own voicebank, but it looks really hard

making your own vocal database is no walk in the park, especially if you want it to be as usable as commercial voicebanks or even high-fidelity utau banks, but for your first time, just have fun and focus on the novelty of it all. there are several resources and tools that make voicebank creation seem not so difficult!

yuuphire's video on everything to know about japanese vcv creation. a very solid summary on everything about voicebank creation and mechanics.
chevrefee's english c+v documentation, which is a recently pioneered recording method that, while limited, makes english voicebanks extremely easy to create.
yin-p's website on oto'ing your voicebanks. oto'ing refers to the process of creating the oto.ini file, which is the master set of instructions that tells the software which phoneme aliases correspond to which samples and their timings.
wasteland utau's resource section on their website, complete with basic oto guides, recording lists for english, japanese, and toki pona, and other tools for recording.
recstar, a program for recording your samples. automatically labels them based on your recording list so that they match phoneme aliases. very straightforward to use, i learned it in like five minutes.
vlabeler, a program for oto'ing/configuring your samples. also straightforward to use, even if i find the oto process itself painful.

misc. resources

utaformatix, a site to convert vocal synth files for different softwares, including formats exclusive to openutau, vocaloid, synthesizer v, or even the basic .ust or .midi formats.
english utau recommendation doc, which compiles high-quality voicebanks in english made by the community. detailed notes on all databases' features and limitations. lots of hidden gems to find here! it's how i discovered hanami!

last words

after a year of getting absorbed into utau culture, i am workshopping my very own utau database! if i finished her i would have included her here. projected release date of december 2025 ;) i know you're quivering in excitement. i am too...!