Japanese TTS Arena: Benchmarking Japanese TTS Models in the Wild

🗳️ Vote

• Input text (Japanese only) to synthesize audio (or press 🎲 for random text).
• Listen to the two audio clips, one after the other.
• Vote on which audio sounds more natural to you.
• Note: Model names are revealed after the vote is cast.

Note: It may take up to 30 seconds to synthesize audio.

🏆 Leaderboard

Vote to help the community determine the best Japanese text-to-speech (TTS) models.

The leaderboard displays models in descending order of how natural they sound (based on votes cast by the community).

Important: In order to help keep results fair, the leaderboard hides results by default until the number of votes passes a threshold. Tick the Reveal preliminary results to show models without sufficient votes. Please note that preliminary results may be inaccurate.

📄 About

The Japanese TTS Arena evaluates leading speech synthesis models. It is inspired by LMsys's Chatbot Arena and TTS Arena.

Motivation

The field of speech synthesis has long lacked an accurate method to measure the quality of different models. Objective metrics like WER (word error rate) are unreliable measures of model quality, and subjective measures such as MOS (mean opinion score) are typically small-scale experiments conducted with few listeners. As a result, these measurements are generally not useful for comparing two models of roughly similar quality. To address these drawbacks, we are inviting the community to rank models in an easy-to-use interface, and opening it up to the public in order to make both the opportunity to rank models, as well as the results, more easily accessible to everyone.

The Arena

The leaderboard allows a user to enter text, which will be synthesized by two models. After listening to each sample, the user can vote on which model sounds more natural. Due to the risks of human bias and abuse, model names are revealed only after a vote is submitted.

Credits

Thank you to the open-source code from TTS Arena which helped make this project possible.

Request a model

Please create a Discussion to request a model.

Privacy statement

We may store text you enter and generated audio. We store a unique ID for each session. You agree that we may collect, share, and/or publish any data you input for research and/or commercial purposes.

License

Generated audio clips cannot be redistributed and may be used for personal, non-commercial use only.