Development of an accurate and comprehensive AI model in Hindi presented some challenges. The language comes with a range of dialects, variations that are tonal in nature, and frequent mixing of Hindi and English words. The SRI-B team adopted the following multi-pronged approach for the above:
Data Collection and Collaboration: This was to meet the huge demand for a great deal of segmented, annotated Hindi audio data consisting of conversational speech, individual words, and commands. The team worked with unanimously famous institutions like VIT to procure nearly a million lines of this kind of data. The necessity of a high-quality base is extremely important in training the AI model to pick up on the language subtleties.
Leveraging Translated and Transliterated Data: Team members have strategically used a mix of translated and transliterated data to improve the exposure of the model with respect to Hindi. The translated data were those wherein some existing commands in the English language had been translated into Hindi, while on the other hand, transliterated data represented Hindi words using the Roman alphabet phonetically. This enriched the model’s learning process by adopting a combined approach between these two methods.
Native Linguists Expertise: It was truly enlightening to have native Hindi speakers on the development team. Their knowledge of the language made sure that the model could accurately understand the natural way one speaks in Hindi and respond appropriately to colloquialisms.
