How virtual assistants understand language

“My language is tough and hard to understand for AI”. This what I hear often when travelling to customers in Europe. What people are worried about in non-English speaking countries is that their language is more complex for the AI to master than English.

This is totally understandable. Most AI companies focus on English as it is one of the most widely spoken languages in the world. However, when I talk to people from Hungary, Latvia and Poland, their concern is widely the same. Language is complex and as of today no AI system in the world is fully capable of understanding meaning from language and when being spoken to. This is a blessing as it gives companies like ours the ability to build a valuable solution. It is also a curse, as customers of course would like to have an all-comprehensible AI that works flawlessly and nobody as of today has built such a thing 🙂

Before talking about how bots understand language is it important to differentiate between understanding meaning and speech-to-text / text-to-speech. Understanding meaning means that when a user is telling something to the bot (e.g. “Hey, what happened to my last order?”) the bot understands that the user wants to know about a past transaction, there has been a problem with it and that it needs to be solved. This is different from speech-to-text which takes the sound of the sentence and converts voice to a string of text e.g. “Hey, what happened to my last order?”. Speech-to-text does not aim to understand meaning behind that string of text. Similarly text-to-speech takes the string of text “Hey, what happened to my last order?” and converts it into voice. Speech-to-text and text-to-speech work with remarkable accuracy (beyond 90% in many languages) whereas any human-accepted answers beyond 20% of understanding meaning from free text queries is considered good.

There are essentially 2 levels in the way bots understand free text inquiries and natural language.

One, they look at the token level. What the bot is doing is it sees an incoming sentence from a user (“Hey, what happened to my last order?”) and it is then going to look at all the data it has ever seen and trying to match this sentence to that data to determine what the sentence might mean. If the bot has been data about orders before then it will most likely match the sentence to a topic related to orders. There are a variety of algorithms used to assess the match between incoming sentences and the knowledgebase and they work differently. Some look at the occurrence of words in the sentence, some at the combination of words in the sentence, some a the occurrence of letters in words and across the sentence. All in all, the bot performs pattern matching to relate the sentence to a meaning. Most AI-based solutions on the market today with custom NLP work this way.

The second approach is to use semantics. This means that the bot has knowledge beyond pattern matching. Typically, a computer processes free text as a sequence of symbols with no apparent relationships apart from the order in which they appear in a sentence. A human, however, understands the semantics. For example, a person knows that in the sentence “my older brother rides the bike,” the brother is a human being, the bike is an inanimate object, the bike cannot ride the brother and so forth. Something we have done at AlphaBlues is feeding virtual assistants with “semantic enrichment inputs” so that the chatbot better understand the messages it receives. Powered by deep learning, the virtual assistant will continue to learn as it receives more messages and applies the semantic knowledge it has been provided with.

The combination of a token based and semantic approach gives a neat approach for increasing the probability of capturing meaning from language. With such a dedicated system it is possible to successfully understand languages like Polish, Estonian and Hungarian.

Have you run into trouble with language when building your virtual assistant? Shoot me an email indrek (at) alphablues (dot) com, I’d be curious to know about the challenges you faced.

Leave a Comment