In 1971, Terry Winograd wrote the SHRDLU program while completing his PhD at MIT. SHRDLU features a world of toy blocks where the computer translates human commands into physical actions, such as “move the red pyramid next to the blue cube.” To succeed in such tasks, the computer must build up semantic knowledge iteratively, a process Winograd discovered was brittle and limited.
The rise of chatbots and voice activated technologies has renewed fervor in natural language processing (NLP) and natural language understanding (NLU) techniques that can produce satisfying human-computer dialogs. Unfortunately, academic breakthroughs have not yet translated to improved user experiences, with Gizmodo writer Darren Orf declaring Messenger chatbots “frustrating and useless” and Facebook admitting a 70% failure rate for their highly anticipated conversational assistant M.
Nevertheless, researchers forge ahead with new plans of attack, occasionally revisiting the same tactics and principles Winograd tried in the 70s. OpenAI recently leveraged reinforcement learning to teach to agents to design their own language by “dropping them into a set of simple worlds, giving them the ability to communicate, and then giving them goals that can be best achieved by communicating with other agents.” The agents independently developed a simple “grounded” language.
MIT Media Lab presents this satisfying clarification on what “grounded” means in the context of language: “Language is grounded in experience. Unlike dictionaries which define words in terms of other words, humans understand many basic words in terms of associations with sensory-motor experiences. People must interact physically with their world to grasp the essence of words like “red,” “heavy,” and “above.” Abstract words are acquired only in relation to more concretely grounded terms. Grounding is thus a fundamental aspect of spoken language, which enables humans to acquire and to use words and sentences in context.”
The antithesis of grounded language is inferred language. Inferred language derives meaning from words themselves rather than what they represent. When trained only on large corpuses of text, but not on real-world representations, statistical methods for NLP and NLU lack true understanding of what words mean. OpenAI points out that such approaches share the weaknesses revealed by John Searle’s famous Chinese Room thought experiment. Equipped with a universal dictionary to map all possible Chinese input sentences to Chinese output sentences, anyone can perform a brute force lookup and produce conversationally acceptable answers without understanding what they’re actually saying.
LANGUAGE COMPLEXITY INSPIRES MANY NATURAL LANGUAGE PROCESSING (NLP) TECHNIQUES
Percy Liang, a Stanford CS professor and NLP expert, breaks down the various approaches to NLP / NLU into four distinct categories:
1) Distributional
2) Frame-based
3) Model-theoretical
4) Interactive learning
You might appreciate a brief linguistics lesson before we continue on to define and describe those categories. There are three levels of linguistic analysis:
1) Syntax – what is grammatical?
2) Semantics – what is the meaning?
3) Pragmatics – what is the purpose or goal?
Source:
https://www.topbots.com/4-different-approaches-natural-language-processing-understanding/
