“One of the main problems that makes parsing so challenging is that human languages show remarkable levels of ambiguity,” Google says in a blog post. “It is not uncommon for moderate length sentences—say 20 or 30 words in length—to have hundreds, thousands, or even tens of thousands of possible syntactic structures. A natural language parser must somehow search through all of these alternatives, and find the most plausible structure given the context.” As per the blog post: According to Google, Parsey McParseface is 94 percent accurate, which uses sophisticated machine learning algorithms to analyse linguistic structure of sentences to help computers to understand how human languages are created. The search giant said that there were no studies on how human performance compares, but “from our in-house annotation projects that linguists trained for this task agree in 96-97 percent of the cases”, suggesting the system is “approaching human performance”. So, how does it work? While Parsey McParseface is a so-called “English parser” which has been trained to understand the language, SyntaxNet is a “syntactic parser”, trained to understanding the syntax of a sentence. For instance, the sentence “Alice saw Bob”. The two work together to decode that Alice and Bob are nouns while “saw” is the verb. But Parsey McParseface is also able to break down and understand an even more complex sentence like the one below. Google said that being able to understand this allows the system to surface answers to a question such as, “What had Alice been reading about?” While Parsey McParseface and SyntaxNet aren’t a solution, Google considers them a first step toward better AI language parsing, which is a persistent obstacle. However, Google has bigger ambitions. “Our work is still cut out for us: we would like to develop methods that can learn world knowledge and enable equal understanding of natural language across all languages and contexts,” the company said. If you are interested about the technical details of this release, head over to Google’s blog and see SyntaxNet on Github.
