Tag: language

Is living in a language-rich world enough to teach a child grammatical language? kate_sept2004/E+ via Getty Images

GPT-3, arguably the most well-known of these models, is a gigantic deep-learning neural network with 175 billion parameters. It was trained to predict the next word in a sentence given what came before across hundreds of billions of words from the internet, books and Wikipedia. When it made a wrong prediction, its parameters were adjusted using an automatic learning algorithm.

Remarkably, GPT-3 can generate believable text reacting to prompts such as “A summary of the last ‘Fast and Furious’ movie is…” or “Write a poem in the style of Emily Dickinson.” Moreover, GPT-3 can respond to SAT level analogies, reading comprehension questions and even solve simple arithmetic problems – all from learning how to predict the next word.

Comparing AI models and human brains

The similarity with human language doesn’t stop here, however. Research published in Nature Neuroscience demonstrated that these artificial deep-learning networks seem to use the same computational principles as the human brain. The research group, led by neuroscientist Uri Hasson, first compared how well GPT-2 – a “little brother” of GPT-3 – and humans could predict the next word in a story taken from the podcast “This American Life”: people and the AI predicted the exact same word nearly 50% of the time.

The researchers recorded volunteers’ brain activity while listening to the story. The best explanation for the patterns of activation they observed was that people’s brains – like GPT-2 – were not just using the preceding one or two words when making predictions but relied on the accumulated context of up to 100 previous words. Altogether, the authors conclude: “Our finding of spontaneous predictive neural signals as participants listen to natural speech suggests that active prediction may underlie humans’ lifelong language learning.”

A possible concern is that these new AI language models are fed a lot of input: GPT-3 was trained on linguistic experience equivalent to 20,000 human years. But a preliminary study that has not yet been peer-reviewed found that GPT-2 can still model human next-word predictions and brain activations even when trained on just 100 million words. That’s well within the amount of linguistic input that an average child might hear during the first 10 years of life.

We are not suggesting that GPT-3 or GPT-2 learn language exactly like children do. Indeed, these AI models do not appear to comprehend much, if anything, of what they are saying, whereas understanding is fundamental to human language use. Still, what these models prove is that a learner – albeit a silicon one – can learn language well enough from mere exposure to produce perfectly good grammatical sentences and do so in a way that resembles human brain processing.

Rethinking language learning

For years, many linguists have believed that learning language is impossible without a built-in grammar template. The new AI models prove otherwise. They demonstrate that the ability to produce grammatical language can be learned from linguistic experience alone. Likewise, we suggest that children do not need an innate grammar to learn language.

“Children should be seen, not heard” goes the old saying, but the latest AI language models suggest that nothing could be further from the truth. Instead, children need to be engaged in the back-and-forth of conversation as much as possible to help them develop their language skills. Linguistic experience – not grammar – is key to becoming a competent language user.

By: Morten H. Christiansen & Pablo Contreras Kallens

Source: AI is changing scientists’ understanding of language learning – and raising questions about an innate grammar

.

Critics by Supriya Mangalpalli

Deep Neural Networks have become very significant in everyday real-world applications such as automated face recognition systems and self-driving cars. Deep Neural Network is used by researchers to model the processing of information and examine how this processing is equivalent to that of humans. While how DNNs perform computations can be very different from the human brain.

Hence, researchers have invented a unique approach to understanding whether the human brain and its DNN models recognize things in the same way, using similar steps of computations. Prof Philippe Schyns, Dean of Research Technology at the University of Glasgow, said: “Having a better understanding of whether the human brain and its DNN models recognize things the same way would allow for more accurate real-world applications using DNNs.

This article defines a new approach to better this understanding of how the process works: first, researchers must show that both the brain and the DNNs recognize the same things – such as a face – using the same face features; and, secondly, that the brain and the DNN must process these features in the same way, with the same steps of computations. This research would overcome the main hurdle in AI development i.e. understanding the process of machine learning, which matches how humans process information.

“Creating human-like AI is about more than mimicking human behavior – technology must also be able to process information, or ‘think’, like or better than humans if it is to be fully relied upon. We want to make sure AI models are using the same process to recognize things as a human would, so we don’t just have the illusion that the system is working.”

This research would provide us with more accurate and reliable AI technology that will process information similar to the human brain…

Related contents:

Wednesday’s Child: Olive is an expressive teen who wants to study neuroscience FOX 11 Los Angeles

How Westpac’s ‘Life is eventful’ Effie winning campaign used neuroscience AdNews

.

Marketing Program Apps To Buy:

.