Chicxulub meteorite found guilty; Good news and bad news for LLMs

It’s the last week before Christmas and not a creature is stirring, not even a mouse model bred to exhibit characteristics of ADHD for in vivo studies of central nervous system stimulants. This week, we reported on the discovery of the world’s oldest-known mammalian ancestor; the surveying of 1,300 burial mounds in western Azerbaijan; and a quasiparticle present in all magnetic materials. Additionally, researchers have shut down another alternate theory about the end of the dinosaurs; LLMs are becoming more brain-like, according to one study, but another one suggests that they exhibit signs of cognitive decline:

Rock, paper, volcano

We all know that a massive meteorite impact in the Gulf of Mexico killed off the dinosaurs 66 million years ago. But what some researchers presuppose is… maybe they didn’t? Actually, geologists have theorized that massive volcanic eruptions on the Indian Peninsula may have expelled enough ash and CO₂ into the atmosphere to cool the planet to uninhabitability.

But climate researchers at Utrecht University have now ruled out this scenario, finding that while the roughly contemporaneous volcanoes had a measurable impact on the climate, those impacts were different in degree, type and timescale compared to a meteorite impact.

The researchers analyzed soil particles in ancient peats to reconstruct air temperature across the time period covering both the volcanoes and the meteorite impact. They found that the volcanoes occurred 30,000 years prior to the impact and corresponded to a 5-degree-Celsius global atmospheric cooling, likely the result of atmospheric sulfur input. About 10,000 years later, temperatures were climbing back, probably aided by volcanic CO₂ emissions.

“These volcanic eruptions and associated CO₂ and sulfur release would have had drastic consequences for life on Earth. But these events happened millennia before the meteorite impact and probably played only a small part in the extinction of dinosaurs,” says Lauren O’Connor at Utrecht University.

12 brains agree

Researchers at Columbia University and Feinstein Institutes for Medical Research Northwell Health report that as LLM models advance in sophistication and function, they’re also becoming more brain-like. By creating a kind of book club for 12 recent LLMs and training them on a single text, the researchers extracted the internal representations of the text—called “embeddings”—and compared them to the results of a human experiment.

In the human experiment, they recorded the neural responses of neurosurgical patients as they listened to the text. Then the researchers attempted to predict the recorded neural responses to words from the word embeddings derived from the LLMs.

“The ability to predict the brain responses from the word embeddings gives us a sense of how similar the two are,” says Gavin Mischler, first author of the paper.

Using computational tools, they tried to assess which LLM layers aligned most closely to human neural responses—specifically, those in brain regions associated with language processing. They conclude that as LLMs become more powerful, their embeddings become more similar to the brain’s responses to language.

However

While it’s great that LLMs have their own researcher-mediated book clubs now, a report in the Christmas issue of the British Medical Journal finds that nearly all of the leading LLMs exhibit signs of mild cognitive impairment in standard diagnostic tests for dementia. According to the researchers, this is the first study of cognitive decline in large language models.

The Montreal Cognitive Assessment test consists of short tasks and questions to assess attention, memory, language and spatial skills; it is widely used to detect cognitive impairment and early signs of dementia. The instructions the researchers gave the LLMs were identical to those given to human patients; the LLMs were scored and the results were evaluated by a neurologist. They report that all of the chatbots exhibited poor performance in visuospatial skills, executive tasks and the clock drawing test. The models tended to score more highly in naming, attention, language and abstraction.

In a concluding leap of science-fictional conjecture, the authors write, “Not only are neurologists unlikely to be replaced by large language models any time soon, but our findings suggest that they may soon find themselves treating new, virtual patients—artificial intelligence models presenting with cognitive impairment.”

© 2024 Science X Network