This article is part of our coverage of the latest AI research.
Amidst its wave of layoffs and falling stock price, Meta (Facebook) faced another crisis after releasing its latest AI announcement: Galactica.
Galactica is “a large language model that can store, combine and reason about scientific knowledge,” according to an article published by Meta AI. It’s a transformer model that’s been trained on a carefully curated dataset of 48 million documents, textbooks and lecture notes, millions of compounds and proteins, scientific websites, encyclopedias, and more.
Galactica was supposed to help scientists navigate the vast amount of published scientific information. Its developers touted it as capable of finding citations, summarizing academic literature, solving math problems, and performing other tasks that help scientists research and write papers.
🪐 Presentation of Galactica. A great linguistic model for science.
It can summarize academic literature, solve math problems, generate Wiki articles, write scientific code, annotate molecules and proteins, and more.
Explore and get weights: pic.twitter.com/niXmKjSlXW
— Papers with Code (@paperswithcode) November 15, 2022
In collaboration with Papers with Code, Meta AI used Galactica and launched a website that allowed visitors to interact with the model.
However, three days after Galactica’s release, Meta had to shut down the online demo after a deluge of criticism from scientists and tech media about the model’s incorrect and biased output.
While Galactica was obviously not a success, I think its brief history provides us with some useful lessons about LLMs and the future of AI research.
1- Be careful how you present your model
Image source: 123RF
Large language models represent impressive advances in artificial intelligence and have even become the basis of several commercial products. In the past two years, LLMs have continued to push the boundaries of what is possible with deep neural networks. Galactica is no exception. If you read the paper, there is a lot to learn about how to curate data, configure tiles, and adjust the model architecture to do more with less.
However, LLMs are also a controversial topic. When it comes to topics like understanding, reasoning, planning, and common sense, scientists are divided on how to evaluate LLMs. Some dismiss LLMs as stochastic parrots, while others go so far as to consider them sensitive.
What is clear is that even if we want to attribute some form of intelligence to LLMs, we must recognize that it is fundamentally different and largely incompatible with human intelligence, even if on the surface it produces results very convincing
This, unfortunately, is where I think Meta AI went wrong. In their paper, they used some of these contested terms, such as “reason about scientific knowledge.” And on Twitter, the model presented herself in a way that created the impression that she can write her own scientific papers.
To their credit, Meta and Papers with Code explicitly state on the Galactica website that “There are no guarantees for truthful or reliable output from language models, even large formats with high-quality data like Galactica.” They also recognize that Galactica works best when used to generate content about well-quoted concepts. And they warn that in some cases, Galactica could generate text that looks authentic but is inaccurate.
But the use of vague terms in the paper, website and tweets was enough to overshadow these warnings and provoke a backlash from scientists and researchers who are (rightly) exhausted by the unwarranted hype surrounding the grand language models. (I won’t go into those criticisms here because they’ve been extensively reported by the tech media.)
I asked #Galactica some things I know and worry about. In every case, it was wrong or biased, but it sounded right and authoritative. I think it’s dangerous. Here are some of my experiments and my analysis of my concerns. (1/9)
— Michael Black (@Michael_J_Black) November 17, 2022
2- Benchmarks are often misleading
Benchmarks are one of the thorniest problems in AI research. On the one hand, researchers need a way to evaluate and compare their models. On the other hand, some concepts are really difficult to measure.
Galactica makes impressive progress on some of the benchmarks used to measure the reasoning, planning, and problem-solving capabilities of AI systems. With a maximum size of 120 billion parameters, Galactica is considerably smaller than other state-of-the-art language models such as GPT-3, BLOOM, and PaLM. But according to Meta AI experiments, Galactica outperforms these SOTA models by a comfortable margin on benchmarks like MMLU and MATH.
However, the problem with these benchmarks is that we typically view them from a human intelligence perspective. As a simplified example, take chess, which was long considered the ultimate challenge for AI. We view chess as a complicated intelligence challenge because, on the way to chess mastery, humans must acquire a set of cognitive skills through hard work and talent. This is why we expect chess masters to be able to make intelligent decisions about a larger set of tasks that require long-term planning but are not directly related to chess. But from a computational perspective, you can take a shortcut to finding good chess moves using pure computation, a good algorithm, and the right inductive biases. And you don’t need any of the general intelligence skills that human chess masters have.
Scientists go to great lengths to create benchmarks that cannot be “cheated” by computational shortcuts. But it is a very difficult feat. Computer scientist Melanie Mitchell has extensively studied the shortcomings of the benchmarks used to evaluate reasoning in deep learning models. And according to their findings, even some of the most carefully crafted benchmarks are prone to computational shortcuts.
This means that while benchmarks are a good tool for comparing machine learning models against each other, they are not anthropomorphic measures of cognitive abilities in machines.
3- Recognize the limits and powers of the LLM
One of the biggest challenges of large language models is that they can create results that are human-convincing but not based on human cognition. Models like Galactica can be extremely powerful but also dangerously deceptive.
As some researchers have pointed out, the production of Galactica may seem real but not based on actual facts. This doesn’t happen all the time, but it happens often enough to make you want to review the suggestions provided by the language model instead of blindly accepting them. This applies not only to Galactica, but also to other LLMs used for reasoning and problem-solving tasks such as source code generation.
But does it mean that Galactica should be dismissed as useless in math, science and programming? Absolutely not. In fact, there is plenty of evidence to show that LLMs, with all their shortcomings, can be very effective tools. Take GitHub Copilot, a programming AI tool powered by OpenAI’s Codex model. Several studies show that Copilot makes the work of programmers much more pleasant and productive.
That said, I’m a little disappointed with the scientists and media who jumped on the Galactica failures as an opportunity to attack deep learning, large language models, and the work done by Meta researchers. With the right interface and rails, a model like Galactica can be a good addition to scholarly search tools like Google Scholar.
(Of course, it also generates misquotes most of the time, so it’s more of a “hey, this might be an article to refer to, or it might be totally made up” situation than a “here’s your quote”).
— William Ryan (@thewilliamryan) November 18, 2022
Otherwise, we should regard Galactica’s initial failure as another science experiment. And as the history of scientific discovery has shown time and time again, every failed experiment brings us one step closer to success.