The Meta AI Bot contributed mock research and nonsense before it was pulled offline

meta I paused it Artificial intelligence (AI) Bot last week, just two days after it was released to the public. The bot, called Galactica, has been trained on 106 billion codes of open-access scientific data and texts. This includes papers, textbooks, scholarly websites, encyclopedias, reference materials, knowledge bases, and more,” the company said The Daily Beast.

It was supposed to help academics and researchers find papers and monographs quickly and concisely, but instead was overwhelmed by massive amounts of misinformation that incorrectly cited eminent scientists.

Scientists’ reputations can be put on the line when they are incorrectly cited in the text, said Carl Bergstrom, a professor of biology at the University of Washington. Cnet The problem with Galactica is that it has been promoted as a way to get facts and information. But instead, he said, it acted like a “fantasy version of the game where you start with half a sentence, and then let autocomplete fill in the rest of the story.”

Within hours of Galactica’s launch, users began reporting racist and inaccurate articles, with one person Post a photo To respond to a request regarding language bias. The bot’s response falsely claimed that black people “have no language of their own” and immigrants “don’t speak a language different from that of the country to which they immigrate”.

Other information generated included a bogus study about the benefits of eating crushed glass and bogus information about researchers at Stanford University creating a “gaydar” program for Find gay people on facebook.

A Meta AI spokesperson told CNET, “Galctica is not a source of truth, it is a research experiment that uses [machine learning] systems to learn and summarize information.” He added, Galactica “is exploratory research that is short-term in nature with no product plans.” Meta AI chief scientist Yann LeCun told the outlet that the bot was removed because the team was “too dumbfounded by the vitriol on Twitter.”

Two days after Galactica’s launch, the bot and Meta’s chief AI scientist were temporarily suspended by the Meta AI team Yann LeCun tweeted“Galactica demo is currently offline. It is no longer possible to have some fun by misusing it. Happy?”

Meta AI (ex Call Facebook AI search) developed Galactica as a method To “organize science” and condensation Huge amount of scientific information found on the Internet. The idea was to have artificial intelligence Doing things like solving math problems, writing scientific code, and drafting research abstracts.

meta It is not noted on the Galactica website that the AI ​​model has limitations that can cause “hallucinations”. The site advises users to check any information that pops up and continues to say “There are no guarantees of real or reliable output from language models, even large ones on high-quality data like Galactica,” he said, adding that the generated text may look “very authentic and very confident,” but it could be faulty.

“I imagine that even with many of the expected flaws, there are desirable uses for such a system,” Vincent Konitzer, a professor of computer science at Carnegie Mellon University in Pittsburgh, told The Daily Beast.

“My impression is that Meta could have done better by putting more effort into this public release, by doing more serious user studies first, calling attention to desirable uses of the system, and being honest and upfront about unwanted uses.”