数学测试难倒了人工智能模型:9.90 和 9.11 哪个数字更大? || Maths test stumps AI models: which number is bigger, 9.90 or 9.11?

AI Summary:

---------------

内地歌手孙楠获得了 13.8% 的网络投票,超过了获得 13.11% 投票的美国歌手 Chanté Moore。一些本地网民对这一排名进行了嘲讽,称后一数字更大。一位评论者建议询问人工智能。他们得到的结果喜忧参半。

吴说,GenAI本身并不具备数学能力,只能根据训练数据预测答案。他说,一些 LLM 在数学测试中表现出色可能是因为 "数据污染",这意味着算法记住了答案,因为类似的问题已经出现在其训练数据中。

"人工智能的世界是标记化的--数字、单词、标点符号和空格都被同等对待,"吴说。"因此,提示符的任何变化都会对结果产生重大影响。

数学问题表明,不仅在中国大陆,在世界其他地方,人工智能技术也在不断发展。

"绝大多数专家认为,由于人工智能技术发展如此之快,制定统一的国家人工智能立法的时机可能还不成熟,"Zheng 说。

古德赛德在 X 上发表文章说,他无意削弱法律硕士,而是旨在帮助理解和弥补他们的失败。

"他写道:"以前众所周知的 LLMs 问题(如糟糕的数学)现在得到了很好的缓解,剩下的错误让用户感到震惊--任何频率的降低也是严重性的延迟增加。"他写道:"我们应该做好准备,迎接这种情况在许多任务领域不断发生。
---------------

Mainland artist Sun Nan received 13.8 per cent of online votes to edge out US singer Chanté Moore, who received 13.11 per cent of votes. Some local netizens poked fun at the ranking, claiming that the latter number was larger. Ask AI, one commenter suggested. The results they got were mixed.

GenAI does not inherently possess mathematical capabilities and can only predict answers based on training data, according to Wu. He said some LLMs perform well on maths tests possibly because of “data contamination”, which means that the algorithm memorised the answers because similar questions were already in its training data.

“The world of AI is tokenised – numbers, words, punctuations and spaces are all treated the same,” Wu said. “Therefore, any change in the prompt can affect the result significantly.”

The maths issue shows that AI technology continues to evolve not only on the mainland, but elsewhere around the world.

“The vast majority of experts believe the timing to craft unified national AI legislation may not yet be right since the technology is evolving so rapidly,” Zheng said.

In a post on X, Goodside said he does not intend to undermine LLMs, but aims to help understand and fix their failures.

“Previously well-known issues in LLMs (e.g., bad maths) are now mitigated so well the remaining errors are newly shocking to users – any reduction in frequency is also a delayed increase in severity,” he wrote. “We should be ready for this to keep happening across many task domains.”

via News - South China Morning Post (author: Wency Chen)
 
 
Back to Top