Dialogue on the Dark Side of the Moon: Kimi Model Supports 2 Million Characters of Lossless Input, Multi-Modal Model to be Released Within the Year

Article source:Titanium Media

Following the $2.5 billion valuation of the bombing, Yang Zhilin team again dropped "shock bomb".

Image credit: Generated by Boundless AI

Titanium Media App has learned that on the morning of March 18th, theDomestic AI startup Moonshot AI announced a new breakthrough in the large model of long context window technology, Kimi intelligent assistant has supported 2 million words of ultra-long non-destructive context, in just five months, "long text" input volume increased by 10 times, and from now on open product "Internal test" will be launched from now on.

On the morning of the 18th, Xu Xinran, vice president of engineering of the Dark Side of the Moon, told Titanium Media App and others that the order of magnitude increase in the length of the large model lossless context will also further help to open up the imagination of AI application scenarios, including the analysis and understanding of the complete code base, the intelligent body Agent that autonomously completes complex tasks in multi-steps, the lifelong assistant that does not forget key information, and the multimodal model of a truly unified architecture. The

Xu Xinran emphasized that over the past period of time Kimi is constantly improving the technical capabilities of large models, especially the demand for more complex models is increasing. And in the whole process, Kimi model itself and the user has been mutual promotion of each other's growth.

Dr. Shik Lin Yang, Founder of Dark Side of the Moon, said that lossless long context will be a key underlying technology if we lead to general artificial intelligence (AGI). All model architecture evolution throughout history has essentially been about improving effective, lossless context length. There may be a Moore's Law for context length, but both metrics, length and lossless compression level, need to be optimized at the same time for meaningful scaling.

The co-founder of the dark side of the moon Zhou Xinyu, on the other hand, told Titanium Media App that the dark side of the moon is about to launch its own multimodal large model within this year. At the same time, commercialization is also advancing rapidly.

When asked why not do multimodal models before, Zhou Xinyu responded, "If you come up with something that others also have, there is no new value for the world, we should not fall into the '100-module war' circle, we do not do to follow things."

Dark Side of the Moon was founded in March 2023 and is said to be theA key player in the domestic big model field. Its core team has been involved in the development of several large models such as Google, Huawei Pangu, Zhiyuan Wudao and so on.

Founder and CEO of Dark Side of the Moon, Shiklin Yang graduated from Tsinghua University with an undergraduate degree, and graduated from Carnegie Mellon University with a PhD majoring in computer science. He has worked at Google Brain (Google Brain) and the US startup FAIR, and studied under Apple's head of Artificial Intelligence, Ruslan Salakhutdinov, and has many years of entrepreneurial experience, and has collaborated with several Turing Award winners in publishing He has published papers with many Turing Award winners, and is the technical contributor of the earliest big models in China, such as Pangu and Wudao.

Meanwhile, Shilin Yang is also the most highly cited researcher in the field of NLP (Natural Language Processing) in China under the age of 35, and the first author of two important papers on Transformer-XL and XLNet - both core technologies in the field of large language modeling. The other two co-founders, Xinyu Zhou and Yuxin Wu, both have over 10,000 Google Scholar citations.

Personnel.The Dark Side of the Moon team now numbers over 80 people.

Financing.Within less than a year of its establishment, Dark Side of the Moon has completed two rounds of financing totaling more than $1.3 billion, with investors including Sequoia China, Zhenge Fund, Xiaohongshu, Meituan, Ali and others. A financing in February this year became the largest single round of financing received by a domestic AI big model company so far.

After two rounds of investment, Dark Side of the Moon is now valued at or $2.5 billion.

At the technical product level, since its establishment, Dark Side of the Moon has completed the layout from the generalized big model to the upper layer application.

Among them, the big model foundation layer, the dark side of the moon has trained hundreds of billions of levels of self-research general big model, and obtained the domestic big model filing approval; application layer, in October 2023, the dark side of the moon launched the world's first intelligent assistant product Kimi support input 200,000 Chinese characters, support for 200,000 Chinese characters of the long text input, the main non-destructive memory as well as the "Long Context ( Long Context", known by netizens as ChatGPT Chinese Pingtai, good at reading long text, searching the web, can be used for meeting minutes, auxiliary programming, copywriting and other scenarios.

According to SimilarWeb, the number of visits to Kimi increased dramatically after the Chinese New Year. According to public data, in January 2024, Kimi's intelligent assistant had 1.42 million visits, ranking first among the "AI ChatBots" products of large model startups; the month-on-month growth rate of 94.1% also ranked first among large model startups.

Xu Xinran said that Kimi may have an average growth rate of 100% or more every month at present.

At the meeting this morning, Xu Xinran announced that effective immediately, Kimi Chat and Moonshot Big Model under Dark Side of the Moon will unify their names and be renamed Kimi Intelligent Assistant and Kimi Big Model respectively.

"Let's just simplify it and unify it later so that everyone remembers Kimi," Xu Xinran told Titanium Media App and others.

Specifically, the first change that longer text input can bring compared to the previous 200,000 word length is the unlocking of more super long and complex tasks. What used to be limited to organizing 50 resumes can now grow linearly to 500.

The Dark Side of the Moon team put forward the "10-minute law" and pointed out Kimi's ability to learn a new field quickly, and that it only takes 10 minutes for an AI to approach the level of a junior expert in any new field, in a field that takes humans 10,000 hours to become an expert in.

Xu Xinran shows the printing thickness of a million-word book

For example, upload hundreds of thousands of words of Texas Hold'em tutorial documents, and then put forward a tournament opening, Kimi can analyze the situation on the playing field and provide guidance on card playing strategy; at the same time, Kimi can also read nearly a million words of Chinese medicine diagnosis and treatment manuals, "Zhen Huan Zhuan" novels, etc., and it can answer all of them.

In addition, in addition to English and Chinese, Kimi can also directly read the code base files, and then write a detailed and clear code base design document in Chinese, so that even the old and uncommented code can be quickly sorted out the structure.

For its part, Dark Side of the Moon says that feedback from many Kimi Intelligent Assistant users shows that the 200,000-word lossless long context has helped them open up new worlds of AI applications and bring greater value, but they still encounter situations where the length of conversations exceeds the limit as they attempt more complex tasks and decipher longer documents. This is a direct result of the need to continue to improve the lossless context length of large model products. In addition, the intelligent search of Kimi's intelligent assistant is even more dependent on the lossless long context capability of Big Model.

Dark Side of the Moon points out that it is because the commands users issue to Kimi are becoming more and more complex that the team has also been working to improve the complexity of the commands Kimi can follow, and the ability to retrieve information. At the same time, as the user's use of the scene from work gradually expanded to all aspects of life, the team in addition to the web terminal complemented the WeChat small program, iOS terminal and Android terminal and so on.

Xu Xinran reveals that Kimi is taking full advantage of her "silicon-based life" and continues to evolve at night.

Zhou Xinyu emphasized that, for the consideration of user co-creation, Kimi's positioning is more like an "intelligent assistant" than a "chatbot", because the ordinary dialogue on the big model of its own iterative help is limited.

In the post-session dialog exchange.Zhou Xinyu said, for the cue word tutorial of the high call, has been in preparation, is expected to be released in about a month; at the same time, the multimodal model is also in continuous development, while the audio processing capabilities, overseas version of the same in the demand pool.

Xu Xinran told Titanium Media App that the Dark Side of the Moon's AI Infra (infrastructure) team is also continuing to improve the energy efficiency ratio, all using self-developed technology. Kimi is said to be three times more responsive than when it was first released with the exact same hardware.

At present, Kimi intelligent assistant is still completely free. However, with the expansion of the user group and the growth of usage, it is inevitable that there will be a shortage of arithmetic power. Xu Xinran revealed that the first half of this year is expected to open the commercialization mode.

"I think this stuff is all paid content. Our thinking point is not to do (commercialization) in terms of value for money. Rather, it's about what problem we should be trying to help the user solve. In the case of being able to solve the problem, we will commercialization continuously open and planning, by then you (customers) will know which is better." Zhou Xinyu said.

(This article was first published in Titanium Media App, written by | Lin Zhijia)