OpenAI：困于“遥遥领先”

Where did Altman, a large-scale series that caused a sensation all over the world, come to an end, but his troubles did not end. It is inseparable from Microsoft's busy work that he was reinstated in a short time. Since this year, Microsoft has been helping his good brothers to become bigger and stronger. Not only has it invested an additional $100 million, but it has also called the manpower of Microsoft Research Institute on a large scale to put down the basic scientific research projects at hand and fully put the basic large models into products and arm them to the teeth. But what many people don't know is that the person in charge of Microsoft Research Institute received it this month. The first substitute created by a secret project was Microsoft's first large model application. It was revealed that Microsoft was trying to gradually replace the large model originally integrated into it with a self-developed version. At the developer's conference last month, Microsoft announced that it had changed its name to today's market positioning, which was quite similar. It was hard not to make people think about brand-new, but Microsoft's original intention was not that its technical ability was flawed, nor did it foresee the differences of management. The real reason was a bit ridiculous because its technical ability was too strong to drive Rambo. The opportunity of Gini's take-away delivery to promote Microsoft's self-developed large model is a failure. At a time when the world is sensationalized, computer scientists are busy with a project code named "benchmarking", hoping to build a sparse model. This is a special super-large model. Only certain parts of the model will be activated when processing tasks. For example, when users need to generate a summary, the model will automatically activate the part that is most suitable for the job, and it is not necessary to mobilize the whole large model every time. Compared with the traditional large model sparse model, it has a faster response. Speed and higher accuracy, more importantly, can greatly reduce the cost of reasoning. In Chinese, it means killing a chicken without using a bull's-eye knife, which is exactly what Microsoft values about Google's summary of the advantages of sparse models. When talking about the cost of large models, public opinion always talks about the training cost of single digits and astronomical expenses, but for most technology companies, model training and data center construction are only one-time capital expenditures, which is not unacceptable for the expensive reasoning required for daily operation. This is the first threshold to prevent technology companies from going further, because under normal circumstances, large models don't have obvious scale effect like the Internet. Every query of users needs new reasoning calculation, which means that the more users use products, the more serious the computing cost of technology companies will increase exponentially. Previously, Microsoft charged US dollars for assisting programmers to write codes based on the transformation of large model applications. According to the Wall Street Journal, due to the expensive reasoning cost, per capita lost money every month. Heavy users of US dollars can even bring the loss of US dollars to Microsoft every month. The big model application can't make ends meet, which is the primary reason to promote Microsoft's self-developed big model. The big model is still far ahead in technology and has long been at the top of the major lists, but at the cost of expensive use. Some researchers have done a calculation, and the theoretical price is almost twice the reasoning cost of the open source model, not to mention the comprehensive upgrade. However, except for a few scenarios such as code generation and solving complex mathematical problems, most of the work is done. Actually, it can be handed over to the version and open source model with smaller parameters. The startup company is a living case. Its business is to provide tools for summarizing audio and video content. It has been used by users for about 10,000 months to support its services in the early days. Later, the company tried to replace the underlying large model with open source and found that users did not perceive the difference, but the monthly reasoning cost was reduced from US dollars to less than US dollars, that is to say, Lamborghini, which provides customers with undifferentiated power, but the business of most customers is actually Delivery of take-away food constitutes a far-ahead problem, so not only Microsoft's early big customers have been replaced by cheaper technical solutions, reducing the reasoning cost and getting out of the way. Odibe and Yadea are cheaper, which has become a subject that must be solved. This is the sparse model project mentioned above. In fact, not only Google is also engaged in related research and has made progress. At the conference last month, Jeff Dean, Google's chief scientist and former head of Google's brain, even mentioned in his speech that sparseness will be the most important in the next decade. Jeff Dean, one of the important trends, has also published a paper on sparse models. It is the high cost brought by being far ahead that makes Microsoft ponder the possibility of self-reliance. In fact, it has also noticed this problem. At the developer conference on March, Lamborghini, whose price reduction was lower than that of the closed-source large model developed by its biggest competitor, was introduced. Although it was not cheap enough, it was at least more affordable than other cars. Unfortunately, a farce that was enough to be included in the history of science and technology a few days later was making this effort greatly discounted. It was revealed that on the weekend when Altman negotiated with the board of directors to return, more than 100 customers had contacted friends, and the paradox of commercialization would still exist even without this civil strife. This should start from the new model and product design ideas. Soon after, he went to the developer community and put in this blockbuster. Users can customize the chat robot with different functions by using natural language. By the day when Altman was reinstated, users had uploaded a chat robot with different functions, and the average daily activity was comparable. Compared with a large community, the well-known model with different functions is not open source, and it is far ahead of the problem, but for individual developers and small businesses, it has incomparable advantages over the two open source models. One is the low development threshold of out-of-the-box use. In overseas forums, some small teams that use the basic model to develop their own products will describe them as wrapping paper. Because of the powerful universal ability of the model, they sometimes only need to develop a model and find a suitable scenario to get orders, such as developers. If the model needs to be further fine-tuned, a low-rank adaptive lightweight model fine-tuning technology is also provided. Simply put, the general principle is to disassemble the large model first and then do adaptive training for the specified task, so as to improve the ability of the large model under this task. The main focus is on adjusting the internal structure of the model, which does not require much industry data for fine-tuning. However, when customizing the open source model, developers sometimes use full-scale fine-tuning. Although it performs better on specific tasks, full-scale fine-tuning needs to be updated and pre-trained. Every parameter of the large-scale model requires a very high amount of data. In contrast, the model is obviously more friendly to ordinary developers. Secondly, it has been mentioned that the large-scale model does not have scale effect, but there is actually a premise in this sentence, that is, when there are enough computing requests, the test shows that the less computing requests are sent to the server in each batch, the efficiency of computing power utilization will decrease, which will lead to a sharp rise in the average cost of a single calculation. It is possible to bundle millions of computing requests from all customers at one time, but it is difficult for individual developers and small and medium-sized enterprises to do this because there are not so many active users. 比特币今日价格行情网_okx交易所app_永续合约_比特币怎么买卖交易_虚拟币交易所平台

注册有任何问题请添加 微信：MVIP619 拉你进入群

文章转载注明 网址：https://netpsp.com/?id=62718