Sora涌现：OpenAI又一次暴力美学的胜利

The author Zhao Jiannian participated in a video theme generation meeting in Sharon Light-year ago, and there was an interesting interactive video generation meeting. The options were how fast to welcome the moment in half a year, a year or more. Yesterday, the accurate answer was released. Yesterday, a brand-new video generation model was released. With the performance advantages visible to the naked eye and the long video generation time, it is far ahead in the field of video generation after text and images. It is worth mentioning that we are one step closer to general artificial intelligence. The star company originally released a new video model yesterday, but its official tweet has been deleted rapidly due to the collision. The co-founder of one of the leaders in video generation tweeted that the competition had started. Yesterday, a technical document was released, but no genius innovation technology was released in terms of model architecture or training methods. It was more the optimization of the existing technical route, but the same secret as it was born more than a year ago was the tried and tested scaling law. When the video model was large enough, The problem of the ability to produce intelligent emergence lies in the violent aesthetics of large-scale model training, which is almost well known. Why is it the secret of data again this time? The technical route from generating video has mainly gone through four stages: circular network generation against network autoregressive model diffusion model. Today's leading video models are mostly diffusion models, such as autoregressive model, which has also become a hot research direction because of its better multimodal ability and expansibility. For example, Google released a new model in June. From the name, we can see that it combines the dual characteristics of diffusion model and autoregressive model. The architecture of this new model was put forward by the University of California, Berkeley and the University of new york in. In the technical document, a way of training video model with visual patches as video data is proposed. This is inspired by the big language model, which gracefully unifies various modes of text, codes, mathematics and various natural languages, while unifying images and videos to train a network. The dimension of low visual data This network receives the original video as input and outputs a potential representation that is compressed in time and space. It trains in this compressed potential space and then generates a video. It also trains a corresponding decoder model to map the generated potential representation back to the pixel space to represent the past image and video generation methods usually resize or trim the video to a standard size, which wastes the quality of video generation, such as second video with resolution. After digitizing pictures and videos, the original data of videos and images with different resolution duration and aspect ratio can be trained without compressing the data. This data processing method has brought two advantages to model training. First, the sampling flexibility can be used to sample the vertical video of widescreen video and all the videos in between directly create content for different devices with their native aspect ratio, and the prototype can be quickly made at a lower size before the video is generated at full resolution. Let's all use the same model. The second improved framework and composition. According to experience, it is found that training videos with the original aspect ratio can improve composition and framing. For example, the common model of cutting all training videos into squares sometimes produces videos with only part of the main body visible, compared with which the framing of videos is improved. The model on the left of the square crop is trained. At the level of language understanding, it is found that training highly descriptive video subtitles can improve text fidelity and video integrity. For this reason, we applied the re-captioning technology introduced in. First, we trained a highly descriptive caption generator model, and then used it to generate text captions for videos in the training data set. In addition, similar to this, we also converted short user prompts into longer detailed captions and sent them to the video model, which enabled us to generate high-quality video prompts that accurately followed the user prompts. Besides text, we also supported image generation, video generation and video generation. This function was enabled. It is enough to perform various image and video editing tasks, create a perfect circular video animation static image, and expand the video forward or backward in time. The secret of calculation is still violent aesthetics. The technical details of the model are not disclosed in the technical documents. Elon Musk once criticized that it is no longer the original intention when it was established, but only expressed a core idea. In 2006, he first proposed the secret of model training, and according to the model performance, it will continue to improve like Moore's Law on the basis of large computing power, large parameters and big data. It is not only applicable to language models but also to multi-modal models, that is, following this set of violent aesthetics, we found the emergence ability of large language models and finally developed epoch-making models, so it started the video in March without warning, showing excellent expansion characteristics in various fields, including language modeling, computer vision image generation and video generation. The following figure shows that the video quality is remarkable with the increase of training calculation scale under the same sample during the training process. In order to improve it, it is found that the video model shows many interesting new functions in large-scale training, which enables it to simulate some aspects of people, animals and the environment in the real world. The appearance of these attributes does not have any clear inductive deviation to objects, etc. It is purely a model scaling phenomenon, so naming the video generation model as a world simulator or a world model can be understood as making machines learn like humans understand the world, said NVIDIA scientists. Re-thinking is a data-driven physical engine, which is a simulator for simulating many worlds, whether real or fantasy. Through some de-noising and gradient mathematics, we can learn complex rendering, intuitive physics, long-term reasoning and semantic foundation. The chief scientist Yang Likun once put forward the concept of world model in January, and the official announced that the general world model declared that we would use generative methods to simulate the whole world, and only by having the ability to have the world model for a long time, we could express our result table. Ming-extended video generation model is a promising way to build a general simulator of the physical world. Specifically, the world model has three characteristics: consistency can generate videos with dynamic camera motion. With the camera moving and rotating, people and scene elements move in three dimensions in unison. A major challenge for video generation systems is to maintain time consistency when sampling long videos. Discovery is usually not always effective in modeling short-term and long-term dependencies. For example, the model can retain people, animals and objects even if they are blocked or leave the framework. 比特币今日价格行情网_okx交易所app_永续合约_比特币怎么买卖交易_虚拟币交易所平台

注册有任何问题请添加 微信：MVIP619 拉你进入群

文章转载注明 网址：https://netpsp.com/?id=57819