In the training of large multi -modal models, recent developments have been driven by efforts to eliminate modeling barriers and unite architecture in domains. Despite these progress, many current models still rely on separate trained ingredients, such as specific encoders and decoders.
In a new paper JetFarmer: an automatic production model of raw images and textA Google Deep Mind Research Team has introduced a jetformer, a ground -breaking autoography, decoder only transformer designed to model raw data directly. This model maximizes the possibility of raw data without depending on any trained ingredients, and is capable of understanding and producing both text and images without interruption.
The team summarizes the keynote innovations in the jetformer:
- Take advantage of the routine flow for image representation: The main insight behind the jetforor is the use of a powerful routine flow. The raw image patch has been unprofessional due to the complexity of their structure as the traditional autonomous encoded pixels. The jetformer flu model indicates a damaged, vomiting representation that connects with a multi -modal model without interruption. In estimation, the overturning of the flow enables the straight image to regulate the simple image.
- Leading the model in high -level information: To focus on essential high -level information, researchers use two modern strategies:
- Progressive Goyce More Noise: During training, gouge noise is added and gradually reduced, encouraging the model to prefer the initial features in the learning process.
- Arrangement of spare pin in photo data: The jetfarrum allows the atoisigracing model to select useless dimensions in natural images. As an alternative, the principal component analysis (PCA) has been detected to reduce the dimension of important information.
The team reviews JetFarmer on two challenging works: Contemporary Class Common Image Generation and Web Scale Multi Moodle Generation. The results suggest that the jetformer is competitive with less flexible models when large -scale data is trained, which specializes in both image and text generation works. Its last training ability further highlights its flexibility and effectiveness.
JetFarmer represents an important leap in facilitating multi -modal architecture by combining modeling methods for text and images. It is a new era in productive modeling from the end of the modern use of normal use of its flow and emphasizing the preference of high -level feature. This research laid the foundation for more united multi -modal system, and pave the way for a more integrated and effective approach to the development of the AI ​​model.
Paper JetFarmer: an automatic production model of raw images and text Is on Archeo.
Writer: Hecate he | Editor: China Zhang