GauGAN 2: an AI capable of translating words into images!

⇧ [VIDÉO] You may also like the content of this partner (after the ad)

Nvidia recently announced the release of the second version of its GauGAN artificial intelligence. He was already capable of transforming sketches into photorealistic images. Now you can generate them simply from words!

Putting into an image what we have in mind… It is a bit the goal of GauGAN. The latter had already offered the public an innovative possibility, that of generating photorealistic landscapes from user-drawn sketches. This time, with GauGAN2, scientists have chosen to give pride of place to words. With your new system, simply describe the scene you want to appear in a few words, such as “sunset on a beach”, to see the corresponding image appear.

Do you fancy a sunset on a mountain, on rocks or in rainy weather? Just specify it to change the landscape. And for its creators, AI can even make it possible to render fantastic landscapes with a little dexterity: “Imagine, for example, recreating a landscape of the iconic planet of Tatooine in the Star Wars franchise, which has two suns. All you need is the text ‘desert dune sun’ to create a starting point, after which users can quickly draw a second sun. “

After passing the words, users can adjust the image to their liking. Clicking a button creates an “image segmentation”. That is, a partition that classifies pixels into different elements. From there, the user can, going through the phases of sketching, erasing, adding elements, influencing the final image. For example, you can roughly trace the contours of a rock using the “rock” tag, to make a rock of the shape you choose appear in the base image. Being the iterative process, if words are added, they can also be integrated in the form of concrete elements, superimposing the existing image.

Landscape-based artificial intelligence

A base of 10 million different landscapes was used to “train” Nvidia’s artificial intelligence. To do this, they created a specific “artificial neural network” system for the operation of AI. Specifically, it is a system inspired by the functioning of biological neurons, which later addressed statistical methods. Artificial intelligence “feeds” on a large amount of data to extract logical connections and process them to obtain a result. Therefore, this AI’s food was very well cared for.

The model, Nvidia explains, is based on confrontational generative networks (hence the name GAN in GauGAN). In short, this is a kind of unsupervised learning algorithm, which is designed to generate realistic images. How it works: Two networks position themselves as “adversaries” and must somehow challenge each other. The first network generates a sample, in this case an image. Your opponent, the other network, is trying to determine if the proposed sample is real or if it was designed from scratch by AI. Therefore, this operation forces the AI ​​to make efforts to get as close as possible to reality and plausible images.

This amazing authoring tool could be used by artists, in particular to create benchmarks that exactly match their needs. Per Nvidia’s explanation, the goal is also to allow beginners to create landscapes from next to nothing.

We test it for you …

On the novice side, the demo Nvidia currently offers on its website is not always satisfactory. The passage of words to the image can give rise to some surprises, such as this stream in the middle of a dense forest that responded to the request “forest next to a lake”, that is to say “forest next to a lake”. From a lake “…

GauGAN2 AI generated landscape for a first “forest by a lake” request.

However, after a few tries (by modifying the style by clicking on the die icon), it is possible to get images that meet the demand of the landscape. Note that AI seems to have a preference for natural landscapes.

forest by a lake gaugan 2

New attempt at “forest by a lake”.

The results obtained for the “city with skyscrapers” type queries were at times quite uplifting. Regarding the integration of new elements through the sketches, the test was not very successful despite a certain relentlessness on the part of its dedicated journalist. GauGAN2 also sometimes shows some creativity in assembling concepts that it captures in images. Thus we were able to obtain this beautiful image, at the top of the article, worthy of the most beautiful postcards, simply entering “beach town next to a huge forest with big trees and clouds”, that is, “heavenly beach town next to a huge forest with tall trees and clouds’ in the text bar, then click the dice several times to change the style.

By spending more time there, there is no doubt that it should be possible to achieve even more uplifting results. As this is a demo, in any case we can expect the final version to be more intuitive. For the curious, the trial version is available in AI demos …

– Article revised and improved by Jonathan Paiano

A video demonstration:


PHP Script, Elementor Pro Weadown, WordPress Theme, Fs Poster Plugin Nulled, Newspaper – News & WooCommerce WordPress Theme, Wordfence Premium Nulled, Dokan Pro Nulled, Plugins, Elementor Pro Weadown, Astra Pro Nulled, Premium Addons for Elementor, Yoast Nulled, Flatsome Nulled, Woocommerce Custom Product Ad, Wpml Nulled,Woodmart Theme Nulled, PW WooCommerce Gift Cards Pro Nulled, Avada 7.4 Nulled, Newspaper 11.2, Jannah Nulled, Jnews 8.1.0 Nulled, WP Reset Pro, Woodmart Theme Nulled, Business Consulting Nulled, Rank Math Seo Pro Weadown, Slider Revolution Nulled, Consulting 6.1.4 Nulled, WeaPlay, Nulledfire

Back to top button