Artificial intelligence for generating images is a fascinating tool with great potential. I have addressed some issues regarding the ethics of using these tools in this post, "Intelligenza artificiale e arteArtificial Intelligence and Art - Part 1". Now that I've got the basics down, though, I'd like to move on to the more practical part: what can artificial intelligence software do with my drawings from the world of iCosini?
The birth of the project
Diego, a friend of mine who is passionate about technology and IT, contacted me to propose a project: "I have artificial intelligence software on my computer, will you give me permission to use your designs to understand the possibilities and limits of this technology?"
We discussed this for a while, and I admit that I was a little afraid: what if my drawings, which I cherish, then get out of hand? After weighing up the possibilities, I still allowed myself to be tempted by the possible advantages. And from there, a world opened up.
1. The Starting Drawing: Quietanzare
Years ago, Diego gave me the opportunity to look inside the cockpit of a train: I was fascinated by that console, completely incomprehensible to my eyes. Buttons, knobs, lights and screens. A very unusual vision, for someone like me who gets on the train as a passenger.
Photo: a cockpit, or driver's desk, of a Tilo train - source tilo.ch
But as often happens to me, the filter of iCosini took over, which makes me see the world in an imaginative and creative way. From that vision came the drawing 'Quietanzare', dense with details and wacky ideas, then printed in very few copies. You can watch the creation video of this drawing by clicking here..
2. The education of artificial intelligence
Starting from this drawing, Diego tried to 'train' and 'instruct' his artificial intelligence to understand what was represented in my drawing and what the volumes and spaces were.
First, he selected the part of the drawing he was interested in, thus eliminating all the side lettering present in my prints (which, by the way, is the real hilarious part of this painting, in my opinion, but useless for the purpose of this project).
He then created simple elaborations to emphasise the walls and the window in front. The perspective in my drawing is far from perfect, so it must not have been easy to understand it.
Photo: study of the parts of the image, walls and window, based on both the final drawing and the preliminary sketch I had made
The drawing was processed and studied by computer, creating different 'maps': for example, one with the contours of objects or with volumes and shadows. In addition to these elaborations, Diego also provided some conceptual images of cockpits or control rooms for reference. A real learning and teaching process: both by the computer and by the person who has to give direction to the artificial intelligence, the conductor.
Photo: some 'study' maps of my design
3. Image generation
Hence the great job of 'guiding' the artificial intelligence to create something interesting. Everything is done through written descriptions and commands relating to the generation models, details that for me remain encapsulated in a sense of magic and mystery.
An example of an inserted description:
a black triangle standing in front of a control room inside a train full of all sorts of odds and ends, console, a window in the background with railroad tracks and green grass
Negative prompt: out of frame, censored, (asian), ((painting)), ((frame)), (((cropped))), (((watermark))), ((logo)), ((barcode)), ((UI)), ((signature)), ((text)), ((label)), ((error)), ((title)), stickers, markings, speech bubbles, lines, cropped, lowres, low quality, artifacts, watermark, signature, , signature, cartoon, 3D, render
Steps: 20, Sampler: Euler a, CFG scale: 7, Seed: 3535552657, Size: 1024x512, Model hash: c35782bad8, Model: realisticVisionV13_v13VAEIncluded, ENSD: 31337, ControlNet-0 Enabled: True, ControlNet-0 Module: scribble, ControlNet-0 Model: control_sd15_scribble [fef5e48e], ControlNet-0 Weight: 1, ControlNet-0 Guidance Start: 0, ControlNet-0 Guidance End: 1, Score: 6.08
Or something like this:
a character standing in front of a control room inside a train full of all sorts of odds and ends, console, a window at the end of the room with railroad tracks, octane render, 8 k, exploration, cinematic, trending on artstation, by beeple, realistic, 3 5 mm camera, unreal engine, hyper detailed, photo - realistic maximum detail, volumetric light, moody cinematic epic concept art, realistic matte painting, hyper photorealistic, concept art, volumetric light, cinematic epic, corona render, movie concept art, (light particle:1.2), (game concept:1.3), (depth of field:1.3), global illumination,Highly Detailed,Trending on ArtStation, Dreams of happiness, very beautiful, inspiring, thought provoking, digital art, Intricate, Elegant, Scenic, Hyper-Detailed, Delicate; Complex, surreal concept art, aesthetic, smooth, sharp focus
The results are as varied as Diego's attempts to steer the machine towards something sensible and pleasant. The purpose of all this was to assess the potential and limits of this tool: a classic experiment of a new tool that is still unknown.
Photo: some of the first results
Conclusions (of this first test)
I will draw some conclusions right away, before leaving you with a roundup of the images generated in this experiment: as I had already expressed in the post Part 1 on artificial intelligence, this new tool offers incredible opportunities for a relatively small amount of time (don't hold it against Diego, who spent hours and sleepless nights working on the project!). How much time would I have to spend learning how to use an image rendering tool to create a model with so much detail? Moreover, it must be considered that this intelligence has only studied a few images... what is the potential of an intelligence studying all the images on the Internet?
The thing that made my heart beat faster was to see that image I had in my mind come back into reality: when I saw the train driver's bench, I made a mental interpretation of it, which I then reproduced in the form of a sketch, or ink drawing. I was the first filter from reality to fantasy. But the artificial intelligence seems to have taken it a step further, trying to render and bring back to reality that image that I only managed to sketch in ink, going even further than I had imagined.
I am thrilled, I am fascinated, and I thank Diego for this somewhat mystical experience: I know it is only the beginning, other experiments and projects will surely follow. Let me know what you think of this project, or if you have any ideas about other experiments.
Below, now, I leave a roundup of the most beautiful, strangest or most interesting results, with some comments.
Some of the results of the experiment
Some results a little dark, almost disturbing. The train seems to be formed with pipes or wire... but who is that little man? Ahah! The ninja has turned into a dark-skinned man with a hood!
In these images, the cockpit has turned into a room, with waiting seats, and even a person. Is it a friend of the ninja? Judging from the outside, however, it looks more like a spaceship.
In these examples, the buttons, and lights, and control knobs are spectacular. In the central image, an expanse of water can be seen.
The style here is given by the addition of 'Art Nouveau' and 'Fred Calleri' in the description: it is far from what I had imagined, but I find it extremely interesting. I like the walls decorated with those inlays.
A strange, almost abstract elaboration.
Here there seems to be a floor covered with a finely decorated carpet. The buttons are on the walls, to the right. On the left, however, the roundel has been transformed: it was originally a clock, but here I cannot tell whether it has become a window or a mirror.
One of the difficulties encountered was making the artificial intelligence realise that the one at the back, in front, is a window showing the outside environment. Here in fact it is a wall, with buttons and various machines on it. What I like is the moss, as if it were a small forest or an ecosystem.
I love this representation: it seems to tell me 'Debora, stop dreaming strange things, what you have seen and represented is just a triangular object leaning against a window'!
Something psychedelic, no? But yes!!!
This one made me laugh too much: on the left, in the original drawing, there is a clock and a 'quality inspector' who in my head is supposed to be checking the driver... in this representation, the two have merged and a creepy face with some kind of helmet has appeared.
This is one of my favourite representations: it really looks like a complicated machine as I had imagined it. Full of wires, buttons, lights and handles. A bit of a steampunk style. The window doesn't show the outside, which is a pity, but the rest is very close to what I had thought.
And this is the futuristic science-fiction version of what I had in my head: a transparent, glass vehicle with a thousand knobs and machines. With this atmosphere of dream and future. I love it, and I really wish I had thought of it.
If you want, let me know which image you prefer and why. Isn't artificial intelligence a fascinating tool?
COMMENTS