AI against artists? : Here’s what DALL-E 2 just can’t do


(This article by an engineering professor Karl D.Stephan originally appeared at Engineering Ethics Blog (August 14, 2022) under the title “AI Illustrator still looking for work: DALL-E 2 shortcomings” and is reproduced with permission.)

The field of artificial intelligence has made great strides over the past two decades, and each time a new breakthrough in AI is announced, critics worry that another area of ​​activity human has fallen victim to automation and will vanish from the earth when machines replace the people who do it now.

Karl D.Stephan

Until recently, the art illustrator profession seemed reasonably immune to the onslaught of AI innovations. Only a human, it seems, can start with a set of plain language verbal instructions and come up with a finished work of art that fulfills those instructions. And that was mostly true until the AI ​​folks started tackling this problem.

The AI ​​Research Lab Open AI made his work known in this area using a “transformer” type program which has had two versions so far, DALL-E and now DALL-E 2. I’m sure the late surrealist artist Salvador Dali would be delighted with the honor of having a robot artist named after him, but the connection between his works and the productions of DALL-E 2 is perhaps closer than researchers would like. In an article by IEEE Spectrumjournalist Eliza Strickland points out the shortcomings of DALL-E 2 productions and speculates on the general usefulness of these systems.

As is the case with most such systems mimicking human tasks, the researchers’ first step was to compile a large number of examples (650 million in the case of DALL-E 2) to train the software to appearance of illustrations. The images and accompanying descriptive text are all sourced from the internet, which has its own biases, of course. Researchers learned from previous fiascos with AI software that allowed random users to request sketchy or offensive products, so they were very careful to pre-screen training images and (in some cases) manually censor what DALL-E 2 comes with. They have not yet released the program for general use, but have carefully screened users under controlled conditions. If you just let your imagination run wild with the scenario of letting horny teenagers go wild with a program that will make a picture of anything they describe to it, you can see the potential for abuse.

For some purposes, DALL-E 2 is quite suitable. If you want a generic type of image that would look good as filler for a brochure about a meeting, and just want to show a few people in a corporate setting, DALL-E 2 can do that. But so are tons of free clipart websites. If you want an image that conveys specific information, however – a diagram, say, or even text – DALL-E 2 tends to fall flat on its digital face. The Spectrum journalist had the privilege of making a few text requests to DALL-E 2 for specific images. She asked for an image of “a tech journalist writing about a new AI system capable of creating some remarkable and weird images.” She retrieved three photo-like photos, but they were all of guys, and only one seemed to have anything to do with AI. Then she asked for “an illustration of the solar system, drawn to scale.” All three results had a sun-like thing somewhere, and planet-like things, and white circular lines on a black background showing the orbits, but the number of planets and their appearance were pretty random. So technical illustrators (those who stay after software like Adobe Illustrator made every man or woman their own illustrator) doesn’t need to file for unemployment insurance right away. DALL-E 2A still a long way to go.

There is a fundamental question lurking behind AI feats like this. This can be phrased in many ways, but it basically boils down to this: will AI software ever show anything that amounts to human-like general intelligence? And believe it or not, your humble scribe, with Gyula Klimaa philosopher then at Fordham University recently published an article addressing this question, and we concluded that the answer was “No”.

As you can guess when a philosopher gets involved, the details are somewhat complicated. But we started from the idea that the intellect, which is a specific power of the human mind, is based on the use of concepts. In the limited space I have, I can best illustrate concepts with examples. The specific house I live in is a special thing. There is only one house exactly like mine. I can remember it, I can form a mental picture of it, and I can even imagine it with a different trim color than it actually has. And software can also do what amounts to this kind of mental operation. In my mind, my house is a perception of a real, individual thing.

However, take the concept Of house.” Not my house or your house, just “house”. Any mental image you have that is inspired by “house” is not the same as “house” – this is just one example. The idea or concept denoted by the word “home” is not reducible to anything specific. The same goes for ideas such as freedom or conservatism. You cannot draw a picture of conservatism , but you can draw an image of a particular curator.

In our article, we gave strong evidence in favor of the idea that because concepts cannot be reduced to representations of individual things, AI programs will never be able to use them. In the article, we used examples of an art-generating AI program that resembles DALL-E 2 in some ways, in that it was trained on thousands of artworks. art, then designed to generate images resembling works of art. The results showed the same kind of fidelity to superficial detail and complete lack of underlying consistency that the DALL-E 2 productions showed. As one of the OpenAI researchers quoted in the Spectrum the article noted, “DALL-E doesn’t know what science is. . . . [S]o he tries to invent something that is visually similar without understanding the meaning. That is to say without having any idea of ​​what he is doing.

The big question is whether new AI research will produce programs that truly understand the concepts and use that understanding to guide their production of art, text or otherwise. Klima and I think not, and you can check out our article to find out why. But we could be wrong, and only time and further AI research will tell.

Sources: Eliza Strickland’s article “DALL-E 2’s Failures Reveal the Limits of AI” appeared on pages 5-7 of the August 2022 print issue of IEEE Spectrum. “‘Artificial intelligence and its natural limits” by Karl D. Stephan and Gyula Klima appeared in vol. 36, no. 1, p. 9-18, from AI & Society in 2021.

You can also read:

What can AI text-to-image generators do for artists? Developer David Holz talks about his Midjourney generator in terms of communities that grow around art generated from words or phrases. Most of the art seems insignificant or meaningless, as one would expect from mixing images without creativity, but mass-mixed images could dispel an artist’s meltdown.


The new OpenAI art program does NOT claim copyright for the AI. As DALL-E 2, which generates shuffled images in response to keywords, enters the art world, a key question has just been settled. That’s fine, says Robert J. Marks, author of Non-Computable You: “AI-generated images should be no more copyrighted than Google search engine results.”


Comments are closed.