AI Flatland

Flatland

A decade ago, I was teaching 3D animation at RISD. While one can use 3D to teach critical thinking, I did not have experience as a character animator. So, I decided to practice with Mr. Buttons. Mr. Buttons was an old white guy confronting a changing world. The changes were subtle and ironic. With the advent of Veo3, an ai video generator, I decided to give it a go. I made the film a while ago. The film and the following is what it stirred in my head.

I admit, I am drawn to AI applications "trained" to generate images and movies. Like a fly to flypaper. I am afraid of losing what reputation I have as a creative with my friends. They tend to argue with me not to use it. They mention job loss. They talk about deepfakes. They argue plagiarism and the dataset taken without consent from working artists. They refuse to touch it, and they admonish me for doing so. They accuse me of undercutting my own creativity by styling an image after one of the great "master artists" or even someone run-of-the-mill. I know I am not alone in feeling "I cheat" when I use auto-select in Adobe Photoshop. I feel like a cheater when I use any number of things a photographer can do with AI tools. Furthermore, my cellphone has made me lazy with exposure. I am never concerned with how the relationship between incident and reflected light might impact exposure. The use of AI in image making is clearly a complex issue, and I believe I will never find out how to navigate the issues if I join the Neo-Luddites or the Degrowthers and not explore it. AI is here to stay.

Where did I put my ethical compass?

Why become a visual artist when images spill from every screen? For years a 35mm Nikon hung from my neck. Now it gathers dust on a basement shelf beside a lead-lined case of unexposed Kodachrome. Each time I find that relic, Paul Simon's song plays in my head and I refuse to regard the camera as mere junk. Early on, when phones recorded low-quality black-and-white images, I thought these things could never rival a fine-grain film. When I started working with video in 1972, I loved the blue, fuzzy image. I never expected I would be able to shoot 4K video with a thing I carry in my pocket, or that it would not matter whether the video was broadcastable, because that is no longer a goal. Friends have lost jobs because of the phone. I knew photographers who lost jobs as wedding photographers because of video sharing. The job of "photographer's assistant" has withered. The lucrative job of computer programming is vanishing in a puff. But I can't help myself, not because I want making art to be easy, but because AI is making chaos of norms.

Time marches on

I admire the Hollywood writers' strike for many reasons. The WGA was incredibly forward-thinking in demanding strong protections against the use of artificial intelligence to generate scripts, rewrite material, or use writers' past work to train AI models without consent or compensation. They recognized the existential threat AI posed to their livelihoods and creative integrity. Their success in securing these protections in their new contract was a historic win that has implications for many other creative fields. At its core, the strike was about asserting the value of human creativity and authorship in an increasingly automated and profit-driven industry. Writers argued that their unique talent, vision, and storytelling abilities are essential and cannot be replicated by machines or undervalued by studios.

When I "Ask Gemini" about my feelings of guilt when using AI in image making, it returns a block of text in a reassuring voice. "Most phone pictures are fleeting, documenting a moment without deep artistic intent. An artist, however, approaches their subject with a purpose, whether it is to evoke emotion, tell a story, challenge perceptions, or reveal beauty others might miss. Your choice of subject, composition, lighting, and even the 'decisive moment' is a deliberate act of creation, not just a casual snapshot."

Instinctually, I sense this is fundamentally wrong in the same way AI-generated images are banal. At the heart of my discontent is the fact that we can rate most of the available dataset on a bell curve. The voracious appetite for AI training data bloats the belly of the bell curve ever further. The machine is built to land in the middle. Ask it for a man in a derby hat and it hands back the man in a derby hat it has seen most often. Now, we can all become good, competent, average professionals.

I am finding it takes a lot of work to create more specific prompts in the hope of avoiding the bloat of generally accepted practice.

While Henri Cartier-Bresson's concept of the decisive moment is undeniably powerful for a certain type of street photography, it often overshadows the immense practice, perception, and iterative learning that underpin a photographer's true skill. Though insightful at the dawn of photography, over time it has become an average idea used by AI to placate my fear.

That framing completely ignores the idea of practice and the plastic brain. We tend to subvert the abstract recognition of a thing without examining it. Photography is based on the feedback between the seen reality and the resulting image. It is a kind of self-training. When we photograph often, when we paint, or draw, or sing, or write, or do anything over and over, we train our brains on the feedback of the process. The result is in our new ability to perceive what we so often ignore. To some, the image resulting from taking a picture is second to the brain training. I think this is more interesting than the common modernist idea of shape, texture, color and moment.

So, what did I learn in the practice of prompting AI to make this short movie?

I feel something stirring in my brain. It has to do with language. Lately I have been thinking I need to learn to talk again. Working on this short has made me more precise with writing prompts. For instance, my first prompt for Mr. Buttons was…

Mr. Buttons. He is wearing a blue suit with a blue derby hat. He is a bit sallow, pale skinned, large eyes, with a forlorn look on his face.

Since generating over a hundred images and short cuts, the prompt has become:

Mr. Buttons. He is Caucasian with pale skin. His face is clean-shaven. He is a man with a long and slightly rounded nose and large blue eyes. A pair of wire frames with tortoise shell rims framing his bright blue eyes. A gentle paunch softens his once-gaunt frame, and his shoulders are perpetually pitched slightly forward, a habit born from a lifetime of leaning in to observe the world's smallest details. He is dressed in his customary navy-blue wool suit, this is his only suit, the elbows bearing the faint honorable patches of daily wear. A white cotton shirt is accented by a plain robin's-egg blue silk tie. His signature navy-blue derby hat, its brim slightly misshapen from decades of use, sits neatly on his head. In his right hand is a worn black briefcase. At 73, his posture carries the slow caution of a man who has measured his life in the familiar rhythm of his own uncertain footsteps. He walks as if with each step he touches his toe into the vast unknown. Behind faint crow's feet, his large pale-blue eyes move from thing to thing, absorbing the morning's details: the commuters, the changing light, the rumble of the train bringing workers to an awakening metropolis. Mr. Buttons' thin lips are pressed into a contemplative line, withholding judgment as he takes in the full picture.

Clearly you can see the trajectory of brain training in these two prompts.

Even the hat is a lesson. Derby or bowler? I use the words interchangeably, and it took a machine asking for specifics to make me wonder which one I mean. Color went the same way. For the film I wrote a plain blue tie, then a uniformly robin's-egg blue tie. Now I write R55 G142 B174, because none of the robin's-egg swatches was the blue I needed. Naming a color is poetry, and as vague as Starlit Sandstone, the color I chose for the bedroom walls.

Still, Veo3 had problems. No matter how precise I got with the description of Mr. Buttons, there were issues.

It was hard to keep the character consistent from shot to shot with Veo3; each time the prompt generated Mr. Buttons, he became a different man. Originally, because of the suit and bowler hat, the AI made him sharp and confident. AI seemed not to be able to process the suit and bowler hat with the personality. More often than not he spoke with a British accent. So sometimes he was fat, sometimes he was self-assured, once he had a dirty suit, and oddly, when I added more trepidation to his movements, he became a Black man. This set me on my heels a bit. I tried to imagine how many Black men wore bowler hats in the training data. I read my first attempt and realized I didn't specify. No wonder Mr. Buttons was white in the first incarnations. I simply accepted his bowler hat whiteness. But then why did trepidation make him Black? Someone needs to work on that. And I need to take a deeper look at my own assumptions.

Other than the complete inconsistency in character, I have come to understand the fundamental problem with using AI to make a film. The bulk of the dataset is representative of our social bias. Even more subtle, the dataset of images is flat. A film needs a man who can turn around, walk into a room, and cast a shadow on the wall behind him. What the model has to work from is a heap of pictures, each one lying flat. So it makes up a new Mr. Buttons every time I ask. Building three-dimensional space out of flat pictures is a job with no good outcome, like asking directions and hearing, "You can't get there from here." Someone may find a road. I haven't seen one yet.

The AI world is flat.