A lot of my work involves the creation of eCommerce sites, and apps that have a fairly modest, mainstream purposes, that meet the market needs of today. Well, this project was the polar opposite of a typical project.
The app was envisioned to work by taking user input in the form of search keywords. Creative work would then start getting generated. To start with, static images were generated. Eventually, the app was envisioned to generate not just images on-the-fly, but music, videos, posters and entire websites would be generated, live, by the machine learning algorithm. Digital magic, basically.
Reimagining digital creation
This project is a futuristic vision. A moonshot attempt at creating something very, very ambitious — the first product in a completely new era of digital products. One that would change the face of the creative industry throughout the entire world. The biggest thing since Photoshop!
The product was ambitious, and in hindsight, over-ambitious given the resources that we had. It was fascinating to explore uncharted lands in the area of human computer interaction, and as a designer I found this to be a great learning experience.
Possibly the greatest machine learning superpower that will transform the creative industry is the ability to take a creative asset, and apply a completely different style onto it. Say you want a poster. And in that poster you want a picture of a tiger that matches a Japanese Zen theme. With style transfer, you can take any regular photo of a tiger, and have the machine learning algorithm "Zenify" it for you.
All these style transfers would be live generated, by the software, in seconds. You could request a picture of "a baby but with superhero wings flying over a metropolitan skyline", or "take my photo and mix it with Elvis Presley, and have me riding a camel into a desert sunset". The possibilities are limited by your imagination.
This would be a big deal for marketing and advertising agencies who could explore artistic ideas in record time. In fact, people could bounce these ideas around, and have the software generate ideas live as people speak.
Endless stream of ideas
The difference between a machine and a human is that the machine will never get tired. We wanted to take unique advantage of that by offering a product that would endlessly generate idea after idea. The human picks the ideas they like, can pause and tweak what's being generated, but the default mode would be a "digital river of endlessly flowing ideas".
"Hey Siri, design a poster..."
Our line of thought eventually lit up another idea bulb: we could offer this product as a live assistant by integrating this service with Alexa and Siri.
Creative teams could put a HomePod, or Alexa device in the room, hook it up to our service, and then just have a conversation about their project ideas. The Alexa device would feed the audio to our service, which would interpret it, and when someone says "what if we advertised this product with a James Bond theme?" — boom — you have a mockup generated live on the screen. Person points at the screen "Yes! Exactly like that."
Then someone else chips in, "I think this looks too juvenile for our target market. Can you make this about 50% more serious?" Person takes a sip of water — boom — new mockup is generated where the typography is changed from a rounded font to a more serif-looking font, and the colours become more subdued. "Yesss! Now we're talking".
Testing the idea
I built a clickable prototype to get some fresh eyes and fresh perspectives on our product. Even if we can build it, do creatives want it? I interviewed some of my former colleagues, who are also designers, to get a sense for the kind of impression the product would make.
With the 20/20 perspective of hindsight, it's easy to conclude that we were overambitious. Not only was our product vision pushing beyond the possibilities of today's technology, but we didn't simplify the product enough. The value proposition was vague, and I to this day find it hard to describe the product. Even if we had built it, we would've needed to do a lot more work on our marketing and messaging.
Maybe in a parallel universe a few key pieces clicked together, and our venture took off. Machine learning can do some things amazingly well, but it is still far off the "creative AI" vision that we were hoping to build. Maybe in a few years, or a decade or two. Who knows.