
Let AI design your holiday cards!
Do you want to send your friends and co-workers a holiday card, but...
- You’re too busy to go out and buy some.
- You’re tired of giving generic cards.
- You don’t have the artistic skills to design your own.
- All of the above.
Don’t worry, we’re in the same boat. And while we may not have the artistic skills of Van Gogh or Picasso, we do know how to build and train an artificial intelligence system to do it for us!
Our ArtistAI asks you for a word or phrase to inspire them, then generates a unique artwork in the style of a famous artist, and instantly creates a beautiful holiday greeting card that you can e-mail or share.
For example, your boss might be a fan of Picasso:
Or your friend might prefer the bright colors of Vicente Manansala’s works:
ArtistAI has been decommissioned.
How we built this
There are three steps to transform the input text into an artwork: first, ArtistAI finds the nearest object based on your prompt using word embeddings; second it sketches the object using a variational autoencoder. Lastly, it copies the style of any artwork and applies it to the generated sketch via style transfer. So if you give the model an input text, it will pass through the following pipeline:
Figure: Machine Learning Pipeline for Christmas Card Generator
ArtistAI’s drawing system was trained on hand-drawn doodles submitted by real people who played Google’s game Quick, Draw! Millions of players were prompted to draw simple objects such as “book”, “flower”, or “alarm clock,” creating a huge dataset for ArtistAI to learn from.
Step 1: ArtistAI chooses the object nearest to your prompt
The first step in the artistic process was for ArtistAI to be inspired. It takes the prompt and chooses the Quick, Draw! class which is closest to the input. For example, when we type in “reading”, ArtistAI decides to draw “book”; if you input our company name “Thinking Machines” it would choose to draw an alarm clock.
Figure: Using GloVe to find the most similar Quick, Draw! class
To achieve this, we used a word embedding model called Global Vectors for Word Representation or GloVe (Pennington et al, 2014). After training a model on scraped Wikipedia articles, we then compared the similarity of the given text to each of our Quick, Draw! classes. The class with the highest score is then chosen by our AI artist.
Step 2: ArtistAI draws the object
Like many artists, ArtistAI starts by drawing sketches of the object. It generates drawings of the chosen object using a variational autoencoder (VAE) called SketchRNN (Ha et al, 2017). As a generative model, VAEs learns the distribution of a dataset — by showing it enough doodles of a “book”, it learns what a “book” looks like, and can draw its own.
Figure: AI-generated “book” doodles using a variational autoencoder
Step 3: ArtistAI mimics the style of a famous artist
For the final step, we trained ArtistAI to turn its doodles into paintings inspired by famous artworks. Using a technique called neural style transfer, the A.I. tries to understand the strokes and colors of a particular artwork, and transfers it to the doodles. Specifically, we used an arbitrary image stylization technique that allows us to train just one model for any style image (Ghiasi et al., 2017).
Check out our code!
We also open-sourced our implementation on Github, so that you can run the model in your own machines! We pre-selected some of our favorite artwork which have varied styles and color palettes, but you can train the model on any other image you like.
References
- Pennington, Jeffrey, Socher, Richard, et al. (2014). “Glove: Global Vectors for Word Representation”. In: Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pp. 1532-1543.
- Ha, David and Eck, Douglas (2017). “A Neural Representation of Sketch Drawings”. In: arXiv.:1704.03477.
- Ghiasi, Golnaz et al. (2017). “Exploring the structure of real-time, arbitrary neural artistic stylization network”. In arxiv:1705.06830
Images
Reference images of these historically significant artwork were used for informational and educational use only under the fair use policy.
- Ang, Kiukok. Fishermen. 1981. Oil on canvas. Private collection, n.p. © Ang Kiukok.
- Cabrera, Benedicto. Tres Marias. 2003. Acrylic on canvas. Private collection, n.p. © Benedicto Cabrera.
- Degas, Edgar. Dancer with a Bouquet of Flowers (Star of the Ballet). c. 1878. Pastel and gouache on paper. Getty Center, Los Angeles. (Public domain)
- Hokusai, Katsushika. The Great Wave off Kanagawa (English). 1830-1832? Color woodblock print. Metropolitan Museum of Art, New York City. (Public domain)
- Mondrian, Piet. Composition with Red Blue and Yellow. 1929. Oil and paper on canvas. National Museum, Belgrade. (Public domain)
- Monet, Claude. Impression, soleil levant. 1872. Musee Marmottan Monet, Paris. (Public domain)
- Picasso, Pablo. La muse. 1935. Oil on canvas. © Pablo Picasso.
- Picasso, Pablo. Figure dans un Fauteuil (Seated Nude, Femme nue assise). 1909-10. Oil on canvas. Tate Modern, London. (Public domain)
- Signac, Paul. Cassis, Cap Lombard, Opus 196. 1889. Oil on canvas. Gemeentemuseum Den Haag, The Hague. (Public domain)
- Seurat, Georges. A Sunday on La Grande Jatte. 1884. Oil on canvas. Art Institute of Chicago, Chicago City. (Public domain)
- van Gogh, Vincent. The Starry Night. 1889. Oil on canvas. Museum of Modern Art, New York City. (Public domain)
- Warhol, Andy. Marilyn Monroe. 1967. Silkscreen prints. © Andy Warhol Foundation for the Visual Arts / Artists Rights Society (ARS), New York.
- Manansala, Vicente. Pila sa bigas. 1979. Oil on canvas. Private collection, n.p. © Vicente Manansala.
- Munch, Edvard. The Scream (English). 1893. Oil, tempera, and pastel on cardboard. National Gallery of Norway, Oslo. (Public domain)