Do you want to send your friends and co-workers a holiday card, but...

1. You’re too busy to go out and buy some.
2. You’re tired of giving generic cards.
3. You don’t have the artistic skills to design your own.
4. All of the above.

Don’t worry, we’re in the same boat. And while we may not have the artistic skills of Van Gogh or Picasso, we do know how to build and train an artificial intelligence system to do it for us!

Our ArtistAI asks you for a word or phrase to inspire them, then generates a unique artwork in the style of a famous artist, and instantly creates a beautiful holiday greeting card that you can e-mail or share.

For example, your boss might be a fan of Picasso:

Or your friend might prefer the bright colors of Vicente Manansala’s works:

ArtistAI has been decommissioned.

How we built this

There are three steps to transform the input text into an artwork: first, ArtistAI finds the nearest object based on your prompt using word embeddings; second it sketches the object using a variational autoencoder. Lastly, it copies the style of any artwork and applies it to the generated sketch via style transfer. So if you give the model an input text, it will pass through the following pipeline:

ArtistAI’s drawing system was trained on hand-drawn doodles submitted by real people who played Google’s game Quick, Draw! Millions of players were prompted to draw simple objects such as “book”, “flower”, or “alarm clock,” creating a huge dataset for ArtistAI to learn from.

Step 1: ArtistAI chooses the object nearest to your prompt

The first step in the artistic process was for ArtistAI to be inspired. It takes the prompt and chooses the Quick, Draw! class which is closest to the input. For example, when we type in “reading”, ArtistAI decides to draw “book”; if you input our company name “Thinking Machines” it would choose to draw an alarm clock.

To achieve this, we used a word embedding model called Global Vectors for Word Representation or GloVe (Pennington et al, 2014). After training a model on scraped Wikipedia articles, we then compared the similarity of the given text to each of our Quick, Draw! classes. The class with the highest score is then chosen by our AI artist.

Step 2: ArtistAI draws the object

Like many artists, ArtistAI starts by drawing sketches of the object. It generates drawings of the chosen object using a variational autoencoder (VAE) called SketchRNN (Ha et al, 2017). As a generative model, VAEs learns the distribution of a dataset — by showing it enough doodles of a “book”, it learns what a “book” looks like, and can draw its own.

Step 3: ArtistAI mimics the style of a famous artist

For the final step, we trained ArtistAI to turn its doodles into paintings inspired by famous artworks. Using a technique called neural style transfer, the A.I. tries to understand the strokes and colors of a particular artwork, and transfers it to the doodles. Specifically, we used an arbitrary image stylization technique that allows us to train just one model for any style image (Ghiasi et al., 2017).

Check out our code!

We also open-sourced our implementation on Github, so that you can run the model in your own machines! We pre-selected some of our favorite artwork which have varied styles and color palettes, but you can train the model on any other image you like.