Most of what I know from Philippine folklore came from stories that were passed down from one generation to the next. I knew that a kapre is a large creature smoking a cigar because the “auntie of my mother’s friend” said so. Words turn into images, bringing these creatures into life.

Words turn into images, bringing these creatures into life.


Figure: Comics like Trese also captured our imagination on Philippine folklore
(image from Trese / Visual Print Enterprises)

I wonder what will happen if we let a computer do the same. If we provide a machine learning model with text descriptions of folk creatures, what images can it conjure?

To do so, I used a neural network called VQGAN+CLIP (Esser et al, 2021 and Radford et al, 2021) and supplied it with descriptions of Philippine folk creatures. The resulting images can then be thought of as to what the model “imagined” upon reading them —and they’re a bit surreal and creepy!

If you’re interested to learn how VQGAN works, then head to my explainer blogpost.
It’s not magic, it’s math!

Monsters from the machine

Below you’ll see model-generated creatures from text. The descriptions were lifted from The Aswang Project and Wikipedia. On the left you’ll see a clip of the model while it’s generating the image, and on the right you’ll see the final output. Also I kinda suck at CSS so this layout works best on desktop!

1. Nunò sa punso

Generally described as old men with flowing beards. They stand as tall as a three-year old child. In some stories, they wear a salakot, a traditional wide-brimmed hat made of rattan or reeds. They usually reside in an anthill or termite mound, hence the name.

2. Tiktik

It looks like a large, man-sized vulture or raven. It has an unusual ability to fly low and slow, without the winds visibly flapping. Its defining feature is its highly prehensile tongue/proboscis that reportedly has the looks and dexterity of a human hand.

3. Kapre

A tree giant, phenomenally tall, long-legged, god type of hairy humanoid that sits in big trees and smokes cigarattes. It is often seen waiting for people as they walk through a path. If you’re stuck in a place and you keep going around in circles, you’re said to be played around by a kapre.

4. Tikbalang

A half-human, half-horse creature in the mold of the Greek Pan. It has unusually long, powerful legs that resemble the rear quarter of a horse, hooves and all. It does not hurt or kill people. At worst, it leads them astray and plays mischievous tricks on their senses.

5. Manananggal

An old mythical creature in the Philippines that separates from their lower part of body and their fangs and wings give it a vampire-like appearance. It is said to favor preying on sleeping, pregnant women, using an elongated proboscis-like tongue to suck the hearts of fetuses, or the blood of someone who is sleeping.

6. Mantiw

Thirty foot, docile spirits in Iloilo that are often spotted in fields while leaning against coconut and buri palm trees. They have a fair complexion, broad shoulders, and hooked noses. They are reserved and peaceful, but are easily offended when a human attempts to whistle along them.

7. Tiyanak

A vampiric creature that takes on the form of a toddler or baby. Although there are various types, it typically takes the form of a newborn baby and cries in the jungle to attract unwary travelers. Once it is picked up by an unfortunate passerby, it reverts to its true form and attacks the victim.

8. Sigbin

It resembles a large dog or a hornless goat, but has large ears which it can clap like a pair of hands and a long, flexible tail that can be used as a whip. Families known as Sigbinan are said to keep the Sigbin in jars made of clay.

9. Santelmo

A shortened form of St Elmo’s Fire, it appears as a ball of fire and has served as an omen of heavenly intervention to sailors. It is found in fields and swamps and may have been caused by a spirit of a man who died near that area during heavy rain.

10. Bakunawa

A serpent-like dragon that is believed to be the cause of eclipses, earthquakes, rains, and wind. In mythology, it is said that the Bakunawa would rise from the ocean and swallow the moon, causing an eclipse.

Postscript

As for the output, the surreal imagery added a nice touch— it gave our mythical creatures an unearthly and bizarre vibe. I find the nunò and kapre surprising. They both capture what I imagine when I was young.

Implementation details

I didn’t write any code nor train any model to generate these images. Instead, I used the Google Colab Notebook provided by @advadnoun and Katherine Crowson (@RiversHaveWings)— all credits go to them. I simply provided it with the text prompt, and have the model generate the images for me. Cool!

However, I did a few tweaks in the prompt to influence the results. For example, I added tags like artstation, blender or hdr so that it shows more artistic renders rather than watercolored images. Lastly, I also limited the number of iterations to 1500, I noticed that most images converge around this point. When using a GPU, it takes around ten minutes to generate a decent image.

You can definitely create scary monsters by your own, the Colab Notebook provides all the resources you need. I also compiled a list of VQGAN+CLIP implementations, so be sure to check it out! For more inspiration, check out the @images_ai handle, this Youtube channel, or just search for “vqgan clip” on Twitter!

Did you uncover scarier monsters from the machine? Share them in the comments below!

References

  1. Esser, P., Rombach, R. and Ommer, B., 2021. Taming transformers for high-resolution image synthesis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 12873-12883).
  2. Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J. and Krueger, G., 2021. Learning transferable visual models from natural language supervision. arXiv preprint arXiv:2103.00020.