I’ve been experimenting with generating Christmas carols using machine learning algorithms of various sizes. The smallest AIs, trained from scratch on a set of carols, tended to get confused [https://aiweirdness.com/post/168770625987/christmas-carols-generated-by-a-neural-network] about what exactly the carols are celebrating. GPT-2, larger and with some internet pretraining…