Yes, it can but GPT-2-1.5b isn't too great at it. What really struck me looking at the examples is that the random GPT-3 poem samples are practically as good as my GPT-2-1.5b finetuned on hundreds of megabytes of poetry at considerable effort & expense: https://www.gwern.net/GPT-2 That's... both really awesome and dispiriting.