Currently it doesn't work, but saying it'll never work is pretty strong.

This kind of demo shows that deep neural networks can capture the structure of language, if not the semantics, in a very general way. And we have separate evidence that they can (in principle) capture semantic meaning and algorithmic reasoning as well, for example: http://arxiv.org/pdf/1410.5401v2.pdf (the "neural Turing machines" paper from DeepMind)

This is better, but you get pretty far with just markov chains with probabilities for letters actually.

Show me markov chain implementation that can write code letter by letter and I'll give you a car.

(And I mean plain markov chain, not something with additional logic that understands code structure)

comment by samizdatum shows pretty well how well markov chains work without some tweaking.

Feed it all of github, and I'm sure you could come up with some interesting auto complete code generation tools. Of course, coming from github , it'll be poorly documented and filled with buffer overflows :D

I'll agree that this is interesting, but it seems like a lot of people in this thread miss the point: we're working with multi-layer tools now. This enables modeling of multi-layer processes. The code generation as it stands is a obviously a toy, but what happens if we actually think about the real processing layers?

Take this example of code processing, and instead front it with a parser that generates an AST. For now, an actual parser for a single language. Maybe later, a network trained to be a parser. The AST is then fed to our network. What could we get out of the AST network? Could we get interesting static analysis out of it? Tell us the time and/or space complexity? Perhaps we discover that we need other layers to perform certain tasks.

This, of course, has parallels in language processing. Humans don't just go in a single (neural) step from excitation of inner ear cells ("sound") directly to "meaning". Cog sci and linguistics work has broken out a number of discrete functions of language processing. Some have been derived via experiment, some observed via individuals with brain lesions, others worked out by studies of children and adult language learners. These "layers" provide their own information and inspiration for building deep learning systems.

But how will you find that needle in the haystack that works. This is effectively producing random code samples that look syntactically correct.

There is no need for to produce readable code, it makes it easy for humans, but computers have no problems with generating and subsequently understanding unreadable assembly.