ASM.js builds can be quite large, even 10s of MB is not uncommon. Reducing the binary size isn't just "nice to have", it changes the viability of the platform.
I don't think anyone is arguing that binary size isn't relevant. It just has to be weighed against the other parameters we want to optimize, like implementation complexity.