You're probably right; spec conformance is a problem. What should be done about it? If the spec itself isn't all that bad, then we should try to improve implementations. Maybe tools can be developed to help detect implementation flaws. Or if the letter of the spec is the problem, but the protocol/framework itself is good, then why not rewrite an alternative spec, similar to what was done recently for HTTP/1.1?
The spec should be robust against the possibility of implementation errors that result in the system being less secure. This is distinct from the possibility of implementation errors that simply result in the code not working at all or being obviously broken. You can't prevent those, but you also don't need to. The point is that subtle implementation errors should be ruled out as much as possible.
This is a common criterion in cryptography, against which systems and primitives are judged.