That answer exposes the deeper answer, too: "My job as a developer is to understand user needs so that our software can help them fulfill them, so if I don't understand why someone needs a feature I should dig in further before implementing it so I don't implement the wrong thing or the right thing in the wrong way." Not always possible given shedule constraints, though.

Admitedly, those rules don't seem to apply in this case, since if the OS allows you to do something that corrupts data, that's a problem no matter why the user wants to do it. If it can corrupt data, the OS shouldn't allow you to do it, end of story.

I will amend your statement to say that the OS itself should never corrupt your data, which I agree with entirely.

The post was not super clear about exactly what was happening (or it may be that my limited knowledge of Windows storage internals is keeping me from understanding it), but it sounds like the NTFS client requested a cache flush and then was issuing writes during the flush. I don't know what contract these operations have, but it may very well be the case that the user was violating the contract. If Microsoft responded with "don't do that", this may be the case.

But wait! Shouldn't Windows prevent the data from being corrupted? Or shouldn't NTFS fail the writes in this case? Possibly. And most likely inserting the checks to make this happen would increase write latency for every NTFS client, even the ones that don't behave in this way.

This reminds me of another scenario I encountered, with Veritas VxFS running on top of AIX. The user initiated a space reclamation, which was sending what you can think of as a delete to the storage array. And the user was also writing data to the device at the same time. Due to a race condition (which I can describe for you if you really care), the legitimate user data would sometimes be deleted.

Should VxFS have protected users against this case? Yes. Was VxFS violating the SCSI protocol? No. Was the storage array violating the SCSI protocol? No. (Has VxFS fixed this bug, almost three years after I discovered it? No comment.)

It's always a lot more complicated than it seems on the outside.

Calling a write while flushing the same data from another process (as the OP reported) or thread is perfectly legitimate set of operations. If these operations are not supposed to be running simultaneously, that has to be enforced by the OS kernel. The "right" way would be for the OS to serialize these operations internally. But even returning an error for the write might be an acceptable (though not nice) way to handle it. What's absolutely not acceptable is randomly corrupting data.

All I'm saying is that no real, sane OS is going to be capable of protecting itself against every possible misuse. Remember that these are developers coding to an API we're talking about, not end users.

What if you have an API that takes an out pointer. I pass in data that I own, then I free that pointer in another thread. If I'm in userspace, I can blow up with SIGSEGV. If I'm in the kernel, maybe now you've just scribbled all over somebody else's memory. Shame on you for corrupting data.

APIs can always be abused. All the API developer can do is try to protect against obvious forms of abuse. I guarantee you every operating system that supports simultaneous multiprocess execution has some series of APIs that, when called in parallel, will corrupt your data.

Only for a wide interpretation of 'misuse'. In the case you described, the kernel is still doing what it was asked. In the article, it wasn't. Situations where one command invalidates another should be very uncommon and very documented.

A real, sane OS should be resilient to any syscalls in any order without triggering internal bugs.

In the absence of documentation saying otherwise, I would expect the calls to be serialized, and one to happen entirely before the other.

Now if they each did two writes at about the same time, it's no longer the OS's problem if the threads interlace badly.

That's not a straw man. That's the exact topic under discussion in this thread.

"Let's ignore the bug described by the article for a moment and look into this."

And that my friend is the definition of a straw man :-)

If the topic you're discussing is that the clients of an OS API should not encounter a bug...uh, I mean, I don't think you'll find anyone to disagree with you. It's not like Microsoft engineers put that bug in there thinking it would be OK. They even acknowledged that it was a bug...it just wasn't high priority because the caller (presumably; again, I know nothing about these syscalls) is not following best practices. Someone's pet bug got deprioritized. This is not news.

The topic I thought we were discussing was whether it's the OS's responsibility to prevent callers from misusing their APIs in a way that causes stupid things to happen, even when the caller is doing something stupid.

I think the latter is a more interesting conversation, but I apologize if I interrupted your discussion of the former.

So it would seem.

"It's not like Microsoft engineers put that bug in there thinking it would be OK."

You say that now. But the reason my strident response was triggered was because of what you said in your original post that I replied to:

"I don't know what contract these operations have, but it may very well be the case that the user was violating the contract. If Microsoft responded with "don't do that", this may be the case."

That is quite absurd on it's face. I replied with how the contract was nothing more or less than the syscall API and there was no margin for negotiation on kernel's part when it comes to corrupting data when a user program calls those APIs in whatever order it pleases.

Subsequently, your arguments seem to have become more elaborate with a lot more caveats added. I think it has stopped being fruitful for me to respond at this point.

It was corrupting it with data from other files on the disk, which could have been sensitive.

Where this was particularly troubling was that an unprivileged user could then use this as an exploit/attack on the system to get it to leak pages of system files. This is where fun stuff like pass-the-hash begins.

(Going back and reading the post again, it seems that it's even worse than that, since a virtual environment hitting this bug could get data from the host. This opens an attack vector for a guest to bypass the hypervisor and compromise sensitive host data. I don't know if data from other VMs on the system could also be exposed in this way, but that would also be quite bad.)

Anyway, I think my rambling caused my point to get lost, because I wasn't trying to argue that Microsoft are justified in their "don't do that" comment. But no sane, realistic OS can prevent against every hare-brained thing a driver developer is going to do.