I have spent a stupid amount of time trying to come up with a somewhat more secure approach and wound up with `scurl` which requires a cypher, a checksum, a url that points to an immutable file, a target path, always writes the output to disk, and will not write to the target path if the checksum does not match [0]. For an extra level of paranoia add a check to ensure that the target directory exists and is only writeable by the user currently running the command. As far as I can tell there isn't a simpler way to do this that doesn't have some hidden pitfall, and I'm sure even this way has pitfalls, because I only just realized that I had not accounted for the race condition on the target path. There are variants of this which can stream, and there was even some work to try to integrate this functionality into curl itself [1], but they are 1) not portable 2) more complex, longer, and harder to audit and 3) don't protect you anyway. Why not? Because you can't know that a file doesn't match until you read the last byte, at which point all of those bytes will already have been piped into bash. Oops!
I have also spend a stupid amount of time trying to figure out a reasonable workflow that streamlines auditing and update of the immutable links used in this pattern [2]. The same issue is faced by anyone who pins their dependencies. Auditing dependencies is a huge bottleneck, to the point where it is often neglected entirely and/or has the effect of preventing updating dependencies altogether.
Immutability of the data behind the link is critical to prevent checksum failures due to legitimate changes to the underlying data, but also means that you can't take advantage of being able to point users to a single unchanging url. For one type of security the name has to change with the content. For usability and another type of security it is vital to be able to quickly push updates. If you have users with an immutable url pointing to a bad version the only thing you can do is take down that version, but you can only do that if you control the server. If you point to a raw github url that bad version will never go away. Thus it is critical to have tooling in place that makes it easier for users (aka other developers) to find and update those immutable urls in a way that minimized the audit load.
I have also spend a stupid amount of time trying to figure out a reasonable workflow that streamlines auditing and update of the immutable links used in this pattern [2]. The same issue is faced by anyone who pins their dependencies. Auditing dependencies is a huge bottleneck, to the point where it is often neglected entirely and/or has the effect of preventing updating dependencies altogether.
Immutability of the data behind the link is critical to prevent checksum failures due to legitimate changes to the underlying data, but also means that you can't take advantage of being able to point users to a single unchanging url. For one type of security the name has to change with the content. For usability and another type of security it is vital to be able to quickly push updates. If you have users with an immutable url pointing to a bad version the only thing you can do is take down that version, but you can only do that if you control the server. If you point to a raw github url that bad version will never go away. Thus it is critical to have tooling in place that makes it easier for users (aka other developers) to find and update those immutable urls in a way that minimized the audit load.
The sandbox wont save you this time!
0. https://github.com/tgbugs/orgstrap/blob/master/get-emacs.org...
1. https://github.com/curl/curl/issues/1399
2. https://github.com/tgbugs/orgstrap/blob/master/reval.org
3. https://blogs.sciencemag.org/pipeline/archives/2008/02/26/sa...