Possible file download bug?

[Alexey Neyman]

On 09/28/2018 06:11 AM, Paul Smith wrote:
> Crossgcc,
>
>
> I wanted to alert you to a possible bug.  I'm using someone else's product that uses an old copy of Crossgcc and I've found the following issue, and from looking at the latest Crossgcc code I suspect the same bug still exists.
>
>
> The issue is in scripts/functions in the code that downloads a file from the web using wget or curl.  My local ISP 'catches' page load errors and returns their own generated HTML error page and the bug I'm seeing results from a file download believing it succeeded when actually it downloaded just a dummy HTML page.
>
>
> In my case the files were always supposed to be variants on Linux tar files so it easy to use the Linux 'file' command to see if the file was actually an HTML page.  I don't know whether you can do the same or whether the files you are downloaded are more diverse and need more careful checking, perhaps outside of the file download function.
>
>
> However I wanted to alert you to this odd behaviour as it soaked up a few hours this morning identifying the cause and a fix.
It has been an issue with some download servers, too (I think, the one 
hosting libelf is an example; SourceForge during their outages is 
another one): instead of an error response, they return a valid 200 code 
with an HTML page. Current crosstool-NG (on master) offers an ability to 
verify the digest of the download (MD5/SHA-1/SHA-256/SHA-512); such 
broken download would fail this verification - so crosstool-NG won't 
save it to the local cache and will bail out with an error. But, it is 
only on master, no released versions do such verification.

Regards,
Alexey.