System.Net.WebException: An exception occurred during a WebClient request

When using either WebClient or HttpWebRequest to download file from a remote location, sometimes I got the following exception:

System.Net.WebException: An exception occurred during a WebClient request

For some reasons, IIS (the Web server) may deny to serve a request that doesn’t specify the user-agent property in the request header. So, although it’s not really obvious, this can be solved pretty easily by specifying the particular user-agent property. You can set it to anything, it doesn’t matter as long as it exists.

For example, here how you provide the property using WebClient:

using (var wc = new WebClient())
{
	wc.Credentials = CredentialCache.DefaultCredentials;
	wc.Headers.Add(HttpRequestHeader.UserAgent, "anything");
	wc.DownloadFile(fileUrlToDownload, fileNameToSafe);
}

And, here how to do it with HttpWebRequest:

var req = (HttpWebRequest)HttpWebRequest.Create(fileUrlToDownload);
req.Credentials = CredentialCache.DefaultCredentials;
req.UserAgent = "anything";
 
var resp = (HttpWebResponse)req.GetResponse();
 
using (var stream = resp.GetResponseStream())
{
	using (var fstream = new FileStream(fileNameToSafe)
	{
		var buffer = new byte[8192];
		var maxCount = buffer.Length;
		int count;
		while ((count = stream.Read(buffer, 0, maxCount)) > 0)
			fstream.Write(buffer, 0, count);
	}
}
 
resp.Close();

You may also notice that downloading a file using WebClient is much simpler than using HttpWebRequest. The later, however, gives you much more control.

  • http://www.quickstem.com Jimbo

    Thanks a lot – the WebClient really eats [email protected]!

    HttpWebRequest still very simple to use but gives proper errors.

    Glad I came across this page 😛