Rick Strahl's Weblog  

Wind, waves, code and everything in between...
.NET • C# • Markdown • WPF • All Things Web
Contact   •   Articles   •   Products   •   Support   •   Advertise
Sponsored by:
West Wind WebSurge - Rest Client and Http Load Testing for Windows

.NET WebRequest.PreAuthenticate – not quite what it sounds like


:P
On this page:

I’ve run into the  problem a few times now: How to pre-authenticate .NET WebRequest calls doing an HTTP call to the server – essentially send authentication credentials on the very first request instead of waiting for a server challenge first? At first glance this sound like it should be easy: The .NET WebRequest object has a PreAuthenticate property which sounds like it should force authentication credentials to be sent on the first request. Looking at the MSDN example certainly looks like it does:

http://msdn.microsoft.com/en-us/library/system.net.webrequest.preauthenticate.aspx

Unfortunately the MSDN sample is wrong. As is the text of the Help topic which incorrectly leads you to believe that PreAuthenticate… wait for it - pre-authenticates. But it doesn’t allow you to set credentials that are sent on the first request.

What this property actually does is quite different. It doesn’t send credentials on the first request but rather caches the credentials ONCE you have already authenticated once. Http Authentication is based on a challenge response mechanism typically where the client sends a request and the server responds with a 401 header requesting authentication.

So the client sends a request like this:

GET /wconnect/admin/wc.wc?_maintain~ShowStatus HTTP/1.1
Host: rasnote
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.1.3) Gecko/20090824 Firefox/3.5.3 (.NET CLR 4.0.20506)
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en,de;q=0.7,en-us;q=0.3
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Connection: keep-alive

and the server responds with:

HTTP/1.1 401 Unauthorized
Cache-Control: private
Content-Type: text/html; charset=utf-8
Server: Microsoft-IIS/7.5
WWW-Authenticate: basic realm=rasnote"
X-AspNet-Version: 2.0.50727
WWW-Authenticate: Negotiate
WWW-Authenticate: NTLM
WWW-Authenticate: Basic realm="rasnote"
X-Powered-By: ASP.NET
Date: Tue, 27 Oct 2009 00:58:20 GMT
Content-Length: 5163

plus the actual error message body.

The client then is responsible for re-sending the current request with the authentication token information provided (in this case Basic Auth):

GET /wconnect/admin/wc.wc?_maintain~ShowStatus HTTP/1.1
Host: rasnote
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.1.3) Gecko/20090824 Firefox/3.5.3 (.NET CLR 4.0.20506)
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en,de;q=0.7,en-us;q=0.3
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Connection: keep-alive
Cookie: TimeTrakker=2HJ1998WH06696; WebLogCommentUser=Rick Strahl|http://www.west-wind.com/|rstrahl@west-wind.com; WebStoreUser=b8bd0ed9
Authorization: Basic cgsf12aDpkc2ZhZG1zMA==

Once the authorization info is sent the server responds with the actual page result.

Now if you use WebRequest (or WebClient) the default behavior is to re-authenticate on every request that requires authorization. This means if you look in  Fiddler or some other HTTP client Proxy that captures requests you’ll see that each request re-authenticates: Here are two requests fired back to back:

TwoRequests

and you can see the 401 challenge, the 200 response for both requests.

If you watch this same conversation between a browser and a server you’ll notice that the first 401 is also there but the subsequent 401 requests are not present.

WebRequest.PreAuthenticate

And this is precisely what the WebRequest.PreAuthenticate property does: It’s a caching mechanism that caches the connection credentials for a given domain in the active process and resends it on subsequent requests. It does not send credentials on the first request but it will cache credentials on subsequent requests after authentication has succeeded:

string url = "http://rasnote/wconnect/admin/wc.wc?_maintain~ShowStatus";
HttpWebRequest req = HttpWebRequest.Create(url) as HttpWebRequest;
req.PreAuthenticate = true;
req.Credentials = new NetworkCredential("rick", "secret", "rasnote");
req.AuthenticationLevel = System.Net.Security.AuthenticationLevel.MutualAuthRequested;
req.UserAgent = ": Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.1.3) Gecko/20090824 Firefox/3.5.3 (.NET CLR 4.0.20506)";
WebResponse resp = req.GetResponse();
resp.Close();

req = HttpWebRequest.Create(url) as HttpWebRequest;
req.PreAuthenticate = true;
req.Credentials = new NetworkCredential("rstrahl", "secret", "rasnote");
req.AuthenticationLevel = System.Net.Security.AuthenticationLevel.MutualAuthRequested;
req.UserAgent = ": Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.1.3) Gecko/20090824 Firefox/3.5.3 (.NET CLR 4.0.20506)";
resp = req.GetResponse();

which results in the desired sequence:

PreAuthenticated

where only the first request doesn’t send credentials.

This is quite useful as it saves quite a few round trips to the server – bascially it saves one auth request request for every authenticated request you make. In most scenarios I think you’d want to send these credentials this way but one downside to this is that there’s no way to log out the client. Since the client always sends the credentials once authenticated only an explicit operation ON THE SERVER can undo the credentials by forcing another login explicitly (ie. re-challenging with a forced 401 request).

Forcing Basic Authentication Credentials on the first Request

On a few occasions I’ve needed to send credentials on a first request – mainly to some oddball third party Web Services (why you’d want to use Basic Auth on a Web Service is beyond me – don’t ask but it’s not uncommon in my experience). This is true of certain services that are using Basic Authentication (especially some Apache based Web Services) and REQUIRE that the authentication is sent right from the first request. No challenge first. Ugly but there it is.

Now the following works only with Basic Authentication because it’s pretty straight forward to create the Basic Authorization ‘token’ in code since it’s just an unencrypted encoding of the user name and password into base64. As you might guess this is totally unsecure and should only be used when using HTTPS/SSL connections (i’m not in this example so I can capture the Fiddler trace and my local machine doesn’t have a cert installed, but for production apps ALWAYS use SSL with basic auth).

The idea is that you simply add the required Authorization header to the request on your own along with the authorization string that encodes the username and password:

string url = "http://rasnote/wconnect/admin/wc.wc?_maintain~ShowStatus";
HttpWebRequest req = HttpWebRequest.Create(url) as HttpWebRequest;

string user = "rick";
string pwd = "secret";
string domain = "www.west-wind.com";

string auth = "Basic " + Convert.ToBase64String(System.Text.Encoding.Default.GetBytes(user + ":" + pwd));
req.PreAuthenticate = true;
req.AuthenticationLevel = System.Net.Security.AuthenticationLevel.MutualAuthRequested;
req.Headers.Add("Authorization", auth);
req.UserAgent = ": Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.1.3) Gecko/20090824 Firefox/3.5.3 (.NET CLR 4.0.20506)"; WebResponse resp = req.GetResponse(); resp.Close();

This works and causes the request to immediately send auth information to the server. However, this only works with Basic Auth because you can actually create the authentication credentials easily on the client because it’s essentially clear text. The same doesn’t work for Windows or Digest authentication since you can’t easily create the authentication token on the client and send it to the server.

Another issue with this approach is that PreAuthenticate has no effect when you manually force the authentication. As far as Web Request is concerned it never sent the authentication information so it’s not actually caching the value any longer. If you run 3 requests in a row like this:

        string url = "http://rasnote/wconnect/admin/wc.wc?_maintain~ShowStatus";
        HttpWebRequest req = HttpWebRequest.Create(url) as HttpWebRequest;

        string user = "ricks";
        string pwd = "secret";
        string domain = "www.west-wind.com";

        string auth = "Basic " + Convert.ToBase64String(System.Text.Encoding.Default.GetBytes(user + ":" + pwd));
        req.PreAuthenticate = true;
        req.Headers.Add("Authorization", auth);
        req.UserAgent = ": Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.1.3) Gecko/20090824 Firefox/3.5.3 (.NET CLR 4.0.20506)";
        WebResponse resp = req.GetResponse();
        resp.Close();


        req = HttpWebRequest.Create(url) as HttpWebRequest;
        req.PreAuthenticate = true;
        req.Credentials = new NetworkCredential(user, pwd, domain);
        req.UserAgent = ": Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.1.3) Gecko/20090824 Firefox/3.5.3 (.NET CLR 4.0.20506)";
        resp = req.GetResponse();
        resp.Close();

        req = HttpWebRequest.Create(url) as HttpWebRequest;
        req.PreAuthenticate = true;
        req.Credentials = new NetworkCredential(user, pwd, domain);
        req.UserAgent = ": Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.1.3) Gecko/20090824 Firefox/3.5.3 (.NET CLR 4.0.20506)";
        resp = req.GetResponse();

you’ll find the trace looking like this:

Manually

where the first request (the one we explicitly add the header to) authenticates, the second challenges, and any subsequent ones then use the PreAuthenticate credential caching. In effect you’ll end up with one extra 401 request in this scenario, which is still better than 401 challenges on each request.

Getting Access to WebRequest in Classic .NET Web Service Clients

If you’re running a classic .NET Web Service client (non-WCF) one issue with the above is how do you get access to the WebRequest to actually add the custom headers to do the custom Authentication described above? One easy way is to implement a partial class that allows you add headers with something like this:

public partial class TaxService 
{ protected NameValueCollection Headers = new NameValueCollection(); public void AddHttpHeader(string key, string value) { this.Headers.Add(key,value); } public void ClearHttpHeaders() { this.Headers.Clear(); } protected override WebRequest GetWebRequest(Uri uri) { HttpWebRequest request = (HttpWebRequest) base.GetWebRequest(uri); request.Headers.Add(this.Headers); return request; }
}

where TaxService is the name of the .NET generated proxy class. In code you can then call AddHttpHeader() anywhere to add additional headers which are sent as part of the GetWebRequest override. Nice and simple once you know where to hook it.

For WCF there’s a bit more work involved by creating a message extension as described here: http://weblogs.asp.net/avnerk/archive/2006/04/26/Adding-custom-headers-to-every-WCF-call-_2D00_-a-solution.aspx.

FWIW, I think that HTTP header manipulation should be readily available on any HTTP based Web Service client DIRECTLY without having to subclass or implement a special interface hook. But alas a little extra work is required in .NET to make this happen

Not a Common Problem, but when it happens…

This has been one of those issues that is really rare, but it’s bitten me on several occasions when dealing with oddball Web services – a couple of times in my own work interacting with various Web Services and a few times on customer projects that required interaction with credentials-first services. Since the servers determine the protocol, we don’t have a choice but to follow the protocol. Lovely following standards that implementers decide to ignore, isn’t it? :-}

Posted in .NET  CSharp  Web Services  

The Voices of Reason


 

John Sheehan
February 18, 2010

# re: .NET WebRequest.PreAuthenticate – not quite what it sounds like

Good article. Nice overview of the problem. I ran into a similar issue with RestSharp which Daniel Crenna (TweetSharp) pointed out and helped resolve to make RS use this method by default with basic auth.

Matt
February 18, 2010

# re: .NET WebRequest.PreAuthenticate – not quite what it sounds like

Brilliant article. This is something that's caught me in the past - you explain it so well I've added a link back to this on my own blog for future reference. Thanks.

Josh Stodola
February 19, 2010

# re: .NET WebRequest.PreAuthenticate – not quite what it sounds like

Great detailed explanation, as always!

JW
March 18, 2010

# re: .NET WebRequest.PreAuthenticate – not quite what it sounds like

So glad I found this. I have this issue calling a JAX-WS on a weblogic server, not exactlly odd ball in my mind so am I doing something wrong on the ejb side?

Jon Hanna
July 18, 2010

# re: .NET WebRequest.PreAuthenticate – not quite what it sounds like

Ironically, this is the exact opposite of the problem with using XHR for ajax, json or similar uses of javascript in Safari. It does (or did, they may have fixed it since) send the Basic headers on the first attempt, with no means to override this behaviour.

This is perfect in the sort of case you are talking about (webserver uses Basic auth and [incorrectly] doesn't challenge if they aren't sent) but is disasterous in cases where a more secure authentication method like Digest or NTLM over an unecrypted connection (quite a good idea when the only sensitive details are the auth details, as encrypted connections have lots of downsides along with their obvious upside). In such a case the correct behaviour works fine, but the behaviour of Safari sends your password in the clear, before then authenticating correctly in response to the server's challenge.

Having found no better solution for a given application but to file a bug report against Safari and block it from the functionality in question (and thank my stars that it was a hobby project rather than anything critical), I'm very glad that WebRequest has the behaviour you complain of here.

The behaviour of WebRequest is correct, and you had to hack around the incorrect behaviour of a server. Doing the same thing when not needed by a malfunctioning server would introduce a security risk.

Kashif
July 28, 2010

# re: .NET WebRequest.PreAuthenticate – not quite what it sounds like

I am working on a CJ webservice , it's working fine in our development XP machine having IIS 6 installed. But same code return 401 UnAuthorized on production Server ( IIS 7.0). I don't understand the issue. This third party service is based on Authorization key only and working fine on development machine.
By the way Production server is shared so we have very limited option to diagnose the problem. Any feedback is much appreciated.

Oisin G.
February 24, 2012

# re: .NET WebRequest.PreAuthenticate – not quite what it sounds like

I've just found this and I find myself having to go the WCF route. I have a WCF service out of my control that's using MTOM streaming AND basic authentication. However, WCF clients won't allow basic authentication in this situation because it's a one-way post of data (which is an implementation detail of the WCF client.) WCF has hard checks to prevent you from enabling transport security in this case. My only choice seems to be to configure the proxy to use no authentication but inject basic auth headers myself. Is it so f***ed up that WCF as a service allows you to enable MTOM and streaming along with basic authentication, but no WCF clients can use it? Java clients don't have a problem as those poor guys have to inject basic auth headers most of the time themselves anyway. Sigh.

Phillip Lanza
September 19, 2012

# re: .NET WebRequest.PreAuthenticate – not quite what it sounds like

Thanks for the informative post. That's almost exactly what I needed to fix a problem that I was having with a legacy importer app that one of my company's clients uses. I changed the auth type from "basic" to "digest" and that fixed the 401 response error. Thanks again!

bex
August 14, 2013

# re: .NET WebRequest.PreAuthenticate – not quite what it sounds like

Great article! It helped me out when I was trying to get the WebLogic login page to stop showing. I did something slightly different, tho: I'm doing a 'PING' request first, and then my actual request. Same as a round-trip, but then I get a JSESSIONID in the response that I can cache in the cookie for future requests.

shabir
September 29, 2013

# re: .NET WebRequest.PreAuthenticate – not quite what it sounds like

Hi
Simply awesome.even i thought lots of time that what this WebRequest.PreAuthenticate is doing .It was very confusing? looking into your post i feel glad.I really appreciate the way you present things. You are simple awesome..

Regards
Shabir Hakim

Elad Nava
October 15, 2013

# re: .NET WebRequest.PreAuthenticate – not quite what it sounds like

Great article! I was looking for a solution to this issue for a long time!


The problem I was experiencing, is that a proxy that requires authentication returned an unexpected error in case I didn't supply the authentication header in the first request.

Bidyut
September 11, 2014

# re: .NET WebRequest.PreAuthenticate – not quite what it sounds like

Out of all the trash available on the WebRequest authentication topic, at various sites , this is the most complete example. It is clear to understand with coherent script flow. I found the following script block to be most useful

"string auth = "Basic " + Convert.ToBase64String(System.Text.Encoding.Default.GetBytes(user + ":" + pwd));
req.PreAuthenticate = true;
req.AuthenticationLevel = System.Net.Security.AuthenticationLevel.MutualAuthRequested;"

Billa
July 19, 2016

# re: .NET WebRequest.PreAuthenticate – not quite what it sounds like

Nice Article! Got more information on why there are two requests?

Is there a way to reset NTLM credentials and force to fall over? I tried but somehow its always sending the credentials. I am not sure is it default cached.

Explained the issue at http://stackoverflow.com/questions/38357792/set-network-credentials-via-config

Vineeth
October 28, 2016

# re: .NET WebRequest.PreAuthenticate – not quite what it sounds like

Hi,

Can you please tell me how to add custom headers, I am using Visual Studio 2008 and .Net 3.5 framework. This is for windows CE application. I am implementing a method which uses custom headers to consume data or post data to web service. In .Net 3.5 I cannot override GetWebRequest(Uri uri) method. Any responses will be very much appreciated.

-Vineeth

omrit
March 13, 2018

# re: .NET WebRequest.PreAuthenticate – not quite what it sounds like

Thank you for this article!!!! I encountered this behavior when login code that used to work no longer works with third party higher version . When i read your article it explained well the reason for the different behavior.


peter
August 18, 2018

# re: .NET WebRequest.PreAuthenticate – not quite what it sounds like

"why you'd want to use Basic Authentication on a web service is beyond me" Could you expand on this please. I am a bit new to all this, but I am writing an IIS hosted web service and had got the impression from my reading that basic auth was the norm. So what would you recommend instead?

Thanks for the article by the way


Rick Strahl
August 18, 2018

# re: .NET WebRequest.PreAuthenticate – not quite what it sounds like

@Peter - Basic authentication is a really old protocol and it is very insecure because it passes username and password in the 'auth token'. IOW, it can be easily compromised by anybody who either listens to an un encrypted connection (man in the middle attack), or even just gets onto your physical machine in any way. The password is there in plain text (well base64 encoded anyway).

If you're using IIS and .NET you can use Forms Authentication for programmatic processing, or if you want to use Windows accounts you can use Windows Authentication. Both use authentication tokens that are not directly associated with your account.


Markus
September 25, 2018

# re: .NET WebRequest.PreAuthenticate – not quite what it sounds like

Hi Rick,
thanks for your article! I had a quite strange problem that surfaced as an IOException stating "An established connection was aborted by the software in your host machine". After verifying that the problem lies on my server (not the other side), I found that it must have something to do with HttpWebRequest authentication:

  • The abort always occured on the second request to the same server (but different url).
  • Fiddler showed me that the request actually was sent and got a correct response, nevertheless .NET threw the error.
  • Fiddler also showed me that the erroneous request did not send the Authentication header although HttpWebRequest.PreAuthenticate was set and it was not the first request (no 401 challenge).

Your blog to the rescue: I manually created the Authentication header according to your example and disabled HttpWebRequest.PreAuthenticate as well as HttpWebRequest.Credentials. And the IOException was gone.


Kumar
June 21, 2022

# re: .NET WebRequest.PreAuthenticate – not quite what it sounds like

Is there a way to enable preemptive authentication at the server, OS, or app.config level? We have a .NET 2.0 C# application running on Server 2016 POSTing to a Linux box running the recipient's web service. But opening up our code to add the header is not an option. We see a 100->401->100->200 sequence for every transmission. I proposed leaving it the way it is for the marginal security benefit. But we need to be prepared if our customer wants to reduce server load & network traffic in the future. Thank you so much for allowing me to address this issue in an informed way with our customer's IT staff!


Rick Strahl
June 21, 2022

# re: .NET WebRequest.PreAuthenticate – not quite what it sounds like

@Kumar - no, this is a client specific thing. The server will do what it must to authenticate - Challenge on no auth header, and accept the header if provided.

The client is who has to decide how to initially send the data to the server.


West Wind  © Rick Strahl, West Wind Technologies, 2005 - 2024