HttpClient: What Do You Want To Ph*ck Today? ™

This is a story about one of the most used components in Windows Runtime framework – and, from a historical perspective, also one of the most flawed collection of components. During previous months, I had to endure several really unpleasant encounters with this devil of a component, whether it was imperfect implementation or missing features… So, in this article, I’ll attempt to outline at least some of these.

Certificate pinning

I’ve been talking about this in one of my earlier posts, and it looks that it’s finally coming. With a bit of luck (and reflection), you might be even able to get it working in Windows 8.1 apps running on newer Windows 10 builds. Details here.

Maximum simultaneous connections

HttpClient allows modifications of the base filter, HttpBaseProtocolFilter, where you can specify some additional constraints, HTTP headers, or ignore certificate errors, for starters. It also allows modification of maximum number of connections to a server.

What’s the matter

It’s broken.

1. For starters, there’s no way to limit connections to just one. You’ll be rewarded with an out-of-range exception.
2. And if you specify any higher number, the result max number of connections will be MaxConnectionsPerServer+1! Not true, because there can be an additional connection due to a cross-domain redirect (e.g. from www.paulos.cz to paulos.cz). Corrected appropriately.

Example

using System;
using System.Threading.Tasks;
using Windows.Web.Http;
using Windows.Web.Http.Filters;

public async Task TestMaxConnectionsAsync()
{
    HttpBaseProtocolFilter filter = new HttpBaseProtocolFilter();
    //filter.MaxConnectionsPerServer = 1; // CRASHES
    filter.MaxConnectionsPerServer = 2;
    HttpClient client = new HttpClient(filter);

    // CORRECTION: this creates TWO different connections, so it's correct
    for (int i = 0; i < 5; i++)
    {
        await client.SendRequestAsync(
            CreateRequest());
    }
}

public HttpRequestMessage CreateRequest()
{
    return new HttpRequestMessage(
        HttpMethod.Get, new Uri("https://paulos.cz/test.php"));
}

Verdict:

Uploading with HttpStreamContent and IInputStream

Do you want to upload something? No problem, as long as you’re not using HttpClient in Windows Phone 8.1. Fortunately, you won’t need to deal with this issue anymore in Windows 10. But, if you’re stuck developing for Windows Phone 8.1, beware, as HttpClient is very picky about what it wishes to send using HttpStreamContent…

What’s the matter

In constructor of HttpStreamContent, you are expected to pass an instance of IInputStream. But it’s a trap, because what the component truly expects is IRandomAccessStream (which, by accident, implements also IInputStream)! If you try to pass something that only implements IInputStream, expect to get nonsense exceptions like The handle is invalid when trying to read a response.

using System;
using System.Diagnostics;
using System.Threading.Tasks;
using Windows.Security.Cryptography;
using Windows.Storage.Streams;
using Windows.Web.Http;

public async Task TestInputStreamQueryAsync()
{
    var stream = new InMemoryRandomAccessStream();
    await stream.WriteAsync(
        CryptographicBuffer.ConvertStringToBinary("Hi!", BinaryStringEncoding.Utf8));
    IInputStream inStream = stream.GetInputStreamAt(0);

    HttpClient client = new HttpClient();
    HttpRequestMessage request = new HttpRequestMessage(
        HttpMethod.Put, new Uri("https://paulos.cz/test.php"));
    request.Content = new HttpStreamContent(inStream);

    // !! BOOM !!
    HttpResponseMessage response = await client.SendRequestAsync(request); 
    string responseText = await response.Content.ReadAsStringAsync(); 
    Debug.WriteLine(responseText);
}

System.Exception: The handle is invalid. (Exception from HRESULT: 0x80070006 (E_HANDLE))

What makes this even worse

1. Sometimes using a pure IInputStream can ruin the result query to the server (in WP8.1). Of course, it depends on how lucky you are and how well your IInputStream implementation is written. Note that there is usually only one correct way. You’ll understand after trying some methods like CopyToAsync() :-)

2. If you convert a typical .NET System.IO.Stream to an IInputStream using extensions in System.IO.WindowsRuntimeStreamExtensions such as AsInputStream(), you won’t get any exception in HttpClient! Guess why. Behind the scenes, it will wrap the stream in a instance of a class that implements IRandomAccessStream. Because that interface also implements IInputStream, it can be used without further casts. Thus, AsInputStream() equals AsRandomAccessStream() for all practical purposes. But they might change it in the future, so I wouldn’t count on it.

So, I believe you can already see the result of this code:

using System.Diagnostics;
using System.IO;
using Windows.Storage.Streams;

public void TestMemoryStreamToIInputStream()
{
    MemoryStream memStream = new MemoryStream();
    IInputStream inStream = memStream.AsInputStream();
    IRandomAccessStream raStream = inStream as IRandomAccessStream;
    
    if(raStream != null && inStream == raStream)
    {
        Debug.WriteLine("inStream is IRandomAccessStream");
        
        if (raStream.CanRead)
            Debug.WriteLine("CanRead");
        if (raStream.CanWrite)
            Debug.WriteLine("CanWrite");
    }
}

inStream is IRandomAccessStream
CanRead
CanWrite

Verdict:

Keep-alive connections over HTTPS

HttpClient supports keep-alive connections, and it even supports doing more things in parallel! While the second statement definitely applies, the first is a bit of a lie. At least in Windows Phone 8.1.

What’s the matter

Keep-alive is working properly over plain HTTP. And there, it works quite well. But don’t you dare to try it with HTTPS! HttpClient will try to create a new connection for every request. When a max has been reached, it will kill the old connection and create a new one in place. Not only this creates an additional load on the servers, but also in the client, which has to initialize new TCP connection and perform a TLS handshake each and every single time (which isn’t cheap).

using System;
using System.Threading.Tasks;
using Windows.Web.Http;

public async Task TestHttpsKeepAliveAsync()
{
    HttpClient client = new HttpClient();
    
    // this creates five different connections
    for (int i = 0; i < 5; i++)
    {
        await client.SendRequestAsync(
            CreateRequest(true));
    }
    
    // this reuses one connection (not on my domain, though, I'm all HTTPS :)
    for (int i = 0; i < 5; i++)
    {
        await client.SendRequestAsync(
            CreateRequest(false));
    }
}

public HttpRequestMessage CreateRequest(bool https)
{
    return new HttpRequestMessage(
        HttpMethod.Get, 
        new Uri("http" + (https ? "s" : "") +
            "://paulos.cz/test.php"));
}

Verdict:

Conclusion

It might seem that HttpClient is a lousy component, though in reality it’s just a commander that, by default, sends the request through a HttpBaseProtocolFilter and expects a response in a form of HttpResponseMessage. Actually, you can write your own IHttpFilter to either extend, or completely reimplement the HTTP-requesting chain. But let’s face it: who’s going to do that? And who’s going to pay us for it? Plus, HttpBaseProtocolFilter is sealed and it’s not possible to only rewrite some parts.

Also, as I said before, this f*ckup is not a completely isolated incident in Microsoft’s web client history. For example, HttpWebRequest in Windows Phone 7 forced you to throw another exception after you got an exception to get the true status code of a request, if it was >= 400; otherwise you’d be only getting 404 Not Found. A variation of this also happened in Windows Phone 8 with HttpClient from Microsoft HTTP Client Libraries, where 404 Not Found was also the response for a broken connection or DNS resolving error.

And some of the issues I described (IInputStream and keepalive) have been already solved in Windows 10.

That’s a sad ending note right there!