Rick Strahl's Weblog  

Wind, waves, code and everything in between...
.NET • C# • Markdown • WPF • All Things Web
Contact   •   Articles   •   Products   •   Support   •   Advertise
Sponsored by:
Markdown Monster - The Markdown Editor for Windows

Using an alternate JSON Serializer in ASP.NET Web API


:P
On this page:

The new ASP.NET Web API that Microsoft released alongside MVC 4.0 Beta last week is a great framework for building REST and AJAX APIs. I've been working with it for quite a while now and I really like the way it works and the complete set of features it provides 'in the box'. It's about time that Microsoft gets a decent API for building generic HTTP endpoints into the framework.

DataContractJsonSerializer sucks

As nice as Web API's overall design is one thing still sucks: The built-in JSON Serialization uses the DataContractJsonSerializer which is just too limiting for many scenarios. The biggest issues I have with it are:

  • No support for untyped values (object, dynamic, Anonymous Types)
  • MS AJAX style Date Formatting
  • Ugly serialization formats for types like Dictionaries

To me the most serious issue is dealing with serialization of untyped objects. I have number of applications with AJAX front ends that dynamically reformat data from business objects to fit a specific message format that certain UI components require. The most common scenario I have there are IEnumerable query results from a database with fields from the result set rearranged to fit the sometimes unconventional formats required for the UI components (like jqGrid for example). Creating custom types to fit these messages seems like overkill and projections using Linq makes this much easier to code up. Alas DataContractJsonSerializer doesn't support it. Neither does DataContractSerializer for XML output for that matter.

What this means is that you can't do stuff like this in Web API out of the box:

public object GetAnonymousType()
{
   return new { name = "Rick", company = "West Wind", entered= DateTime.Now };
}

Basically anything that doesn't have an explicit type DataContractJsonSerializer will not let you return. FWIW, the same is true for XmlSerializer which also doesn't work with non-typed values for serialization. The example above is obviously contrived with a hardcoded object graph, but it's not uncommon to get dynamic values returned from queries that have anonymous types for their result projections.

Apparently there's a good possibility that Microsoft will ship Json.NET as part of Web API RTM release.  Scott Hanselman confirmed this as a footnote in his JSON Dates post a few days ago. I've heard several other people from Microsoft confirm that Json.NET will be included and be the default JSON serializer, but no details yet in what capacity it will show up. Let's hope it ends up as the default in the box. Meanwhile this post will show you how you can use it today with the beta and get JSON that matches what you should see in the RTM version.

What about JsonValue?

To be fair Web API DOES include a new JsonValue/JsonObject/JsonArray type that allow you to address some of these scenarios. JsonValue is a new type in the System.Json assembly that can be used to build up an object graph based on a dictionary. It's actually a really cool implementation of a dynamic type that allows you to create an object graph and spit it out to JSON without having to create .NET type first. JsonValue can also receive a JSON string and parse it without having to actually load it into a .NET type (which is something that's been missing in the core framework). This is really useful if you get a JSON result from an arbitrary service and you don't want to explicitly create a mapping type for the data returned.

For serialization you can create an object structure on the fly and pass it back as part of an Web API action method like this:

public JsonValue GetJsonValue()
{
    dynamic json = new JsonObject();
    json.name = "Rick";
    json.company = "West Wind";
    json.entered = DateTime.Now;

    dynamic address = new JsonObject();
    address.street = "32 Kaiea";
    address.zip = "96779";
    json.address = address;

    dynamic phones = new JsonArray();
    json.phoneNumbers = phones;

    dynamic phone = new JsonObject();
    phone.type = "Home";
    phone.number = "808 123-1233";
    phones.Add(phone);

    phone = new JsonObject();
    phone.type = "Home";
    phone.number = "808 123-1233";
    phones.Add(phone);
    
   
    //var jsonString = json.ToString();

    return json;
}

which produces the following output (formatted here for easier reading):

{
    name: "rick",
    company: "West Wind",
    entered: "2012-03-08T15:33:19.673-10:00",
    address: 
    {
        street: "32 Kaiea",
        zip: "96779"
    },
    phoneNumbers: [
    {
        type: "Home",
        number: "808 123-1233"
    },
    {
        type: "Mobile",
        number: "808 123-1234"
    }]
}

If you need to build a simple JSON type on the fly these types work great. But if you have an existing type - or worse a query result/list that's already formatted JsonValue et al. become a pain to work with. As far as I can see there's no way to just throw an object instance at JsonValue and have it convert into JsonValue dictionary. It's a manual process.

Using alternate Serializers in Web API

So, currently the default serializer in WebAPI is DataContractJsonSeriaizer and I don't like it. You may not either, but luckily you can swap the serializer fairly easily. If you'd rather use the JavaScriptSerializer built into System.Web.Extensions or Json.NET today, it's not too difficult to create a custom MediaTypeFormatter that uses these serializers and can replace or partially replace the native serializer.

Here's a MediaTypeFormatter implementation using the ASP.NET JavaScriptSerializer:

using System;
using System.Net.Http.Formatting;
using System.Threading.Tasks;
using System.Web.Script.Serialization;
using System.Json;
using System.IO;

namespace Westwind.Web.WebApi
{
    public class JavaScriptSerializerFormatter : MediaTypeFormatter
    {
        public JavaScriptSerializerFormatter()
        {
            SupportedMediaTypes.Add(new System.Net.Http.Headers.MediaTypeHeaderValue("application/json"));
        }

        protected override bool CanWriteType(Type type)
        {
            // don't serialize JsonValue structure use default for that
            if (type == typeof(JsonValue) || type == typeof(JsonObject) || type== typeof(JsonArray) )
                return false;

            return true;
        }

        protected override bool CanReadType(Type type)
        {
            if (type == typeof(IKeyValueModel))                
                return false;
            
            return true;
        }

        protected override System.Threading.Tasks.Task<object> OnReadFromStreamAsync(Type type, System.IO.Stream stream, System.Net.Http.Headers.HttpContentHeaders contentHeaders, FormatterContext formatterContext)
        {
            var task = Task<object>.Factory.StartNew(() =>
                {                    
                    var ser = new JavaScriptSerializer();
                    string json;

                    using (var sr = new StreamReader(stream))
                    {                        
                        json = sr.ReadToEnd();
                        sr.Close();
                    }

                    object val = ser.Deserialize(json,type);
                    return val;
                });

            return task;
        }

        protected override System.Threading.Tasks.Task OnWriteToStreamAsync(Type type, object value, System.IO.Stream stream, System.Net.Http.Headers.HttpContentHeaders contentHeaders, FormatterContext formatterContext, System.Net.TransportContext transportContext)
        {            
            var task = Task.Factory.StartNew( () =>
                {

                    var ser = new JavaScriptSerializer();                    
                    var json = ser.Serialize(value);
                    
                    byte[] buf = System.Text.Encoding.Default.GetBytes(json);
                    stream.Write(buf,0,buf.Length);
                    stream.Flush();
                });

            return task;
        }
    }
}

Formatter implementation is pretty simple: You override 4 methods to tell which types you can handle and then handle the input or output streams to create/parse the JSON data.

Note that when creating output you want to take care to still allow JsonValue/JsonObject/JsonArray types to be handled by the default serializer so those objects serialize properly - if you let either JavaScriptSerializer or JSON.NET handle them they'd try to render the dictionaries which is very undesirable.

If you'd rather use Json.NET here's the JSON.NET version of the formatter:

// this code requires a reference to JSON.NET in your project
#if true

using System;
using System.Net.Http.Formatting;
using System.Threading.Tasks;
using System.Web.Script.Serialization;
using System.Json;
using Newtonsoft.Json;
using System.IO;
using Newtonsoft.Json.Converters;

namespace Westwind.Web.WebApi
{
    public class JsonNetFormatter : MediaTypeFormatter
    {
        public JsonNetFormatter()
        {
            SupportedMediaTypes.Add(new System.Net.Http.Headers.MediaTypeHeaderValue("application/json"));
        }

        protected override bool CanWriteType(Type type)
        {
            // don't serialize JsonValue structure use default for that
            if (type == typeof(JsonValue) || type == typeof(JsonObject) || type == typeof(JsonArray))
                return false;

            return true;
        }

        protected override bool CanReadType(Type type)
        {
            if (type == typeof(IKeyValueModel))
                return false;

            return true;
        }

        protected override System.Threading.Tasks.Task<object> OnReadFromStreamAsync(Type type, System.IO.Stream stream, System.Net.Http.Headers.HttpContentHeaders contentHeaders, FormatterContext formatterContext)
        {
            var task = Task<object>.Factory.StartNew(() =>
                {
                    var settings = new JsonSerializerSettings()
                    {
                        NullValueHandling = NullValueHandling.Ignore,
                    };
                   
                    var sr = new StreamReader(stream);
                    var jreader = new JsonTextReader(sr);

                    var ser = new JsonSerializer();
                    ser.Converters.Add(new IsoDateTimeConverter());
                     
                    object val = ser.Deserialize(jreader, type);
                    return val;
                });

            return task;
        }

        protected override System.Threading.Tasks.Task OnWriteToStreamAsync(Type type, object value, System.IO.Stream stream, System.Net.Http.Headers.HttpContentHeaders contentHeaders, FormatterContext formatterContext, System.Net.TransportContext transportContext)
        {            
            var task = Task.Factory.StartNew( () =>
                {                    
                    var settings = new JsonSerializerSettings()
                    {
                         NullValueHandling = NullValueHandling.Ignore,                                                  
                    };
                    
                    string json = JsonConvert.SerializeObject(value, Formatting.Indented, 
                                                              new JsonConverter[1] { new IsoDateTimeConverter() } );                    

                    byte[] buf = System.Text.Encoding.Default.GetBytes(json);
                    stream.Write(buf,0,buf.Length);
                    stream.Flush();
                });

            return task;
        }
    }
}
#endif

 

One advantage of the Json.NET serializer is that you can specify a few options on how things are formatted and handled. You get null value handling and you can plug in the IsoDateTimeConverter which is nice to product proper ISO dates that I would expect any Json serializer to output these days.

Hooking up the Formatters

Once you've created the custom formatters you need to enable them for your Web API application. To do this use the GlobalConfiguration.Configuration object and add the formatter to the Formatters collection. Here's what this looks like hooked up from Application_Start in a Web project:

protected void Application_Start(object sender, EventArgs e)
{

    // Action based routing (used for RPC calls)
    RouteTable.Routes.MapHttpRoute(
        name: "StockApi",
        routeTemplate: "stocks/{action}/{symbol}",
        defaults: new
        {
            symbol = RouteParameter.Optional,
            controller = "StockApi"
        }
    );
// WebApi Configuration to hook up formatters and message handlers // optional RegisterApis(GlobalConfiguration.Configuration); } public static void RegisterApis(HttpConfiguration config) { // Add JavaScriptSerializer formatter instead - add at top to make default //config.Formatters.Insert(0, new JavaScriptSerializerFormatter()); // Add Json.net formatter - add at the top so it fires first! // This leaves the old one in place so JsonValue/JsonObject/JsonArray still are handled config.Formatters.Insert(0, new JsonNetFormatter()); }

One thing to remember here is the GlobalConfiguration object which is Web API's static configuration instance. I think this thing is seriously misnamed given that GlobalConfiguration could stand for anything and so is hard to discover if you don't know what you're looking for. How about WebApiConfiguration or something more descriptive? Anyway, once you know what it is you can use the Formatters collection to insert your custom formatter.

Note that I insert my formatter at the top of the list so it takes precedence over the default formatter. I also am not removing the old formatter because I still want JsonValue/JsonObject/JsonArray to be handled by the default serialization mechanism. Since they process in sequence and I exclude processing for these types JsonValue et al. still get properly serialized/deserialized.

Summary

Currently DataContractJsonSerializer in Web API is a pain, but at least we have the ability with relatively limited effort to replace the MediaTypeFormatter and plug in our own JSON serializer. This is useful for many scenarios - if you have existing client applications that used MVC JsonResult or ASP.NET AJAX results from ASMX AJAX services you can plug in the JavaScript serializer and get exactly the same serializer you used in the past so your results will be the same and don't potentially break clients. JSON serializers do vary a bit in how they serialize some of the more complex types (like Dictionaries and dates for example) and so if you're migrating it might be helpful to ensure your client code doesn't break when you switch to ASP.NET Web API.

Going forward it looks like Microsoft is planning on plugging in Json.Net into Web API and make that the default. I think that's an awesome choice since Json.net has been around forever, is fast and easy to use and provides a ton of functionality as part of this great library. I just wish Microsoft would have figured this out sooner instead of now at the last minute integrating with it especially given that Json.Net has a similar set of lower level JSON objects JsonValue/JsonObject etc. which now will end up being duplicated by the native System.Json stuff. It's not like we don't already have enough confusion regarding which JSON serializer to use (JavaScriptSerializer, DataContractJsonSerializer, JsonValue/JsonObject/JsonArray and now Json.net). For years I've been using my own JSON serializer because the built in choices are both limited. However, with an official encorsement of Json.Net I'm happily moving on to use that in my applications.

Let's see and hope Microsoft gets this right before ASP.NET Web API goes gold.

Posted in Web Api  AJAX  ASP.NET  

The Voices of Reason


 

Harry M
March 09, 2012

# re: Using an alternate JSON Serializer in ASP.NET Web API

It will be pretty interesting if MS do include JSON.NET by default - it may have been around for ever, but as with all software regression bugs still appear from time to time e.g (http://json.codeplex.com/workitem/22195). How can MS handle officially sanctioning/supporting products outside of their control?

Nikolas Tziolis
March 09, 2012

# JSON.NET will be the default serializer in ASP.NET WebAPI

On his blog Scott Hanselmann confirmed that JSON.NET will be the default serializer and that MS will provide support for it as they do for other Open Source libraries they ship like jQUery, modernizer etc.

Also see the comments:
http://www.hanselman.com/blog/OnTheNightmareThatIsJSONDatesPlusJSONNETAndASPNETWebAPI.aspx

Andy
March 09, 2012

# re: Using an alternate JSON Serializer in ASP.NET Web API

I assuming it will be "will ship with JSON.NET" only in the sense of being part of the NuGet packages downloaded with Web API. And possibly extending this to the approach for jQuery, where I believe Microsoft provide support from PSS (Product Support Services).

I'm looking forward to easier camel casing of json properties with JSON.NET.

Tim
March 09, 2012

# re: Using an alternate JSON Serializer in ASP.NET Web API

Have you tried using the JsonSerializer contained within ServiceStack.Text found here https://github.com/ServiceStack/ServiceStack.Text. All the performance benchmarks I've found shown the serializers contained within ServiceStack.Text to be far more performant than the competition.
Here is one example of someone's findings http://daniel.wertheim.se/2011/02/07/json-net-vs-servicestack/.

Christian Weyer
March 13, 2012

# re: Using an alternate JSON Serializer in ASP.NET Web API

Hey man - here are also some JSON implementations of formatters over at https://github.com/thinktecture/Thinktecture.Web.Http

This is meant to be a library that provides convenience extensions from the community for the ASP.NET Web API framework (Beta 1) - includes custom media type formatters, message handlers, action filters and more.

Terrence Spencer
March 13, 2012

# re: Using an alternate JSON Serializer in ASP.NET Web API

Rick, maybe I got this wrong, but in this line:
What this means is that you can do stuff like this in Web API out of the box:

Didn't you mean.

What this means is that you can NOT do stuff like this in Web API out of the box:

Domain
March 13, 2012

# re: Using an alternate JSON Serializer in ASP.NET Web API

Awesome article as always. Just started to use web api a moth ago. What a great thing!

Mahesh
March 21, 2012

# re: Using an alternate JSON Serializer in ASP.NET Web API

I agree with Tim. ServiceStack serializer is the best choice

Marcus
March 27, 2012

# re: Using an alternate JSON Serializer in ASP.NET Web API

Thanks! Works fine for serialization. However on POST, trying to use upshot commitChanges(), I get "This DataController does not support operation 'Insert' for entity 'JObject'.". Full exception stacktrace:

{"ExceptionType":"System.InvalidOperationException","Message":"This DataController does not support operation 'Insert' for entity 'JObject'.","StackTrace":" at System.Web.Http.Data.DataController.ResolveActions(DataControllerDescription description, IEnumerable`1 changeSet)\r\n at System.Web.Http.Data.DataController.Submit(ChangeSet changeSet)\r\n at System.Web.Http.Data.SubmitActionDescriptor.Execute(HttpControllerContext controllerContext, IDictionary`2 arguments)\r\n at System.Web.Http.Controllers.ApiControllerActionInvoker.<>c__DisplayClass2.b__0()\r\n at System.Threading.Tasks.TaskHelpers.RunSynchronously[TResult](Func`1 func, CancellationToken cancellationToken)","InnerException":null}

The request payload was a simple entity:
[{"Id":"0","Operation":1,"Entity":{"__type":"MyEntity:#MyNamespace","Collection_Id":1,"Name":"MyTestThing"}}]

Rick Strahl
March 27, 2012

# re: Using an alternate JSON Serializer in ASP.NET Web API

@Marcus - what's your method input type look like? You're getting JObject which is Json.net but it's not a fixed type. If you are doing some sort of update that expects a strong type you need to specify that strong type on the inputs to the method.

Marcus
March 28, 2012

# re: Using an alternate JSON Serializer in ASP.NET Web API

Hi Rick, Here's my method:

public void InsertMyEntity(MyEntity entity)
{
InsertEntity(entity);
}

It previously worked before I switched serializers from the default into JSON.NET. Here are some other people having the same problem:

http://forums.asp.net/p/1779716/4877166.aspx/1?SPA+JSON+Datetime+madness

http://stackoverflow.com/questions/9473714/mvc-4-single-page-application-and-datetime/9681171#comment12003372_9474366

Rick Strahl
March 28, 2012

# re: Using an alternate JSON Serializer in ASP.NET Web API

@Marcus - looks like this is related to DataController/DbDataController. I've not seen any issues with plain ApiController, so it appears there's an issue with DataController where they use a separate parsing mechanism than the plain ApiControllers use.

Pretty sure that will get fixed (or might already be fixed in the latest repo on CodePlex) now that Json.net is the default parser.

mh
March 31, 2012

# re: Using an alternate JSON Serializer in ASP.NET Web API

Hi,

I am using DbDataController and receive the following error:

"This DataController does not support operation 'Insert' for entity 'JObject'."

I opened the System.Web.Http.Data.dll assembly in .NET Reflector. It seem that DataController is internally using DataContractJsonSerializer.

Noel Abrahams
April 02, 2012

# re: Using an alternate JSON Serializer in ASP.NET Web API

Hi, Rick,

It looks like the settings in the JsonNetFormatter are not being used:
 var settings = new JsonSerializerSettings() 


It's possible to add the settings in the call to Serialize object:

string json = JsonConvert.SerializeObject(value,
                                Formatting.Indented,
                                settings
                                );


But strangely there doesn't seem to be a way to do that in the call to Deserialize.

I think it's great that MS are looking to make JSON.net the default serializer but IMHO the API of this library needs a little tidying up.

Thanks.
Noel

Shrike
May 22, 2012

# re: Using an alternate JSON Serializer in ASP.NET Web API

In CanReadType we also should ignore JsonValue/JsonObject:
        protected override bool CanReadType(Type type)
        {
            if (type == typeof(JsonValue) || type == typeof(JsonObject) || type == typeof(JsonArray))
                return false;
            if (type == typeof(IKeyValueModel))
                return false;
 
            return true;
        }

Rick Strahl
May 22, 2012

# re: Using an alternate JSON Serializer in ASP.NET Web API

@Shrike - Actually all that changes with the default use of Json.net in later builds of Web API. I'll be updating this post once the RC ships...

There's an update that works with the current nightlies here:
https://github.com/RickStrahl/AspNetWebApiArticle/blob/master/AspNetWebApi/Code/WebApi/Formatters/JsonNetFormatter.cs

Amir
July 29, 2012

# re: Using an alternate JSON Serializer in ASP.NET Web API

if you need to pas back a .Net object as JsonValue, the way which has been mentioned above needs too much coding and not easy to maintain. I suggest this easy and simple way:


public JsonValue ConvertToJsonObject(object input)
{
var jsSerializer = new System.Web.Script.Serialization.JavaScriptSerializer();
var serializedJson = jsSerializer.Serialize(input);
return System.Json.JsonValue.Parse(serializedJson);
}

Rick Strahl
July 29, 2012

# re: Using an alternate JSON Serializer in ASP.NET Web API

@Amir - Uh, no... You'd have to call that on every request. The point of a formatter is that it's a global feature - you install it once and it works on all requests. You shouldn't have to muck with the low level semantics of serialization inside of a Web API method - that defeats the entire purpose.

Chris
August 04, 2012

# re: Using an alternate JSON Serializer in ASP.NET Web API

Hi Rick,

Very helpful. One change I did on OnWriteToStreamAsync to allow é, è, ä...

byte[] buf = System.Text.Encoding.Default.GetBytes(json);


with:

byte[] buf = System.Text.Encoding.UTF8.GetBytes(json);


Thanks,
Chris

Sam Matthews
August 18, 2012

# re: Using an alternate JSON Serializer in ASP.NET Web API

Hi, I'm trying to implement your System.Json formatter in a ASP Web API app. I have used System.Json extensively and now I find that they dumped it and are forcing you to use Newtonsoft (@#$@#). However when I try to install the formatter it's not able to find resolve IKeyValueModel, and FormatterContext. I have made no changes to the target net framework (4.0).. and I can't believe they are messing up the underlying support so much!! Any ideas on how I can install the formatter to support System.json?

Thanks,
Sam

Preeti
November 27, 2012

# re: Using an alternate JSON Serializer in ASP.NET Web API

Hello,
I have created an iphone app and i send a json object to a web api in mvc 4. How do i recieve the json object ion the api. Can i get any example? Where is the sent json retrieved? And also, i dont want xml response. i want json response in return. The web API is basically for login validation. Please help. Thank You.

Rick Strahl
November 27, 2012

# re: Using an alternate JSON Serializer in ASP.NET Web API

@Preeti - if you have a matching object in your object model that maps the JSON properties, you can use that. Alternately you can receive dynamic JSON in Web API by receiving: JObject as the receiving API Method parameter and you can then parse the object dynamically as described in this post:

http://www.west-wind.com/weblog/posts/2012/Aug/30/Using-JSONNET-for-dynamic-JSON-parsing

Because WebAPI now uses JSON.NET natively (ie. the JSON.NET example isn't necessary) you can access JOBJECT directly as a parameter signature:

public bool Result(JObject myIPhoneJsonData)
{
   ...
   return true;
}

Thomas Dupont
June 11, 2014

# re: Using an alternate JSON Serializer in ASP.NET Web API

Hi,

Thanks a lot for your article. It's simple and ingenius.

However, in class "JsonNetFormatter", methods "[On]ReadFromStreamAsync" and "[On]WriteToStreamAsync" create object "setting", which isn't used.

I suggest this supplement:
        public override Task<object> ReadFromStreamAsync(Type type, 
                                                            Stream stream, 
                                                            HttpContent contentHeaders, 
                                                            IFormatterLogger formatterLogger)
        {
            var task = Task<object>.Factory.StartNew(() =>
            {
 
                var sr = new StreamReader(stream);
                var jreader = new JsonTextReader(sr);
 
                var ser = new JsonSerializer();
                ser.Converters.Add(new IsoDateTimeConverter());
                ser.NullValueHandling = NullValueHandling.Ignore;
                ser.TypeNameHandling = TypeNameHandling.Auto;
 
                object val = ser.Deserialize(jreader, type);
                return val;
            });
 
            return task;
        }
 
        public override Task WriteToStreamAsync(Type type, 
                                                    object value, 
                                                    Stream stream, 
                                                    HttpContent contentHeaders, 
                                                    TransportContext transportContext)
        {
            var task = Task.Factory.StartNew(() =>
            {
                string json = JsonConvert.SerializeObject(value, Formatting.Indented,
                                                          _JsonSettings);
 
                byte[] buf = System.Text.Encoding.UTF8.GetBytes(json);
                stream.Write(buf, 0, buf.Length);
                stream.Flush();
            });
 
            return task;
        }


Thank,
Tom.

Dennis
March 25, 2015

# re: Using an alternate JSON Serializer in ASP.NET Web API

Rick,

Have you updated this article lately. Looks like ASP.NET Web API implements JSON.NET already or am I confused? Yeah I know it is two years old a lot has changed.

Rick Strahl
March 25, 2015

# re: Using an alternate JSON Serializer in ASP.NET Web API

This article refers to a pre-release version and yes JSON.NET is the default serializer now. However, this article still serves as guide for replacing the default serializer with something else.

Matt Watson
December 04, 2015

# re: Using an alternate JSON Serializer in ASP.NET Web API

Check out this blog post of JSON performance tips http://stackify.com/top-11-json-performance-usage-tips/

West Wind  © Rick Strahl, West Wind Technologies, 2005 - 2024