Web API – problems and solutions

This is the second post in the Fall 2012 series on ASP.NET Web API.

It was most recently updated in September 2012, and is valid for a ASP.NET MVC 4 configuration.

In the first post, we introduced Web API with entry-level “hello world” functionality. In this post, we move beyond that to cover a more common scenario. Problems are introduced, and then solutions are offered.

An  example app has been posted; it includes the source code for the scenario described below. You can learn from it, and then modify it as problems arise.

.

Scenario

We need a web service that will model the typical and classic product-supplier relationship. If you are a retailer, you sell products. A supplier will make a number of products.

The web service will be open (for now), and will not be secured. That’s a problem for a later post.

.

Download the code examples

The scenario in this post has been captured in a “before” and “after” (Automapper) set of code examples. If you wish to follow along during the post, here they are:

Web API, multiple entities, before using Automapper

Web API, multiple entities, after using Automapper

.

Domain data model

The following is the problem domain’s data model. There are two entities:

.

Creating controllers

You have learned that it is easy to create controllers. For example, you can follow the previous “hello world” post procedure, or the Lab 1 procedure, to create a controller for the Supplier entity, a context, and a store initializer. (You should do that now to get some practice.) In your store initializer, create two suppliers, and two products for each supplier.

Next, create a controller for the Product entity.

.

What is “model binding”?

In your controller’s PUT and POST methods, there is an “if” statement block, which begins with:

if (ModelState.IsValid)

What is this? ASP.NET MVC “model binding”.

Model binding takes care of mundane mapping of properties between JSON (submitted in the body of a PUT or POST request) and an entity object. Notice that both the PUT and POST methods include a parameter for an entity object. The ASP.NET runtime will attempt to construct an entity object, and fill its properties with data submitted in the request.

The “if” statement is simply asking if the entity object that was constructed is valid.

.

Testing your work to this point

It is easy to test your work by executing GET requests in Fiddler. You will notice that the results do not include the associated objects. The reason is due to the property signature. For example, the Supplier class is written as follows:

public class Supplier
{
    public int Id { get; set; }
    public string Name { get; set; }
    public int Rating { get; set; }
    public ICollection Products { get; set; }
}

.

With that signature, the navigation property will return null. That is the default behaviour.

However, adding the “virtual” modifier to the navigation properties will enable the objects they refer to to be fetched. When will they be fetched? Well when the property is accessed (dereferenced). When will that happen? When the object graph is serialized. The (JSON or XML) serializer will access/fetch each object in the entire object graph as it builds the serialization text.

Change the navigation property by adding the “virtual” modifier.

    public virtual ICollection Products { get; set; }

.

Edit the Product class too. Add the “virtual” modifier to its navigation property. This enables an Entity Framework feature called “lazy loading”. More about that concept is found later in this post.

Build your project, and run the tests again in Fiddler.

Uh oh. The response status is 500 (internal server error). Its error text includes the following:

Self referencing loop detected for property ‘Supplier’ with type ‘System.Data.Entity.DynamicProxies… (etc.)

.

Why did that happen? Well while serializing a Supplier, it began to serialize the supplier’s products. Wile serializing the first Product, it encountered a Supplier property. Then boom! it failed.

Can we fix this? Well, yes. Configure the serializer to detect and gracefully handle “self-referencing loops”. Add this code to Global.asax.cs:

GlobalConfiguration.Configuration.Formatters
    .JsonFormatter.SerializerSettings
    .PreserveReferencesHandling =
    Newtonsoft.Json.PreserveReferencesHandling.All;

.

Run your tests again. This time the “get all suppliers” request returns the following (click to expand):

{
  "$id":"1",
  "$values":[
    {
      "$id":"2",
      "Products":{
        "$id":"3",
        "$values":[
          {
            "$id":"4",
            "Supplier":{
              "$ref":"2"
            },
            "Id":1,
            "Name":"Cheez Whiz",
            "MSRP":2.49,
            "Cost":1.25
          },
          {
            "$id":"5",
            "Supplier":{
              "$ref":"2"
            },
            "Id":2,
            "Name":"Kraft Dinner",
            "MSRP":1.99,
            "Cost":0.69
          },
          {
            "$id":"6",
            "Supplier":{
              "$ref":"2"
            },
            "Id":3,
            "Name":"Stove Top Stuffing Mix",
            "MSRP":2.39,
            "Cost":0.88
          }
        ]
      },
      "Id":1,
      "Name":"Kraft Foods",
      "Rating":8
    },
    {
      "$id":"7",
      "Products":{
        "$id":"8",
        "$values":[
          {
            "$id":"9",
            "Supplier":{
              "$ref":"7"
            },
            "Id":4,
            "Name":"Saltines",
            "MSRP":3.69,
            "Cost":2.2
          }
        ]
      },
      "Id":2,
      "Name":"Bland Inc.",
      "Rating":3
    }
  ]
}

.

Whoa. What’s with the $id, $values, and $ref?

Those are generated by the Json.NET serializer. They enable round-trip serialization-deserialization. However, they’re not suitable for use in an API that may be used by clients that do not use the Json.NET library.

There is another problem: How do you create the JSON to POST a new object or PUT an existing object? That task is a challenge, and filled with procedural rules.

Is there a solution? Yes. Use a “data transformation object” (its initialism is “DTO”).

Before continuing, remove (or comment out) the serializer configuation code in Global.asax.cs, and remove the “virtual” modifier from your entity classes.

.

More about lazy loading, and eager loading

You have just learned that the “virtual” modifier for a navigation property enables an Entity Framework feature called “lazy loading“. This feature fetches the object(s) on the other side of the navigation property only when requested.

However, as we’ve learned above, this feature does not blend well with a Web API. During serialization, the navigation property object(s) get fetched, which may mean that a circular reference loop is attempted, which leads to an exception. There is a workaround, but we’ve just ruled out its use.

The solution is to NOT use the”virtual” modifier. Then, use “eager loading“. How?

Use the DbSet extension method “Include()”. For example, using the data model in this example, you would fetch a Product collection or object. In the statement, add the Include() method. (See the code samples for typical usage.)

.

Find, FirstOrDefault, and SingleOrDefault methods

You know that a DbSet has a Find() method (which takes a simple “ID” argument, compared to a lambda expression).

The return type of the Include() method is NOT a DbSet. Therefore, we cannot use the Find() method.

We CAN use FirstOrDefault, or SingleOrDefault. What’s the difference?

FirstOrDefault – This method expects to fetch a collection of zero or more objects. Then, the first one in the sequence is returned. If there’s nothing in the collection, null is returned.

SingleOrDefault – This method expects to fetch exactly zero or one object. If zero or more than one, null is returned.

So, there may be a slight preference to using SingleOrDefault.

.

Data transformation objects (DTOs)

A data transformation object is based on a class definition. The easiest way to get started learning this concept is to create another source code class file  (in your Models folder) named “DTOs.cs”.

Copy your entity classes (from your Entities.cs source code file) to this new source code file. For each class, do the following steps:

  1. Change the type name; we suggest that you add “DTO” as a suffix (e.g. “ProductDTO”)
  2. If desired, remove some properties
  3. For to-many association collection properties, change the type to the DTO type
  4. For to-one association properties, change the type to “int” and edit the property name

.

In step 2, you can remove some properties if you wish. For example, in the Product class, there is a “Cost” property. You probably do not want that property to be delivered for public users. It would typically be for internal use only.

In our Product DTO class, let’s remove the “Cost” property.

This brings up an interesting question: Can we have more than one DTO for a class? Yes. There’s no limit to the number and purpose of DTOs. Continuing with the Product class discussion above, you may have one DTO that delivers data to public users, and it would not include the “Cost” property. However, when you want to add or modify a Product object (a task that would be done by internal users only), you would definitely include the “Cost” property. Therefore, it’s likely that you’ll have multiple DTO classes defined for an entity class. However, not in this example.

In step 3, change the type from the entity class type (e.g. Product) to the DTO type (e.g. ProductDTO).

The reason why we do step 4 is to enable us to simplify or flatten an object representation. For example, assume that you have a Product object. It has a Supplier property, and its type is Supplier. Therefore, the value of the Supplier property is an object graph for a Supplier. Probably more than you need.

Instead of an object graph, you can identify the Supplier property by its unique ID. So, we “flatten” the Supplier object, and use only the Supplier “Id” property. When you have the Supplier Id, you have all you need to perform operations on a Supplier.

After you edit your DTOs.cs source code file, it will have the following DTO classes:

public class ProductDTO
{
    public int Id { get; set; }
    public string Name { get; set; }
    public double MSRP { get; set; }
    public int SupplierId { get; set; }
}

public class SupplierDTO
{
    public int Id { get; set; }
    public string Name { get; set; }
    public ICollection<ProductDTO> Products { get; set; }
}

.

Update your controllers to use DTOs

You can now update your controllers to work with the new DTOs.

GET strategies

For the GET methods, change the return types to use the DTOs.

Next, we must replace all the code in the methods.

For example, assume you are working on the ProductsController, and with the “get all products” method.

Fetch all the products. Make sure that you include the navigation property, using the Include() extension method.

Next, make a generic collection (e.g. a List<T>) to hold the DTOs.

Then, go through the fetched Product collection. For each Product object, create a ProductDTO. Assign the relevant properties. Add the ProductDTO to the ProductDTO generic collection.

Finally, return the ProductDTO generic collection.

The following (which is in the downloadable example app) is the code:

public IEnumerable<ProductDTO> GetProducts()
{
    // Fetch all products
    var products = db.Products.Include(s => s.Supplier);

    // Create a ProductDTO collection
    List<ProductDTO> pdtos = new List<ProductDTO>();

    // Make DTOs from the entity objects
    foreach (var p in products)
    {
        ProductDTO pd = new ProductDTO();
        pd.Id = p.Id;
        pd.Name = p.Name;
        pd.MSRP = p.MSRP;
        pd.SupplierId = p.Supplier.Id;
        pdtos.Add(pd);
    }

    // Return the results
    return pdtos.AsEnumerable();
}

.

You can write the code for the “get one specific product” method, using a similar strategy. Obviously, you do not have to make a collection. Just fetch the specific product, and then create and return a single DTO.

.

POST strategies

Your POST strategies will differ, depending upon the kind of object that you need to create.

For example, when you are creating a new Supplier, you will simply provide data for its Name and Rating properties. In other words, you will not provide data for any products yet.

When you are ready to create a new Product, you will know which supplier is making/providing the product. Therefore, when you create a new Product, you will provide data for all of its properties (except, of course, for its Id property). When using a DTO, the data type of the Supplier-related data will be an int, for the “SupplierId” property.

When adding a new product, you must do one more thing: You must ensure that the SupplierId is valid. Your initial strategy (which may change later) will be to confirm that the related Supplier exists before committing to create a new Product object. You’ll see that in the code below.

// POST api/Products
// {"Name":"Tofu Brick","MSRP":4.99,"Cost":2.66,"SupplierId":2}
// {"Name":"Cottage Cheese","MSRP":2.39,"Cost":1.28,"SupplierId":2}
public HttpResponseMessage PostProduct(ProductDTOAll p)
{
    if (ModelState.IsValid)
    {
        // Attempt to locate the supplier first - if that fails, then throw an exception
        var supplier = db.Suppliers.Find(p.SupplierId);

        // Make sure that we can continue
        if (supplier == null)
        {
            throw new HttpResponseException(Request.CreateResponse(HttpStatusCode.BadRequest));
        }

        // Create a product
        Product product = new Product();
        product.Name = p.Name;
        product.MSRP = p.MSRP;
        product.Cost = p.Cost;
        product.Supplier = supplier;
        // Save it
        db.Products.Add(product);
        db.SaveChanges();

        // Update the product ID before returning it in the response
        p.Id = product.Id;

        // Create and return the response
        HttpResponseMessage response = Request.CreateResponse(HttpStatusCode.Created, p);
        response.Headers.Location = new Uri(Url.Link("DefaultApi", new { id = product.Id }));
        return response;
    }
    else
    {
        return Request.CreateResponse(HttpStatusCode.BadRequest);
    }
}

.

PUT strategy

The coding strategy to update a Supplier object (with data from a DTO) is fairly similar to the template-provided code. We create and configure a Supplier object (based on data from a DTO), and then use the following statement to attach it to the execution context:

db.Entry(supplier).State = EntityState.Modified;

To summarize the strategy for updating a Supplier object, it’s fairly simple.

When updating a Product object, we are going to add one more feature that will complicate matters – we will enable the user to change the SupplierId.

Warning – This may not be a real-world requirement. It is being presented to show you and teach you what you have to think about when working with navigation properties.

.

Therefore, in the code that follows, you will see that the Supplier object is queried, and if successful, assigned to the Product object’s Supplier property.

// PUT api/Products/5
public HttpResponseMessage PutProduct(int id, ProductDTOAll p)
{
    if (ModelState.IsValid && id == p.Id)
    {
        // Attempt to locate the product object (and include the supplier)
        Product product = db.Products.Include(s => s.Supplier)
            .SingleOrDefault(i => i.Id == id);

        // Make sure that we can continue
        if (product == null)
        {
            throw new HttpResponseException(Request.CreateResponse(HttpStatusCode.BadRequest));
        }

        // Look at the supplier property... if it has been changed,
        // we must verify that the new supplier exists
        Supplier supplier = (p.SupplierId == product.Supplier.Id) ?
            product.Supplier :
            db.Suppliers.Find(p.SupplierId);

        // Make sure that we can continue
        if (supplier == null)
        {
            throw new HttpResponseException(Request.CreateResponse(HttpStatusCode.BadRequest));
        }

        // Assign the properties, some of which may have been changed
        product.Name = p.Name;
        product.MSRP = p.MSRP;
        product.Cost = p.Cost;
        product.Supplier = supplier;

        // Assert your intention to change
        db.Entry(product).State = EntityState.Modified;

        // Attempt to make the changes
        try
        {
            db.SaveChanges();
        }
        catch (DbUpdateConcurrencyException)
        {
            return Request.CreateResponse(HttpStatusCode.NotFound);
        }

        return Request.CreateResponse(HttpStatusCode.OK);
    }
    else
    {
        return Request.CreateResponse(HttpStatusCode.BadRequest);
    }
}

.

DELETE strategy

The coding strategy for the DELETE method is adequately covered by the template-provided code. No changes are required.

After studying the code, and knowing about reference loops during serialization, you may question why it’s OK to return an entity object with the response. Well, at the point in time when the object is returned (and then serialized), it has been removed from the data store, and does not hold any references to other objects. So, it will serialize without error.

.

Checkpoint – working with DTOs

At this point in time, you have learned about the benefit of DTOs:

  • They address the reference loop exceptions during serialization
  • They enable the delivery of objects that have a custom “shape”
  • They enable the acceptance of easy-to-create data from clients
  • They hide the details of the actual domain data model from the public

You have also learned some issues with DTOs:

  • You must write lots of code to map between entity objects and DTOs
  • The problem increases exponentially with the number of entity classes in your domain data model

Is there a solution? We suggest that you use Automapper.

.

Introduction to Automapper

Automapper is a small library that virtually eliminates the writing of mapping code.

It was created by Jimmy Bogard in 2008, so it’s pretty mature. The Automapper.org web site will get you introduced to the library, and tell you how to add it to your project.

Automapper documentation is found on its wiki. The examples are terse, and are fine for getting started with simple data models. Your professor’s advice is that you start with a few simple scenarios to gain experience.

Automapper is best-suited to data delivery scenarios. Its design and operations work well with “flattening” and “projection” tasks. Navigation properties to other objects and/or collections are also handled well.

For some situations, it is also useful for working with DTOs that are accepted (from clients) in PUT and POST operations. However, in contrast to its suitability for data delivery scenarios, it has no auto-magic abilities to (for example) materialize an entity object from its unique ID/key.

Therefore, be careful how you think about Automapper, and how you use it.

.

Getting started with Automapper

The documentation covers this topic well. In summary:

  1. Add Automapper to your project
  2. Create “maps” in your Global.asax.cs class
  3. Perform mappings on demand, in your web service methods

.

Update your controllers to use Automapper

As noted earlier, Automapper is best-suited to data delivery scenarios.

Therefore, you will probably use it in most of your get-all and get-one situations.

You may also use it in your POST methods.

In contrast, you may not have to use it in your PUT methods. (Why? There’s a built-in way.)

Finally, you will not have to use it in your DELETE methods.

.

GET strategies

Study the code for the get-all products method. It’s now essentially two lines of code.

// GET api/Products
public IEnumerable<ProductDTO> GetProducts()
{
    // Fetch all products
    var products = db.Products.Include(s => s.Supplier);

    // Map the collection to a ProductDTO collection, and return it
    return Mapper.Map<IEnumerable<Product>, IEnumerable<ProductDTO>>(products);
}

.

The get-all suppliers method is similarly brief. For the get-one methods, the same.

.

POST strategies

The amount of code for POST methods is a bit less with Automapper. When adding a new Product object, we must confirm that the Supplier exists. That kind of code isn’t necessary when adding a new Supplier object.

.

PUT strategy

We noted above that you may not have to use Automapper in your PUT methods, because there’s a built-in way.

The (db) context has a “SetValues(object)” method, which has an argument that accepts an argument of any type, including a DTO. It will then map any readable (scalar or complex) property to the CurrentValues property. See the code example below.

Note that this method will not map navigation properties (to other objects or collections).

.

// PUT api/Products/5
public HttpResponseMessage PutProduct(int id, ProductDTOAll p)
{
    if (ModelState.IsValid && id == p.Id)
    {
        // Fetch the Product from the store, based on the ID
        Product product = db.Products.Include(s => s.Supplier)
            .SingleOrDefault(i => i.Id == id);

        // Make sure that we can continue
        if (product == null)
        {
            throw new HttpResponseException(Request.CreateResponse(HttpStatusCode.BadRequest));
        }

        // Update the properties...
        // This statement extracts the values from "p",
        // where property names (and types) match
        // Then, it assigns those values to the "product" current values
        // This statement works only for scalar and complex properties only
        // It does not affect navigation properties (to other objects or collections)
        // So, we'll have to do that after the SetValues() method runs
        db.Entry(product).CurrentValues.SetValues(p);

        // Look at the supplier property... if it has been changed,
        // we must verify that the new supplier exists
        Supplier supplier = (p.SupplierId == product.Supplier.Id) ?
            product.Supplier :
            db.Suppliers.Find(p.SupplierId);

        // Make sure that we can continue
        if (supplier == null)
        {
            throw new HttpResponseException(Request.CreateResponse(HttpStatusCode.BadRequest));
        }

        // Complete the assignment
        product.Supplier = supplier;

        // Attempt to make the changes
        try
        {
            db.SaveChanges();
        }
        catch (DbUpdateConcurrencyException)
        {
            return Request.CreateResponse(HttpStatusCode.NotFound);
        }

        return Request.CreateResponse(HttpStatusCode.OK);
    }
    else
    {
        return Request.CreateResponse(HttpStatusCode.BadRequest);
    }
}

.


.

.

.

  1. No comments yet.
  1. No trackbacks yet.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 47 other followers

%d bloggers like this: