DPS907 notes – Tue Nov 22

Clients (requestors) interacting with a web service. Public APIs introduction.

 

Assignment 10 update

A small change: It is NOT necessary to post your projects to Azure services.

 

Test today

The weekly test will not be done at the beginning of the class timeslot, due mostly to a delay in getting these notes posted early enough. We will work through some of the content as a group, and have the test later in the timeslot.

 

Introduction to clients that use a web service

In this DPS907 and WSA500 academic course, we have been using Fiddler as a client, when interacting with our web services. You now have a good amount of experience with this, and therefore a better appreciation of the HTTP protocol.

In the near future, it is likely that you will want to, or need to, interact with a web service from a client (requestor) app. What will you need to know? Well, something about the client programming environment, and then you must ensure that your web service is ready for client interaction.

The kinds of clients that you can expect will likely include:

  • Mobile devices, running iOS or Android, or a cross-platform environment or kit
  • Browsers, using JavaScript to make requests
  • Server-based apps that make requests (this kind also includes traditional desktop/laptop native apps)

Each client has a way to create a request, send it, and handle the response. Below, we present some topics to consider as you are researching and designing your clients.

 

Mobile devices as clients

It could be argued that mobile devices – smartphones and tablets – would be useless without web services. Anything worth doing on a mobile device depends on a web service.

The user base for smartphones in 2016 has pretty well settled on devices that run iOS and Android. Yes, there are a few others, but let’s stick with the widely-known and used platforms.

For native apps on both platforms, the programming environment and software development kit (SDK) has an HTTP client class. The names and implementation details vary, but the programmer can create and configure an HTTP client object, enable it to send the request, and then handle the result. Maybe the result ends up in the user interface immediately, or maybe data gets persisted; it all depends on the app’s purpose, workflow, and functionality.

The HTTP client object, as you would expect, needs to be configured. Your experience with Fiddler will help to understand that it needs several pieces of data, including:

  • URL to the resource
  • HTTP request method
  • Request headers that indicate content type and other settings
  • POST and PUT requests will also need entity body data
  • When accessing secured resources, an access token is needed, typically presented as an authorization header

 

Android client

Full disclosure: Your professor does not do Android programming. Therefore, some of the information presented here may be inaccurate or outdated. If the information needs to be corrected, please notify your professor.

In Android, we use an “async task” object to manage the request-response lifecycle. The object can run in the background, and therefore not on the UI thread.

One of the task object’s components is an HttpURLConnection, which is the HTTP client object. It gets configured with the necessary settings (URL, HTTP method, data, etc.), and then executed. Its return value – whatever the web service responds with – is then materialized in a way that makes sense for the app at the time. For example, if the response is JSON, the programmer will materialize it as an object or collection. Alternatively, if the response is a media item (e.g. a photo), it can be materialized as an image/graphical object.

Some programmers use an add-on library or framework to help with HTTP client tasks. These libraries or frameworks can offer a number of benefits, including lowering the amount of repetitive code a programmer has to write for a busy app, and convenience methods and properties for common tasks (e.g. CRUD). One such library is Volley, linked below. A programmer can also write their own library / framework / wrapper, if they wish.

Please read/skim these learning resources:

Android Authority – How to use a web API from your Android app

Packet Zoom – Which Android HTTP library to use?

Android Developers – Transmitting Network Data Using Volley

 

iOS client

In iOS, we use a “task” object to manage the request-response lifecycle. The object can run in the background, and therefore not on the UI thread.

One of the task object’s components is an NSMutableURLRequest, which is the HTTP client object (when embedded in a “session” object). It gets configured with the necessary settings (URL, HTTP method, data, etc.), and then executed. Its return value – whatever the web service responds with – is then materialized in a way that makes sense for the app at the time. For example, if the response is JSON, the programmer will materialize it as an object or collection. Alternatively, if the response is a media item (e.g. a photo), it can be materialized as an image/graphical object.

Some programmers use an add-on library or framework to help with HTTP client tasks. These libraries or frameworks can offer a number of benefits, including lowering the amount of repetitive code a programmer has to write for a busy app, and convenience methods and properties for common tasks (e.g. CRUD). One such library is Alamofire, linked below. A programmer can also write their own library / framework / wrapper, if they wish.

Please read/skim these learning resources:

PeterMcIntyre.com – An introduction to accessing the network from an iOS app

That Thing In Swift – Write Your Own API Clients

RayWenderlich.com – NSURLSession Tutorial: Getting Started

RayWenderlich.com – Alamofire Tutorial: Getting Started

 

Browser JavaScript clients

In the late 1990s – still the early years of the web – a number of different people looked for a way to enable a browser to make an HTTP request, without having to do a full-page post and refresh. The goal was to improve interaction and response time, and therefore the user experience.

In 1998, a Microsoft team introduced what was to become the XMLHttpRequest object in the Internet Explorer browser. This was important, because it enabled JavaScript to make an HTTP request, and then do something with the response (update the UI etc.), smoothly and in the background. It took a few years, but by 2004 or so, all browsers implemented this functionality.

In the author’s view, the year 2004 also marked the beginning of the modern web. A number of factors and events, from many different people and companies, combined to raise the awareness of the potential of XMLHttpRequest. These included a critical mass of users of an amazing Outlook Web Access app (as part of Exchange Server 2003), then-innovative Google Suggest (autocomplete search results as you type), work being done on Google Maps, and Jesse James Garrett’s seminal article, “Ajax: A New Approach to Web Applications”.

The result is that modern web apps are now typically composed of JavaScript-rich clients in browsers, interacting with data-rich web services. Yes, server-based HTML and CSS is also part of the modern web app, delivering page structures (or templates) to browsers at scale. However, the trend is to use the power of the browser and its host, and reduce the reliance on a server to perform document assembly and code generation.

The rise of JavaScript as a legitimate and serious choice for programming led also to the development of a JavaScript-based server runtime solution. The goal was to use JavaScript in all parts of modern web apps and web services, and supplant established platforms and frameworks (like PHP and ASP.NET) at the server. The potential benefit was huge – the ability to use only one language, JavaScript, for all parts of the app. In 2009, Ryan Dahl introduced Node.js, probably the best-known of the genre, and considered by many to be the de facto standard.

A programmer who works only in JavaScript at the client and at the server is often called a full-stack JavaScript developer.

This approach is often called modern web development, or simply web development to full-stack developers.

 

Getting started 

To get started with this topic, we are going to point you to some excellent resources on the Mozilla Developer Network (MDN). Your professor will not rewrite or interpret any of that content, because it is good, and of good quality. Read/skim the following. You will have an opportunity to apply some of what you learn in Assignment 11.

The Ajax page on MDN is a good starting point.

Follow the links that appear to be of interest, but make sure that you click through to these:

  • Getting Started – Ajax basics etc.
  • Using the XMLHttpRequest API
  • Sending and Receiving Binary Data
  • Ajax: A New Approach to Web Applications

 

Do you need a JavaScript framework to do Ajax?

Do you need a JavaScript framework to do Ajax?

No.

Five years ago, the answer would have been “maybe”, “probably”, or “yes”, depending on who you asked. However, modern browsers, and the modern ECMAScript version, ES 6, are combining to make the new answer “no”, or “probably not”.

If you use a search engine and look for “jquery ajax”, you will see millions of results. For years, arguably the best way to do Ajax was with a library/framework like jQuery. However, using plain JavaScript is a viable alternative, and perhaps a better way to get started and learn modern web programming techniques (in the opinion of your professor).

Here are some viewpoints to consider:

A Guide to Vanilla Ajax Without jQuery

You Might Not Need jQuery

You Don’t Need jQuery!

Humourous: Vanilla-js.com

 

Server-based apps as clients

All students in this course have knowledge and experience building ASP.NET web apps and web services.

Recently, in this course, you learned how to create and configure multiple apps to share a common security environment. Clients – Fiddler in our case – could make requests from any of the apps, IA Server, A9Music, and A9Other. So, the precedent has been set – multiple apps that have something in common.

 

Web app, sending requests to a web service

Is it possible for a web app, which serves browser clients, to send requests to a web service? In other words, maybe the web app does not have its own data store, and relies on the web service to be its data store.

Yes, it is possible. And, as you’ll learn now, and in more detail in a few weeks, it is a software architecture that is growing in popularity and use.

As you learned above, Android and iOS native apps have an HTTP client object. And, JavaScript also has one. Well, ASP.NET apps – web apps or web services – can also use an HTTP client.

The .NET Framework has a class named HttpClient. Not surprisingly, it’s used in a manner similar to what you’ve learned above: Create an instance, configure it, execute it (i.e. send the request), and then handle the response.

 

How it works

To get started with the solution, read the following notes from the professor’s web apps course notes:

BTI420 notes – Friday April 8 2016

The concepts in these notes are implemented in these code examples, in that course’s GitHub code example repository, in its Week 12 folder.

  • Web Service Example
  • Load Data From Web Service

 

The key and important principle in the Load Data… app is the design and use of a “factory” that will create web service request objects. These are convenience objects, intended to make the programmer’s life easier.

 

Ensuring that our web service projects are ready for client use

All the work that you have done in the course prepares your apps to be used by many different kinds of clients. In this section, we present some familiar and new considerations that help ensure that our web services are ready for use.

 

Hypermedia representations

If your app – or a part of it – will be public-facing, for general use, then it must deliver hypermedia representations.

Included in this requirement is the presence of an entry point (i.e. the RootController), to help the user get started with your web service. And, an HTTP OPTIONS handler.

 

CORS

If you expect that browser clients (JavaScript) will be using your app, then we must add and configure this feature.

CORS is an acronym for cross-origin resource sharing. The Mozilla Developer Network has an excellent article here:

HTTP access control (CORS)

In summary, browsers will not permit JavaScript-initiated requests to resources outside the origin (host/domain) that hosts the JavaScript code. It’s a safety and security issue.

However, the CORS standard enables a server to include headers in the response that can effectively override that behaviour, by specifying the origins that are allowed to make requests to it.

In our web service app, we must add and configure CORS. How?

Using the NuGet Package Manager, add this package:

Microsoft.Owin.Cors

Next, we must add a statement that adds CORS handling to the request-processing pipeline. Let’s start with the easy-to-implement situation, where the web service will allow requests from anywhere on the web. (In the future, in your own app, you can easily learn how to modify that behaviour, if it better-matches your app’s usage profile.)

If the app’s security configuration includes credential validation, add a “UseCors” statement to the App_Start > Startup.Auth.cs source code file, as shown below:

public void ConfigureAuth(IAppBuilder app)
{
  // Configure the db context and user manager to use a single instance per request
  app.CreatePerOwinContext(ApplicationDbContext.Create);
  app.CreatePerOwinContext<ApplicationUserManager>(ApplicationUserManager.Create);

  // Configure support for CORS
  app.UseCors(Microsoft.Owin.Cors.CorsOptions.AllowAll);

  // etc.

 

If the app’s security configuration relies on an external credential validator (e.g. IA Server), add a “UseCors” statement to the Startup.cs source code file, as shown below:

public class Startup
{
  public void Configuration(IAppBuilder app)
  {
    // This app does NOT create (issue) bearer tokens, but understands how to read (decrypt) and use them
    app.UseOAuthBearerAuthentication(new Microsoft.Owin.Security.OAuth.OAuthBearerAuthenticationOptions());

    // Configure support for CORS
    app.UseCors(Microsoft.Owin.Cors.CorsOptions.AllowAll);
  } 
} 

 

Is there more to learn about CORS in our projects?

Yes. The technique above allows all requestors, and all HTTP methods. In a real-world app, we would need more control. Read this article for a complete treatment of CORS in an ASP.NET Web API app:

Mike Wasson, Microsoft: Enabling Cross-Origin Requests in ASP.NET Web API 2

 

Using a public web API

In class, you have heard that web services are the power behind today’s mobile and connected environment. There are many well-known, widely-used, and world-scale apps that depend upon web services. Without web services, they simply would not exist.

Now that we’re nearing the end of this academic course, you will find that you have learned enough to create an entry-level web service that could serve the needs of an organization (internal-facing), as well as the needs of users on the web, as a public web API. Therefore, when you’re designing a solution that needs a web service, you can write your own, or use a public web API.

Public web APIs are available from organizations large and small, by technology- and web-focused organizations as well as others outside that world, and increasingly by local, regional, and national governments and their agencies and services.

Using a public web API, as a client, will require you to perform a varying set of activities, most of which have been introduced in this academic course. Below, we briefly discuss some of them.

Learn about… Its resources: An obvious first task is to learn about the resources offered by an web API. For example, assume that you want to use an image processing service. What does it do? What kinds of interactions (requests, data transfer) does it support?

Overall design: Next, learn about its overall design. In this academic course, you have learned a specific design and usage style, but it is not the only valid design and style. Learn whether it supports hypermedia representations.

Documentation: In this academic course, you learned a design-and-coding approach which attempts to fulfill a key “RESTful” principle, which is to use hypermedia representations as a way for the client (programmer) to drive/affect application state. A well-done public web API may indeed provide enough info in its responses, but often you will need documentation. When you go through the documentation, don’t assume that you’ve seen it/everything before. APIs can be and are different.

Security considerations: Most public web APIs are secured. To use the web API, you must create an account, and then follow the web API’s rules for usage (authentication, access token presentation, etc.). Some public web APIs name the account a “developer account”, while others name it a “client app account”. While the names vary, the principle is the same: The public web API needs a way to identify the app that is sending the requests.

Taking this a step further, the web API may also require individual human users to create an account. Then, you must learn how to authenticate, and the kind of security standard (e.g. OAuth) it follows.

Hypermedia representations: Most modern public web APIs use JSON as the data format. However, many will use a specially-named hypermedia representation. You may be required to use specific values for Accept and Content-Type headers, and conform to its object design guidelines. One of the important first questions you will have is “how do I access the requested object or collection? (where is it located in the response package of data?)”

 

In summary, there’s no easy path to using a public web API. However, with your accumulated experience, and careful planning and investigation, you can confidently include these web APIs as a part of your overall app solution.

 

 

 

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

%d bloggers like this: