BTI420 notes – Tue Mar 17

More fun topics to help you learn more about web app programming.

.

Announcement: Your professor will not be on campus today, Tuesday March 17. 

Study these notes to replace the classroom session. 

.

Topic coverage today

Internet media types

Data import – load data from a file

.

Working with internet media types

Internet media types includes documents, images, audio, and video. Let’s introduce this topic by working with images (photos, drawings, .jpg, .png, etc.).

These images will be stored in the database-hosted data store.

The C# data type of an image is a byte array (i.e. Byte[]).

On the web, we must know the Internet media type of the image. Therefore, when coding your design model class, you MUST configure two properties – one for the image bytes (named Content, or something similar to that), and one for the internet media type string (named ContentType).

This topic is introduced in the following video:

.

Code example for today

In this Winter 2015 semester, your professor wrote a web app (and web service) to support his other course, DPS923, which is Mobile App Development for iOS.

The web app manages ‘sport’ and ‘venue’ data for this summer’s Pan Am Games in the Toronto area.

The Sport entity class has properties to hold a logo (which is a PNG), and a representative photo (which is a JPG). These properties are in the entity class, because a Sport can have only one logo.

The Venue entity class has properties to hold a representative photo (which is a PNG), and a map of the venue (which is a PDF). As above, these properties are in the entity class, for the same reasons.

sport-venue-classes

.

A portion of this app was posted as today’s code example, named BasicImageHandling.

.

Image upload, introduction

Your app will provide a way for the browser/user to “upload” an image. A number of components are involved:

1. The entity classes (defined in the design model classes source code file)

2. AutoMapper mappings to-from view model classes

3. View model classes, ‘add sport form’, and ‘add venue form’, which help define the HTML Form

4. Controller method that displays the HTML Form

5. A view that includes the necessary <form> and <input type=file…> attributes

6. View model classes, ‘add sport’, and ‘add venue’, which describe the data entered by the user

7. Controller method that accepts the data entered by the user

8. Manager method that saves the data

.

Entity classes

Done, above.

.

AutoMapper mappings

Also done, as a normal task when building the app.

.

View model classes, ‘add sport form’, and ‘add venue form’

The interesting thing to note here is that these ‘add xxxx form’ view model classes do NOT include any properties for the image. Instead – as you will see below – we hand-code the view source code.

.

Controller method that displays the HTML Form

Nothing special here. Done as a normal task when building an app.

.

View that includes the necessary code

The HTML Form MUST use the “enctype” atribute, with the value “multipart/form-data”. If you are using the ‘BeginForm’ HTML helper, here is the recommended constructor syntax:

@using (Html.BeginForm(null, null, FormMethod.Post, new { enctype = "multipart/form-data" }))

.

The file upload user interface element must be coded next. Its name MUST match the property name of the ‘add xxxx’ view model class in the next step. Here is a typical file upload user interface element configuration:

<div class="form-group">
    @Html.Label("Logo", new { @class = "control-label col-md-2" })
    <div class="col-md-10">
        <input type="file" name="LogoUpload" value="" class="form-control" />
    </div>
</div>

.

View model classes, ‘add sport’, and ‘add venue’

The view model class must include a property, of type HttpPostedFileBase. For example:

public class SportAdd
{
    public string Name { get; set; }
    // etc.
    public HttpPostedFileBase LogoUpload { get; set; }
    public HttpPostedFileBase PhotoUpload { get; set; }
}

.

Controller method that accepts the data

Nothing special here. Done as a normal task when building an app.

.

Manager method that saves the data

Some special processing is required here. The data and metadata from the HttpPostedFileBase object must be extracted, then stored. Here’s a typical approach:

public SportBase AddSport(SportAdd newItem)
{
    var addedItem = ds.Sports.Add(Mapper.Map<Sport>(newItem));

    // Handle the uploaded logo...

    // First, extract the bytes from the HttpPostedFile object
    byte[] logoBytes = new byte[newItem.LogoUpload.ContentLength];
    newItem.LogoUpload.InputStream.Read(logoBytes, 0, newItem.LogoUpload.ContentLength);

    // Then, configure the new object's properties
    addedItem.Logo = logoBytes;
    addedItem.LogoContentType = newItem.LogoUpload.ContentType;

    // etc.

    ds.SaveChanges();

    return (addedItem == null) ? null : Mapper.Map<SportBase>(addedItem);
}

.

Image delivery, introduction

This topic is introduced in the following video:

.

A typical web app design strategy is to create a controller that’s dedicated to image delivery, typically named “ImageController“.

The controller will have methods that deliver an image. The method is decorated by a “Route” attribute. Use a separate method for each kind of item that you want to deliver. For example:

  • sport logo
  • sport photo
  • venue photo
  • venue map

When you use Attribute Routing, you must enable that feature in the app.

In the RouteConfig class (in the App_Start folder), add this to the RegisterRoutes method:

routes.MapMvcAttributeRoutes();

.

The method’s structure looks like any ‘get one’ method. The big difference is that it returns a FileContentResult. Deceptively simple. The server will prepare an HTTP response as follows:

  • It will create a Content-Type header, and set its value to the <img>.ContentType value
  • It will then set the response message body to the value of <img>.Content

.

Here’s an example:

[Route("image/sport/logo/{id}")]
public ActionResult GetSportLogoById(int? id)
{
    // Determine whether we can continue
    if (!id.HasValue) { return HttpNotFound(); }

    // Fetch the object, so that we can inspect its value
    var fetchedObject = m.GetSportByIdWithImage(id.Value);

    if (fetchedObject == null)
    {
        return HttpNotFound();
    }
    else
    {
        // Return a file content result
        // Set the Content-Type header, and return the photo bytes
        return File(fetchedObject.Logo, fetchedObject.LogoContentType);
    }
}

.

The GetSportByIdWithImage() method returns an object that DOES include the image bytes and content type string. The object’s type is a view model class named SportBaseWithImage.

.

Using this controller 

So… any time you need an image, you can specify a URL like these:

http://example.com/image/sport/logo/123

http://example.com/image/venue/photo/345

For example, if you create an HTML img element, then set the value of its src attribute to the appropriate image.

.

General discussion of internet media type handling

The section above focused on one scenario. Real life is more complicated. This section includes a general discussion of internet media type handling in a web app.

Watch this video to get introduced to the problem:

.

Get content from a user

A section above covered this situation adequately.

For this discussion, the ‘save’ task can vary. We can save an internet media type (IMT) object:

  1. In the file system
  2. In the data store (i.e. the database)
  3. Using a hybrid mixture of these two

The following discusses some of the problems and solutions for each.

Please note that the author does not favour one approach (or later ‘recipe’) for all situations.

.

Implications – file system storage

You will have to create a folder to hold the IMT objects, maybe called “assets” or something that fits with the web app’s problem domain.

You may be tempted to use an existing folder, but resist that temptation. You cannot use App_Data, because its contents are not publicly accessible. You should not use Content, because that has a specific use now, and may evolve in the future.

File names are important. Their length must be reasonable. They must be unique. To simplify coding and handling, the character set used for the file names should be neutral, especially for a web app that’s used worldwide.

An IMT object is logically associated with an entity object in your problem domain. How do you maintain that association? Carefully. It is likely that the IMT object itself cannot store information about the association, and the file name may not be enough do do the job. Therefore, the entity object must include a property with the IMT object’s file name.

How should a file system based IMT object be delivered to a browser user? Using a URL that maps to its file system location and file name? Or through some other ‘managed’ approach?

Beyond file names, extension names, and created/modified dates, the file system will not support metadata querying. For example, it will become difficult (and non-performant) to handle a query for all photos larger than 1000px wide and 800px tall, for example.

It also may be difficult or impossible to store descriptive metadata in the IMT object (depending upon its format).

In a web app that uses both a data store and the file system, the web app manager now must manage two separate storage locations, and ensure they’re backed up and secure.

.

Implications – data store (database-hosted)

In a database-hosted data store, the IMT object’s data is stored in a byte array (the C# type is Byte[]). This data type maps nicely to both the storage and delivery components involved. As a result, it’s on equal footing with the file system in this respect.

You MUST store the IMT object’s internet media type string. When uploaded, that metadata is available in the Content-Type of the request (and available in the uploaded object’s ContentType property).

An advantage of this approach is that IMT object metadata can be stored alongside its data, as additional properties in the class that holds the IMT object’s data. For example, you can store a “Title”, and a lengthy “Description”. Other properties that are relevant to the IMT object can be stored, and their values can come from the browser user, or by programmatically inspecting the IMT object. (For example, the pixel resolution of an image can be extracted from the data, without user intervention.)

Delivery of the IMT object is managed by your app. Often, a special-purpose controller accepts an identifier in the request URL, and delivers the content accordingly.

Backing up the app’s data is simple, because there’s a single location for all the app’s data.

A frequent and notable criticism of this approach is that an IMT object is transformed when stored, then again when retrieved, from the data store. For large IMT objects, this work can be considerable, and may hurt performance. Acknowledged.

.

Implications – hybrid approach, using both techniques

It is also possible to combine both techniques. In this approach, your app stores the IMT object’s data in the file system, and metadata in the data store.

The file system location can be a new folder, or a subfolder of App_Data. Often, the file name is a GUID.

The data store class includes IMT object metadata, including (most importantly) the file name.

This approach is often used in situations where the IMT object size is large, or where there’s a large number of IMT objects to manage.

.

Entity class design considerations when using the data store

As noted above, when using the data store, an entity class that includes an IMT object MUST have two properties:

  1. A byte array, for the IMT object’s data
  2. A string, for the internet media type

Beyond that, other properties can be added, if desired.

If the design of an entity class is intended to hold one single, distinct, and unique IMT object, then it’s acceptable to simply add these properties to the entity class.

In the ‘base’ view model classes, DO NOT include these properties. (Recall from today’s code example that an IMT object is delivered in a standard and correctly-configured HTTP response.)

If an entity object needs a collection of IMT objects, then create a separate entity class for the IMT object, and configure the to-one and to-many (or whatever) associations needed to meet the needs.

Finally, if the primary purpose of the app is to manage IMT objects, then the entity class(es) must be designed around the needs of the app, and the IMT entity classes become the central part of the design model.

.

Solution ‘recipes’ 

In summary, there are a number of possible approaches for handling internet media types. Your situation will determine the best one to use initially.

The list below briefly describes a number of solution ‘recipes’. All have a storage destination that’s determined by the recipe’s ingredients. The scenario in today’s code example is covered by the recipe number 3. In the future (if/when your professor has time), code examples for the other recipes will be completed.

  1. File system, unmanaged delivery
  2. File system, managed delivery
  3. Data store, IMT data and metadata are properties of an entity class (this ‘recipe’ was implemented above)
  4. Data store, dedicated IMT entity class holds the IMT data and metadata, and may be related to other entity classes
  5. Hybrid, IMT is in the file system (with GUID name), and its metadata properties are in an entity class
  6. Hybrid, IMT is in the file system (with GUID name), dedicated IMT entity class holds the metadata properties, and may be related to other entity classes

.

Data import – load data from a CSV file

Importing data from a file is a common task for many web apps.

In this section, you will learn how to import data from a plain-text comma-separated values file (CSV).

In a later section, you will learn how to import data from a Microsoft Excel document file (XLSX).

This video introduces the ‘data import’ topic:

.

How to get the data file into your app

As a developer, or as a trusted user, there are two common ways to get a data file into your app:

During development, copy/paste the file to your project’s App_Data folder.

In the MSDN articled titled ASP.NET Web Project Folder Structure, the App_Data folder is described.

“Contains application data files including .mdf database files, XML files, and other data store files.”

“The content of [the App_Data folder]… is not served in response to Web requests, but… can be accessed from application code.”

This feature makes it ideal as a private (i.e. not publicly accessible) storage location for your app’s content, including uploaded files.

.

Then, in a code module (e.g. store initializer, or manager method), read and process the contents of the file.

After deployment, enable a trusted user to upload a data file to your app.

In the previous section, you learned how to allow a user to upload an image file to your app. The same technique can be used for this task.

After the file has been uploaded and saved (to the App_Data folder), its contents can be read and processed (probably by a manager method).

A code example, named “NFLQuarterbacks“, is available in the course’s GitHub code repository.

.

Prepare to work with the file system

You will need to work with the file system.

Your web app can be deployed on any web server. When deployed, its files are copied to a location in the file system, and the app is served by the web server software. The file system configuration of a web server can vary, so you cannot predict the file system path of your app, and its App_Data folder.

Therefore, you will need to get the file system path programmatically. In a manager class (or another service class in your app), this code will get you a string with the file system path of a file named “NFLStats2014.csv”:

// File system path to the data file (in this project's App_Data folder)
string path = HttpContext.Current.Server.MapPath("~/App_Data/NFLStats2014.csv");

.

In addition, you may need to perform file system tasks, including read, write, delete, and find. Add this to your source code’s ‘using’ statements:

using System.IO;

.

Reading and processing a CSV file

The comma-separated values (CSV) data format has been used since the 1970s. It is a plain-text data format, which has the column (or field) names in the first row, comma-separated, and data values in the remaining rows.

csv-file-example

.

We could read each line of text in the file, and do string parsing to extract the data values. However, we will not do that.

Instead, we’ll use a nice little library – CsvHelper – written by Josh Close.

You can add it to your project by using the graphical or command-line NuGet Package Manager. (The console command is “install-package csvhelper”.)

.

CsvHelper features

Similar to AutoMapper, it can automatically map CSV column/field names to property names in a class. Non-matching items are ignored.

In addition, it supports customized mapping, when a column/field name does not match a property name. We will use this feature in our app.

We will use a ‘CsvReader’ object. It features two ways to ‘read’ the lines/rows in the CSV file: 1) All at once, and 2) In a loop.

The ‘all at once’ way can deliver the results to a List<TEntity>, which is very convenient for in-memory operations. The reader’s GetRecords<TEntity>() method is used (notice the pluralized name, GetRecords).

The ‘in a loop’ way enables you to process each line/row separately, which is very convenient for saving new objects to a data store, or for data transformation tasks. The reader’s GetRecord<Tentity>() method is used.

.

Configuring and using CsvHelper

As noted above, add the library to your project with NuGet.

Also, as noted above, add a CSV file to your project’s App_Data folder.

In any code module that uses CsvHelper, add “using CsvHelper;” to the list of ‘using’ statements.

Write a view model class that matches the design of the data in the CSV file. If the column/field and property names do not match, then use the information in the “Customized mappings” section below.

Decide how you must process the CSV file. Do you need to display the contents? Save the contents in a data store? Something else? Whatever the decision, prepare the data structures that you’ll need.

Next, open the file for reading, read the contents, process the contents, and close the file. The following code suggests the approach. For complete details, study the Manager.cs source code in the code example.

// File system path to the data file (in this project's App_Data folder)
string path = HttpContext.Current.Server.MapPath("~/App_Data/NFLStats2014.csv");

// Create a stream reader object, to read from the file system
StreamReader sr = File.OpenText(path);

// Create the CsvHelper object
var csv = new CsvReader(sr);

// Configure the custom mapping class, if necessary

// Configure a collection to hold the results

// Process the file contents, all at once, or in a loop

// Clean up
sr.Close();
sr = null;

// Deliver or otherwise process the results

.

Customized mappings

Study the QuarterbackAdd view model class, and study the CSV data file format.

Notice that there are three column/field names that do not match. We will have to map:

  • the CSV data file ‘APG’ column to the ‘AttemptsPerGame’ class property
  • the CSV data file ‘YPC’ column to the ‘YardsPerCompletion’ class property
  • the CSV data file ‘YPG’ column to the ‘YardsPerGame’ class property

.

First, add “using CsvHelper.Configuration;” to the list of ‘using’ statements.

Then, add the following custom mapping class, at the bottom of the view model class source code file:

public class QuarterbackMap : CsvClassMap<QuarterbackAdd>
{
    // This class customizes the property-to-column mappings
    // It identifies the properties that have different
    // column (field) names in the source CSV file

    public QuarterbackMap()
    {
        // First, map all the properties that have matching names
        AutoMap();

        // Then, map the differences

        // When mapping AttemptsPerGame, look for the APG column
        Map(m => m.AttemptsPerGame).Name("APG");

        // When mapping YardsPerCompletion, look for the YPC column
        Map(m => m.YardsPerCompletion).Name("YPC");

        // When mapping YardsPerGame, look for the YPG column
        Map(m => m.YardsPerGame).Name("YPG");
    }
}

.

Next, edit the method that performs the CSV file processing. Immediately after the statement that creates the CsvReader object, ‘register’ the custom mapping class:

// Configure the custom mapping class
csv.Configuration.RegisterClassMap<QuarterbackMap>();

.

Data import – load data from an XLSX file

In this section, you will learn how to import data from a Microsoft Excel worksheet file (XLSX).

.

How to get the data file into your app

Similar to above, it can be done during development of your app, or after deployment. In both cases, store the file in the app’s App_Data folder.

.

Prepare to work with the file system

As above, you will need to work with the file system, and will use the same technique.

.

Reading and processing an XLSX file

The Office Open XML (XLSX) file format has been available since 2006, and is an open non-proprietary standard. It is the default data file format for Microsoft Excel.

xlsx-file-example

.

Recently, Dietmar Schoder released a nice little library to enable quick and easy XLSX file reading, without the need to add large libraries and toolkits to your project. In addition, it does not impose any configuration changes at the web server.

.

Excel library features

It is not as automatic as AutoMapper or CsvHelper. However, it is still easy to use.

Reflecting the hierarchy of Excel and its data file format, it enables you to open a workbook (which is the XLSX file itself), and then work with a specific worksheet within the workbook. A worksheet has rows, and within a row, you will find cells (also known as columns).

.

Configuring and using the Excel library

Add the library to your project. How?

1. Download the Excel library, and unzip it.

2. In Visual Studio 2013, in your project’s Solution Explorer, right-click the References item, and choose “Add Reference…”.

3. On the “Reference Manager” dialog, left side, click/select “Browse”, then click the “Browse…” button in the lower-right area of the dialog.

4. Navigate to the Excel library that you downloaded and unzipped (in step 1 above), and choose it, which adds the library to your project.

Also, as noted earlier, add an XLSX file to your project’s App_Data folder.

In any code module that uses the Excel library, add “using Excel;” to the list of ‘using’ statements.

Write a view model class that matches the design of the data in the XLSX file.

As you did earlier, decide how you must process the XLSX file. Display? Save? Something else? Prepare the data structures that you’ll need.

Next, open the file for reading, read the contents, and process the contents. The following code suggests the approach. For complete details, study the Manager.cs source code in the code example.

// Open the workbook, and get the first worksheet
var ws = Excel.Workbook.Worksheets(path).First();

// Process the rows in the worksheet
for (int i = 1; i < ws.Rows.Count(); i++)
{
    // Create a new object
    var qb = new QuarterbackBase();

    // Create a column variable, which simplifies the coding syntax
    var c = ws.Rows[i].Cells;

    // Configure the new object
    qb.Rank = (int)c[0].Amount;
    qb.Player = c[1].Text;
    qb.Team = c[2].Text;
    qb.Completions = (int)c[3].Amount;
    // etc.
    qb.Rating = c[14].Amount;

    results.Add(qb);
}

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

Advertisements
  1. No comments yet.
  1. No trackbacks yet.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: