Devices + Services: Near Realtime Sensor Data with Windows Azure Mobile Services, .NET MicroFramework, Pusher and ASP .NET MVC

It’s no surprise to people around me that I have a strong desire to code more, so I did just that on a Saturday several weeks back and here is what I built – a new devices + services scenario using a Gadgeteer, the .NET MicroFramework, Windows Azure Mobile Services and Pusher and Windows Azure Web Sites.

Device1

Lets build this:

WP_20130213_003

and get some live graphs that look like this – note up to you to do some better styling :)

graphs2 – see it live here

The idea I set out with was to open up the power (and ease) of Windows Azure Mobile Services to embedded devices that can run .NET MicroFramework.  The scenario I come up with was to capture some sensor data, insert it into a Mobile Service to store the data for historical purposes and also provide a near real time stream of data for clients such as your web browser using Pusher one of our Windows Azure Store Partners.

Capturing Sensor Data

For this scenario I am using the GHI Gadgeteer FEZ Spider and I have already installed all the pre-req software from GHI   An embedded device that runs the .NET MicroFramework and has a bunch of plug and play sensors for rapid prototyping.  To collect the sensor data is quite straight forward

use the designer to wire up your sensors

GadgeteerDesigner

The following code block will show how to open the network and capture sensor data in this case light, humidity and temperature – is quite straight forward and resides within Program.cs.

With that we have now captured our sensor data which is good, but to be useful to me I wanted to store this data off the device. To do this I decided to use Windows Azure Mobile Services

Storing the Sensor Data in Mobile Services Ok the first thing that you will need to do is Create a Mobile Service. Within the Windows Azure Portal Click New+  

plus-new

Select Compute, Mobile Service, Create

Create

Provide a subdomain name for your Mobile Service. This will be the endpoint that your device writes sensor data to and complete the wizard

createMobileService

Once your Mobile Service is provisioned.  Click on it and select the Data Tab and click Add a Table AddATable

Call this table SensorReading and save

SensorReading

Great, with that done we now have a Windows Azure Mobile Service ready to receive our data from our embedded device.  Interestingly Windows Azure Mobile Services already has client SDKs for Windows Store, Windows Phone and iOS apps.  If your an Android dev support for Android coming soon – giggidy. 

All of these client SDKs consume the REST API that Mobile Services provides for you out of the box.  One of the powerful things that the client SDKs offer is a really simple client API for consuming these services for things like data storage and Auth. 

My goal was to provide something that gives a similar experience to .NET MicroFramework devs because hey if you’re already writing software for embedded devices which is hard enough why shouldn’t you get an easy backend too? I invested a Saturday afternoon on this and provided a quick v-slice for the Insert operation.  To do this I took the Windows Phone Client SDK for Mobile Services and started porting it to .NET MicroFramework.  Man I must say I feel for you .NET MicroFramework devs:

  • no<Generics>
  • no LINQ
  • no JSON Serialization
  • limited Reflection capability – where did the GetProperties method go in .NET MF, I noticed PropertyInfo was there but couldn’t actually find GetProperties?

It was like coding back in the dark ages .NET 1.1 ah yes I remember those days. Quite  a wakeup call of how far we have really come!  So needless to say the Saturday afternoon was longer and not as productive as expected, sort of typical right?  Anyhow I got out a partial port that provides Insert functionality and a quick and pretty dirty JSON Serializer.  So here is what it looks like to use it to insert data to your Mobile Service. First we’ll need to add a reference to the (unofficial) Microsoft.Azure.Zumo.MicroFramework assembly. You can find the class library for this in the Where’s teh codez section towards the end

References

Now create a class SensorReading.cs, this will be the entity you will insert in your Mobile Service

To interact with our mobile service lets now create a new instance MobileServicesClient in your Program.cs

One thing you will note here is that I have not looked into how https is handled in the .NET MF so I am just using the http protocol for writing up to the Mobile Service I still need to investigate support for https on .NET MF. Now lets update temperatureHumidity_MeasurementComplete to create a new sensor reading and store insert it into your Windows Azure Mobile Service (backed by a Windows Azure SQL Database)

That’s it pretty easy right! You’ll note from an API consumer perspective that its pretty close to the experience you get for the Win Store and WP client SDKs minus the fact that I didn’t write a JSON Deserializer yet so I am not currently rehydrating the entity that was passed to the Mobile Service with the updated Id or updated properties…

So now that I had my sensor readings being stored in Windows Azure Mobile Services I wanted to look at how I could visualize this data.

Sending the sensor data to listening clients using Pusher

This is where things get pretty sweet. I wanted to visualize my sensor data in a graph as it arrived in my Mobile Service. Recently we announced a new Windows Azure Store partner – Pusher a  WebSocket Powered Realtime Messaging Service.  Within the Windows Azure Store you can quickly provision a Pusher account and utilize it from Mobile Services within minutes as follows.

Press New+

plus-new

Select Store

NewStore

Select the Pusher add-on

StorePusher

Walk through the remainder of the Wizard to select your plan and get your keys

PusherPlan

Note: if the Windows Azure Store is currently not available in your region you can sign up directly on Pusher.com

Next in the Windows Azure Portal select your SensorReading table and click the script tab and set the dropdown to Insert. This will allow us to write code, called Server Scripts, that will be executed every time our Gadgeteer inserts data into our SensorReading table in Mobile Service.

insertscript

Next we replace the insert code of our script to do two things save the data to our database and then use Pusher subscription to send our sensor data as it is received to all listening clients.

 

Graphing the data received via Pusher in a web client running on Windows Azure Web Sites

So now we had our sensor data collected, inserted and stored in Mobile Services and then pushed using pusher to any listening clients.  What I wanted was a web client to visualize the data in a graph as it arrived. You can learn how to create a free Windows Azure Website using ASP.NET MVC and deploy it to Windows Azure here – http://www.windowsazure.com/en-us/develop/net/tutorials/get-started/

The following code shows how you can write a quick, not so pretty code+UI wise,  graph that will receive the sensor data live via Pusher and update the graph.  To do this I am using jqplot and the Pusher JavaScript library. First add your script references to _Layout.cshtml

Then add your jquery.jqplot.min.css to your /Content folder. Next we need to Update our Home/Index.cshtml view to listen/bind to our Pusher Channel and then redraw our jqplot graph as data is pushed directly from Pusher.

If you read through the code you will see pretty clearly that the Pusher implementation is 3 lines of code only – to me this is extremely cool. Itty bitty amount of code, phenomenal cosmic power!

So that’s it now we have live graphs on our website, you can checkout a running version of this code and it live graphs that I deployed to a Windows Azure Web Site here – http://microframework.azurewebsites.net

How much does it cost

Everything that I did here can be done for free with Windows Azure Windows Azure Free Trial and/or the great free tier offerings for Windows Azure Web Sites, Windows Azure Mobile Services and Pusher.

Where’s teh codez?

This is unofficial, is not supported – I did it in my free time and it Works on my machine! :)  disclaimers all being said I really hope that this does open up a lot of doors for you for building out a whole new range of devices + services scenarios using Windows Azure and our Store Partners You can download the .NET MF lib and sample code from this github repo

Summary

I hope this has opened the door to great new devices+services scenarios you can build out for your .NET MicroFramework solutions. With few lines of code and powerful services like Windows Azure Web Sites, Windows Azure Mobile Services and Pusher you can make working in the emerging embedded devices + services space a lot easier then it has been in the past. Please do let me know if you have built something awesome in this space on the Twitterz @cloudnick

  • http://www.leggetter.co.uk Phil Leggetter

    Lot’s of fun, huh, Nick :)

    A while back a guy called Paul Fryer created a Pusher client for the .NET MicroFramework[1] app can also receive data via Pusher. He used it in an app which turned a lamp on and off via Twitter[2].

    This opens up even more opportunity for instant (near realtime) cross device communication. I’m really excited to have more .NET developers using our technology. Thanks so much for the post.

    [1] http://microcontrib.codeplex.com/SourceControl/changeset/view/96133#1803815
    [2] http://blog.automagical.ly/post/31867187520/twitter-box-unveiled-at-portland-maker-faire

  • http://twitter.com/jhalbrecht Jeff Albrecht

    Hi Nick! Cool project.

    I grabbed that .dll and included it in a Netduino 2 Plus application targeting .netmf 4.2. It all went smoothly except when attempting to consume the data in a windows store application. It didn’t like the way the DateTime was serialized, json.net crashed. I then made a test with the same simple class from a windows phone 8 app to check the Azure serialization. Somewhere in .netmf and/or on it’s way to the cloud the serialization failed. In the netduino code I did a DateTime.UtcNow in Azure it looks like: “Time”:”2013-02-19T16:46:55.205K” (bad) the same class from the phone application looks like: “Time”:”2013-02-19T18:39:51.733Z” (good) The ‘K’ from the .netmf is the culprit.

    Any ideas on how to get Azure to serialize it correctly in the MobileServiceClient or possibly I have found a bug in DateTime.UtcNow in the netduino 4.2.2 code? Thoughts?

    But way cool anyway, thanks for blogging your project. I guess I’ll get back to reading about TimeZone in .netmf some more, and failing that I think I can write a json.net json converter.