I’ve had the good fortune of working with my friend Jonathan Sampson recently on figuring out how to help developers build plugin-free experiences. With IE10 Metro going plugin-free, it’s incredibly important to document steps to help developers provide their users with great experiences without the need for proprietary 3rd party add-ons.
If you’ve built a plug-in-free browsing experience for the iPad, a few changes will make it ready for the new IE10 plug-in-free experience on Windows 8. As more browsers adopt the plug-in-free approach, now is a good time to start thinking about it. I’ll show you how to do this in a few steps below by writing code that works well in all modern browsers.
Today we’re going to work with a MSNBC plug-in-free experience for rich media. It breaks down to two things: styles and scripts.
To modify the files of MSNBC, I will be using a proxy application known as Fiddler. You can download this tool from http://fiddler2.com. This tool allows me to modify remote files as though they were on my local machine. If you have direct access to your own site, you can ignore Fiddler, and work directly with your files. Fiddler provides a great way for testing changes without the risk of breaking your live site.
Step 1: Declare Standards mode and valid markup for modern browsers
In order to use the HTML5 elements we’ll be utilizing below, you’ll first need to ensure that you are operating in standards mode. One way to ensure this is to include the HTML5 doctype at the top of your document:
[code lang=”html”]<!DOCTYPE html>[/code]
Step 2: Update your CSS vendor prefixes
The CSS language is constantly in a state of change as new features are suggested, updated, and standardized. In order to allow developers to learn how to use these new features, browser vendors typically offer experimental implementations via prefixed properties.
A key part of using vendor prefixes responsibly is to ensure that prefixes from each vendor are included in your site to allow for the broadest level of feature support. In many cases, especially when building an iPad-centric site, you may have focused solely on -webkit properties, omitting the prefixes which target other browsers such as -o, -ms, and -moz. The end result of this is that you greatly limit the target devices that can render your plugin-free site to as well as provide a degraded experience for users of other modern browsers, many of which could serve up equally engaging functionality.
For instance, we find the following on MSNBC:
[code lang=”css”]background: -webkit-gradient(
linear,
left top,
left bottom,
color-stop(1, rgba(192,192,192,.6)),
color-stop(0.5, rgba(0,0,0,.6))
);[/code]
With the growing trend towards an HTML5 plugin-free experience, it’s important to expand these rules to provide the vendor prefixes of other major browsers as well.
[code lang=”css”]background: -webkit-linear-gradient(
top, rgba( 0, 0, 0, 0.0 ) 0%, rgba( 0, 0, 0, 0.6 ) 50% );
background: -moz-linear-gradient(
top, rgba( 0, 0, 0, 0.0 ) 0%, rgba( 0, 0, 0, 0.6 ) 50% );
background: -ms-linear-gradient(
top, rgba( 0, 0, 0, 0.0 ) 0%, rgba( 0, 0, 0, 0.6 ) 50% );
background: -o-linear-gradient(
top, rgba( 0, 0, 0, 0.0 ) 0%, rgba( 0, 0, 0, 0.6 ) 50% );
background: linear-gradient(
top, rgba( 0, 0, 0, 0.0 ) 0%, rgba( 0, 0, 0, 0.6 ) 50% );[/code]
While more verbose but the benefits to broad browser feature support certainly outweigh the extra typing involved. In addition, there are a number of great tools that can break down this workload, such as SASS and Compass, -prefix-free, or even CSS Snippets in the upcoming Visual Studio 2011.
Also, if you’re working predominantly in JavaScript and would like to save time determining which features are supported by your client’s browser, review the instructions in A Best Practice for Programming with Vendor Prefixes on the IEBlog.
Step 3: Get rid of browser sniffing methods
There are two methods used to determine what the user’s browser and device are capable of. One method, which unfortunately is somewhat popular, is browser sniffing. This method consists of examining the navigator object for certain patterns or values.
[code lang=”javascript”]if ( navigator.userAgent.indexOf("iPad") > -1 ) {
// Load HTML5 Experience
} else {
// Load Flash Experience
}[/code]
The above code looks at the user agent string for the value “iPadâ€, and if found delivers a plug-in-free HTML5 experience. Otherwise, it’s assumed you are on a device that has Flash installed. This will result in a broken experience for non-iPad users who are browsing with plug-ins disabled, even though their browser is capable of handling HTML5 features.
Here is an attempt to find the version of Internet Explorer.
[code lang=”javascript”]if ( tests.IE ) {
j = /msie.(\d\.\d+)/i;
k = navigator.userAgent.match(j)[1];
}[/code]
The user agent string is tested for a pattern that attempts to target the version number. This pattern looks for a single digit, followed by a period, followed by any number of additional digits. While this test will find values like “MSIE 8.0†and “MSIE 9.0â€, it will not identify the latest version of Internet Explorer, which identifies itself as “MSIE 10.0â€, since only one digit is expected before the period.
These are just a couple examples of why browser sniffing is not a best practice. The user agent string is not immutable – it is a read-write value that is easily changed by plugins, or even the user. Most modern browsers include the ability to easily change this value from their development tools, which some users take advantage of to get around poorly-developed websites.
If we disable plugins, or visit MSNBC from a device/browser that doesn’t have Flash, we would expect it to attempt a plug-in-free experience. Unfortunately, this is not the case. Rather than seeing an HTML5 experience, we’re instead asked to download Flash. This is because the site puts the user in one of two categories: an iPad user, or a Flash-enabled user.
Feature Detection
Rather than trying to guess what a browser is capable of by sniffing its user agent string (which will fail you eventually), it is much wiser to actually test features directly in the browser. If you wanted to test the browser’s ability to deliver video and audio via HTML5, you could actually attempt to create these elements via JavaScript, and see if the browser understands them. This practice is called feature detection.
[code lang=”javascript”]if ( !!document.createElement(“videoâ€).canPlayType ) {
// Load HTML5 Video
} else {
// Load Flash Video
}[/code]
In the above example, we start by testing whether the canPlayType method exists on our newly-created video tag. We’re using double-negation to cast the response to a boolean. If the browser understands what a video element is, the canPlayType method will be present. If the video element is unknown to the browser, the canPlayType method will not exist. If this test passes, we load our HTML5 video. If the test does not pass, we attempt to load Flash. Deeper feature detection could take place here, since Flash may not be on the machine, or may be disabled.
Feature detection is the preferred method of determining what a browser is capable of, since there is no guesswork involved. If the browser passes properly-constructed tests, it absolutely supports the features you would like to use.
Many great tools exist to provide feature tests for you. Once such tool, which provides over 40 tests, is Modernizr. This tool creates a global object called “Modernizr†which contains the results of your tests. With Modernizr, testing for HTML5 video support is extremely easy:
[code lang=”javascript”]if ( Modernizr.video ) {
// Load HTML5 Video
}[/code]
MSNBC engages in browser sniffing to see if the device accessing the page is an iPad or not. Our first step is to remove the browser sniffing code, and replace it with feature detection code.
Before we can modify browser sniffing code, we first need to locate it. While in Internet Explorer, pressing F12 will pull up our Developer Tools. Within the tools, open the Script tab and do a search for “userAgentâ€. This search will seek out any instance of this property name in all of the site’s script files. We’re interested in the result from line 41 of http://www.msnbc.msn.com/id/37156949/.
Now that we know what we want to edit, we can open up Fiddler and load up our traffic. Once Fiddler is opened, perform a hard-refresh (Ctrl+F5 in IE) on the MSNBC page. This results in all of the page sessions being listed in Fiddler.
Looking carefully, you’ll notice our resource is the third from the top. Next I will setup an AutoResponder for this session file so that anytime it is requested, my own custom file is substituted in the place of the server response:
- Right-click this session and select “Decode Selected Sessions†from the context menu.
- Select the AutoResponder tab on the right.
- Click the “Enable automatic responses†checkbox in the AutoResponder tab.
- Drag the selected session from the left panel into the AutoResponder tab.
At this point, you should have an entry within your AutoResponder tab with the following rules:
- If URI matches: EXACT:http://www.msnbc.msn.com/id/37156949/
- Then respond with: *200-SESSION_3
Right-click the entry in the AutoResponder and select Edit Response. In the popup that follows, switch to the SyntaxView tab where we will find the source for this file. As expected, line 41 contains our browser sniffing code:
[code lang=”javascript”]if(!(navigator.userAgent.toLowerCase().indexOf("ipad")>-1)){
// Flash Experience
}[/code]
Rather than test the contents of the userAgent, we’re going to instead look for support for the HTML5 video tag. Switch the above condition to the following:
[code lang=”javascript”]if ( !document.createElement("video").canPlayType ) {
// Flash Experience
}[/code]
This test checks to see if we cannot use the video element. If canPlayType comes back as undefined, it will be cast to true and the first code block will be entered, setting up the Flash experience.
Step 4: Update touch and pointer events
Safari supports both a touch event model and a mouse event model. Internet Explorer 10 groups touch, mouse, and stylus events into a single abstract item known as a pointer. In fact, Internet Explorer 10 is the first browser to work for all input types, across all devices. This abstraction cuts down drastically on the amount of effort involved to determine which event model you ought to bind to and how to detect user-interaction. This pointer is then handled through MSPointer events. If necessary, you can determine the type of pointer by accessing the pointerType property.
Due to the fact Internet Explorer doesn’t support Apple’s proprietary event model, which includes touch events like touchstart, touchmove, and touchend, MSNBC’s event listeners will need to be amended to listen for MSPointer events like MSPointerDown, MSPointerUP, and MSPointerMove.
Due to the difference in event model implementations, use a feature detection tool like Modernizr or code like this to target all major event models:
[code lang=”javascript”]if (window.navigator.msPointerEnabled) {
myCanvas.addEventListener("MSPointerMove", paint, false);
} else {
myCanvas.addEventListener("mousemove", paint, false);
myCanvas.addEventListener(“touchmoveâ€, paint, false);
}[/code]
MSNBC only supports touch events, which we will need to change so that visitors who happen to be using a mouse can still interact with the page:
Our events are tied up in http://www.msnbc.msn.com/id/43662671/15:
[code lang=”javascript”]document.addEventListener("touchstart", touchHandler, false);
document.addEventListener("touchmove", touchHandler, false);
document.addEventListener("touchend", touchHandler, false);[/code]
We’re going to update this to include the MSPointer events as well:
[code lang=”javascript”]if (window.navigator.msPointerEnabled) {
document.addEventListener("MSPointerDown", touchHandler, false);
document.addEventListener("MSPointerMove", touchHandler, false);
document.addEventListener("MSPointerUp", touchHandler, false);
} else {
document.addEventListener("touchstart", touchHandler, false);
document.addEventListener("touchmove", touchHandler, false);
document.addEventListener("touchend", touchHandler, false);
document.addEventListener("mousedown", touchHandler, false);
document.addEventListener("mousemove", touchHandler, false);
document.addEventListener("mouseup", touchHandler, false);
}[/code]
First we’re checking for the presence of pointers. Since the MSPointer covers the mouse, fingers, and pens, we don’t need anything else besides them. We fall back, if necessary, to provide both touch and mouse events.
Next we need to create cases for these event types in http://www.msnbc.com/id/44937131/. Currently, MSNBC starts with the following:
[code lang=”javascript”]if ( event.type == "touchstart" ) {
/* Start drag logic */
} else
if ( event.type == "touchmove" ) {
/* Drag logic */
} else
if ( event.type == "touchend" ) {
/* Complete drag logic */
}[/code]
We’ll modify this to listen for all of the registered event types:
[code lang=”javascript”]if ( event.type.match( /(down|start)$/i ) ) {
/* Start drag logic */
} else
if ( event.type.match( /move$/i ) ) {
/* Drag logic */
} else
if ( event.type.match( /(up|end)$/i ) ) {
/* Complete drag logic */
}[/code]
The above uses the match method and a series of regular expressions to determine which event was raised. If the event raised ends with a case-insensitive “down†or “startâ€, we begin our drag code. If the event ends with a case-insensitive “moveâ€, we perform the actual drag logic itself. And lastly, if the event ends with a case-insensitive “up†or “endâ€, we end our dragging event. Note: other events may be caught here as well, like onresizeend and keyup. Be sure to consider this in your project.
The above is an implementation of Ted Johnson’s solution in Handling Multi-touch and Mouse Input in All Browsers.
The drag logic itself initially relies upon the event.targetTouches TouchList. This member does not exist in Internet Explorer. The drag logic attempts to gather the pageX and pageY properties from the first item in the TouchList, however in Internet Explorer these values are found directly on the event object.
[code lang=”javascript”]var curX = event.targetTouches[0].pageX;[/code]
Using the logical OR operator, I instruct curX to hold the value of event.pageX as long as event.pageX is present on the event object. If this property is not found, look within the targetTouches list:
[code lang=”javascript”]var curX = event.pageX || event.targetTouches[0].pageX;[/code]
If event.pageX is not found, we fall back to assigning the value of targetTouches[0].pageX to our variable.
Another important item to keep in mind is that this site initially responds to touchmove. When this event is raised while touching the playlist, the code attempts to reposition the playlist based upon your touch movement. There is no hovering when it comes to touch – you’re either touching, or you’re not.
Now that we have mouse events tied into this logic, we have introduced the possibility for hovering. So while touchmove is free to reposition our playlist when it is over the playlist, we don’t want to do the same for mousemove. In fact, we only want the mousemove event to reposition the playlist when the mouse button is pressed.
For further reading, and examples on how to target all browsers, see Handling Multi-touch and Mouse Input in All Browsers.
Testing both experiences
Recall our feature detection from earlier, how we first check to see if HTML5 video support is in the user’s browser. If it is, we give them HTML5. If it is not, we give them Flash. One easy way to test our work is to use a browser, or document mode, that doesn’t support HTML5 features. This is very easy to test with Internet Explorer:
- Press F12 to reveal the Developer Tools
- Change your Document Mode to Internet Explorer 7 Standards
- Refresh the page
If our feature detection condition was written properly, you should now be watching a Flash-based presentation. Switching your Document Mode back into Internet Explorer 9 Standards (or “Standards†if you’re using IE10), will return you to the HTML5 experience.
Get it Done!
Hopefully this post helps to define the types of changes that will allow your iOS site to work properly in IE10 Metro and other plugin-free environments. By including best practices such as feature detection and responsibly using vendor prefixes for great new features, you should be able to provide your users a great experience, regardless of which browser or device they’re using. To assist with testing in other plug-in-free environments, download Internet Explorer 10 (currently available only in the Windows 8 CP) and begin testing today!
Update: In the rush to get this post up, I realized that I forgot to thank and give credit to Jonathan Sampson for helping investigate and write about the great techniques mentioned above. He was a huge help in generating many of these great techniques. Thanks JS!
Honestly? In my experience iOS is the lowest common denominator. Unless you’ve hacked your stuff to high hell as mentioned above (in which case you should stop scripting and go find another job), it should just work in any browser that actually has something resembling modern support for… anything. This includes click drag support, failover, regression, etc.
I hear ya. Unfortunately, not many sites take your approach. You’d be surprised how much device/browser sniffing is happening even on modern sites.
Didn’t you get the memo? HTML5 is a false messiah, with web-based applications losing relevance with each passing day – sure, lowest-common-denominator applications that bring mediocre user experiences to the masses, despite the browser compatibility issues that will always exist (‘write-once, deploy everywhere’ would, in reality, be massively detrimental to all of the largest technology vendors, dropping to near-zero the differentiation that native platforms make possible), will be confined to the entirely mundane aspects of our business, none worth working on, like consumer sites, for examle. Windows 8,like iOS, is all about native applications, deployed and updated via app stores, not a whole bunch more crap delivered via the traditional web and resulting in (yawn) a yet another lame web experience trapped within IE10. The present and future belong to occasionally-connected, cloud-enabled native applications. Nothing to be gained investing in anything less…
IMHO, they should transfer Silverlight so that it should be HTML5/JS/CSS3 based, and get rid of the Silverlight .NET client profile plugin, let’s say WinRT should be able to work as an ASP.NET server emitting html content instead to render the XAML UI designed on server, whatcha say?
While it’s not been confirmed, the WinRT IS the next Silverlight, and it supports building Metro based apps in HTML5+CSS+JS, emitting XAML to compiler.
Well, for a couple of reasons, but none that any one voice in Redmond is yet including – and likely, not permitted – in a cohesive presentation. First, take all of the mighty powerful things that are Silverlight and set them off to one side. Next, stop to truly consider a couple of things. To start, the essence of Silverlight is a minimalistic and entirely portable runtime that kicks the ass of WPF on the desktop, has the power to go head-to-head with Flash and makes anything in HTML4/5 look as entirely primitive as it is (and as will be the case for years and years to come). Next up, that runtime’s been proven viable on a Windws machine, a Mac, Windows Phone and the XBox, plus its been proven to work on jail-broken iPhones and other devices. Going further, in what areas of Silverlight have we see massive investment in versions 4 and 5? OOB (out-of-browser) capabilities. All of this hopefully begins to suggest what Redmond is holding back – that Silverlight running as a browser plug-in is, to them, only incidental and of ever-diminishing strategic value. Why? Try this perspective, starting with why, when devices of ever-increasing power and decreasing size are bevcoming a commodity, why render anything at all on the server, really? Next, despite the fact that nearly all web applications that don’t use Flash or Silverlight suck and that browser compatibility issues are a complete waste of time and money, what did web applications have to offer that made them worthwhile? Ease of deployment – push your latest version to the web server or web farm and bang, everyone’s running your latest and greatest version. Now, two things have changed, two things that make all the difference: 1) The move toward mobile has made abundantly clear that users vastly prefer native applications over their lower class, low-budget counterparts and 2) the app store model, a key component in Apple’s world dominance, makes it entirely possible – and worthwhile – to bring to market vastly superior, cloud-enabled native applications with just the ease of deployment that used to be exclusive to the web. With the traditional web fading in importance and the major technology vendors having little or nothing to gain from the fallacy of ‘write-once-deploy-anywhere’, what would you do? Pull back from Silverlight in the browser, pull back from Silverlight as a cross-platform vehicle, because neither has the longer term significance once imagined. From that perspective, realize that WinRT is, heart and soul, the same – but perhaps evolving – Silverlight runtime, but a rose by any other name, and what it remains is a runtime capable of spanning Windows 8 on the desktop, tablet, XBox, phone and more. Throw in the deployment of both public and private app store capability and what do you have? A mighty, mighty ecosystem which, if not entirely mishandled, puts you in a place to actually compete in this vastly different world. Silverlight as an HTML5 producer? Nope, not needed. Despite the lip service being paid to HTML5, it was never about that, not for Apple and not for Microsoft. That it will continue to suck for web developers foolishly choosing to or stuck in developing in accordance with ‘web standards’ serves them, and many others, only too well. Go native and go now…
+1 on Shimmy’s approach. New Windows 2012 XAML technology (including LightSwitch) should have options to render the client for WinRT and/or HTML5.
Kay, I think he is just stating that the industry desire to cater to all things “i” plus the years of mental conditioning that you must check browser brand to guarantee functionality of “x” along with the assumption you can bully people into downloading flash (the last one I especially don’t care for) just has created an environment where all of the practices above were acceptable (depending on where you are).
I think the OP is just trying to get the point accross that it is a bad practice, even moreso in todays world. It does seem however that before you make these changes, some of this stuff should be standardized (like the event handling which may be but I haven’t done JS in web dev for quite some time) and worries me that he’s refering to them as MS objects/events rather than events that everyon (except apple) is using. Seems that would just create more desire for everyone to find an alternate way (since everyone seems to hate MS so they can feel like they are part of the hip crowd). The implementation seems to make a lot of sense and be very straight forward though so I’m looking forward to playing around with touch soon.
WinRT is client only.
I’m looking for a server-client solution.
I should be able to develop a WinRT application that is launchable from every browser! That would be awesome, and will beat flash and all the other crap.
Furthermore, say tomorrow HTML5 is dead, whatever technology replaces it, as long as it’s not better than Silverlight, Silverlight-WinRT, or whatever you name that new technology (written in your preferred language including C++ and JS and thus probably even Python, C and others), will be rendered in the client’s browser language seamlessly.
As per LightSwitch, I think this should indeed be related, but the problem of LightSwitch is that it’s too limited. IMHO, LS should generate partial classes allowing the dev complete control of every generator method, feature, UI control etc.
It’s a pity that MSFT has the best development languages and IDE, but doesn’t take advantage and maximize its capabilities. Time will tell.
What do you say boys?
@Shimmy,
HTML5 is a W3C standard so there is a pretty good chance that it will stick around for some time.
What are your thoughts on KnockOut? Isn’t KO the best technology to build cross-browswer apps in a short amount of time?
BTW: I agree with you about LightSwitch being limited. However, it does a very good job of building RAD forms biz apps. V2 is in beta and I’m looking forward to an improved Extensibility Toolkit before V2 RTM.
The bit about Apple’s “proprietary event model” rings false. While Apple ran out an implementation before there was a proper spec, there is now a draft spec: http://dvcs.w3.org/hg/webevents/raw-file/tip/touchevents.html and Android implements roughly the same thing, with the same events.
Labeling this proprietary makes it sound like there was some legal or technical blocker from using the same kind of spec: there was none, and MS’s namespaced events are afaik an arbitrary decision and a bad one.
A major problem with that spec is Apple claiming patents for touch events:
Apple Abuses Patent System Again To Obstruct W3C Open Standard
http://my.opera.com/haavard/blog/2011/12/09/apple-w3c
Until that’s resolved, the spec is basically in a holding pattern and for all intents a proprietary document since the W3C is basically in legal limbo.
Oh great… MS veering off to do their own thing with touch events? Jeez this feels like the 90’s all over again… so much for standards…
Actually, no set standard for touch events has been approved by the W3C. As Tom MacWright mentioned, there is an editor’s draft spec submitted but it’s not been approved and requests for feedback is still completely open.
It’s easy for you to focus on MS but I would argue that the bigger issue that can put us back to the 90’s are the proprietary “-webkit” features that developers are using, especially in mobile, which are leading us into total vendor lock in.
You must really hate Google — with all of their special Chrome additions, it’s like the 90s all over again, only more so.
“var curX = event.pageX || event.targetTouches[0].pageX;”
Can’t event.pageX have a value of 0 when using MSPointer events?
OT: I could not test it, as they are only supported in IE10 version that requires avalability of spare current generation computer, which I lack thereof.