February 20th, 2014

Context aware design

Applying user context to interfaces.

Responsive web design isn’t entirely new concept. Broken down into simple terms, responsive web design is the practice of refactoring a page’s layout based on the size of the screen used to interface with it. Certainly there is more complexity to it, but that’s the gist.

It’s a great idea and it works on some simple CSS3 properties. As a result the same web page might look very different on a mobile phone versus a desktop computer. In a sense this is a primitive form of context aware computing, but what if we could extend that further?

What if we had a page that resized to a format fitting for the device, like in responsive web design, but also could format key portions of the site based on environmental data, such as weather or time of day. In this prototype, I’ve built a simple module that promotes coffee in the morning and tea at night, and if the temperature is under 50 degrees where you are it offers a hot drink. If it’s over 50 degrees, it offers an iced drink.

See the Pen EbAax by Matthew Carver (@matthewcarver) on CodePen.

If you view that demo and it’s a cold night it will read “We have hot tea!”, and if you read it on a hot morning it’ll read “We have iced coffee!”

Context Awareness in Applications

Context awareness is a property used in mobile devices to identify where the user is using an application and how that might affect what the user is doing. For instance, if you open an app for a restaurant it might be able to know that you are a mile away and immediately give you hours and menu, but if you’re 500 miles away from the restaurant you would get information on how to make a reservation for a future visit.

Thanks to recently advances in wearable computing, contextually aware systems have been able to use more and more sensors to gather more data and better inform context. For instance, let’s say you’re wearing a Nike Fuelband and just finished a jog. Your phone might receive data from the Fuelband and automatically route unimportant calls to your voicemail while it waits for your pulse to drop back to normal.

In iOS7, Apple introduced an M7 coprocessor for tracking motion activity. It uses a combination of accelerometer, gyroscope, and compass to detect the type of motion a user is engaged in. Whether they are stationary, walking, running, or in an automobile are all accessible data properties.

While this has obvious implications in fitness and health applications, there is also potential in e-commerce specific apps. For example, a user that is stationary might have more precise control and therefore user interface objects such as buttons can be a standard size, while a user who is walking would have less precision and need larger buttons.

In applications, a lot of this context is ready to be accessed and implemented into project where applicable, but it’s in the realm of native app development. While the type of data that a web client can receive is different, there is currently a lot of potential for designers to create front-end web designs that appropriate themselves to the user’s context.

Context aware UI for the Web

Context is already being applied on the web, but a lot of it is happening in the back end. If a user is logged in they can be served products or content relative to their interests from the server. Facebook is the king of doing this. The news feed algorithm is geared towards giving you exactly the content that you are most likely to engage in an attempt to provide a satisfying experience. This version of contextual awareness is fantastic, but oftentimes the user interface itself is left behind.

There’s a myriad of data available to designers that can be used to optimize a user interface on behalf of the user. A deeply responsive web design that responds to more than just the user’s screen size can provide a more satisfying overall web experience for all users. There are breakpoints that can potentially be shared between touch screen devices and traditional laptop computers. I recently found this while digging deep into a clients’ site and felt unsatisfied with the tappability of certain UI elements in the tablet experience.

The mouse pointer is so much more precise and doesn’t require nearly as much space to be clickable as a touch screen. Instead of using screen width to determine all the aspects of a user interface and delegate the design into one of three buckets, “mobile”, “tablet” and “desktop”, what if we built versatility into our designs?

Level 4 media queries

Media queries gave birth to the responsive web. By using them we were able to distinguish between screen sizes. They revolutionized the way we design web pages and how we collaborate between designer and technologist. As responsiveness becomes more standard, there’s a new set of challenges on the horizon. In the new specifications for CSS4, there are proposals for a fourth level of media queries that address issues with interaction features as well as a media query for environment.

Within the interaction specifications, there are new rules for “pointer” and “hover.” These two provide some contextual awareness around how the client can interact with the site. “Pointer” is to address the issues surrounding the accuracy on the users pointing devices. If the pointing device has a high rate of accuracy, like a mouse or Wacom tablet, it’s identified as “fine”. If it’s not as accurate, in the case of something like a touch screen or the Kinect’s motion control, it’s defined as “coarse”. These media queries might end up looking something like this:

@media (pointer:coarse) {
.btn{…}
};

Similarly, there are specifications around hover, which would indemnify whether or not a browser can “hover.” If the pointer has the ability to hover over parts of the page, it uses the @media(hover:hover) media query. However, in the case of a touch screen you might want to identify with @media(hover:on-demand), which identifies devices that might not have a literal hover, but can activate a hover state with a long press. Finally there’s the @media(hover:none) for devices that have no hovering ability in the pointing system.

In addition to these, there are the “light-level” environmental features. These features give you the ability to serve rules based on the light levels of the environment the user is it. These can be passed the values “dim” for low light environments, “normal” for an ideal environment, and “washed” for a bright environment. This medium query gives you the ability to adjust the color pallet of a site based on the users ambient light. This has huge potential for sites that related to brick and mortar retail locations. Users might try looking to your website to find a store location while walking around on a sunny day and would benefit from high contrast on the screen.

These examples relate to context awareness within the device itself. Unfortunately these features are all parts of the CSS4 specification and it will likely be years before they can start being used. While level 4 media queries can provide some helpful tools in drafting a site against a users device, what if we could gather data from outside of the device and could serve users needs more directly.

Using JavaScript for context awareness.

Let’s say you sell drinks. If it’s hot out, you might want to advertise iced drinks on the homepage, and if it was cold you might want to promote hot drinks. With a little JavaScript, we can create an element that does exactly this. We simply access the devices built in GPS coordinates, available to the browser and using the Forecast API, we can find the weather data for the user’s location.

See the Pen fHpqv by Matthew Carver (@matthewcarver) on CodePen.

Let’s say we want to make this a little more complicated and if the user is visiting at night promote tea but if they are visiting in the morning, we can promote coffee. We can change the text based on the time of day.

See the Pen EbAax by Matthew Carver (@matthewcarver) on CodePen.

While these examples show some simple prototype level examples, there can be a huge value in internalizing a user’s browser. By using this with a social sign in capability, you could rearrange an entire front end of a website based on what the user is most likely interested in.

Say for instance you have a user who is logged in with Facebook. You could potentially access their relationship status to promote gift ideas for their significant other near Valentines day or if they are single promote a chocolate bar to treat themselves to.

While these ideas are currently nothing new for sites like Amazon or big box stores like Target, with a little foresight and some strategic technical thinking, it’s not hard to create a website that is responsive to not only the user’s device screen size, but also the user themselves. Contextual awareness is something that applications have been going for a long time, but somehow it’s seemed to have slipped past

With screen real estate diminishing as mobile and tablet take a bigger and bigger share of the total internet traffic, patience for wasted space is at an all time low. There is near limitless potential for contextual awareness in a website’s front-end. While accessing Apple’s M7 processor in a website currently isn’t possible, we can glean enough data from a user’s browser to create more impactful experiences every time we design a website.