4. MOBILE HCI Syllabus: Mobile Ecosystem: Platforms, Application frameworks - Types of Mobile Applications: Widgets, Applications, Games - Mobile Information Architecture, Mobile 2.0, Mobile Design: Elements of Mobile Design, Tools. Topics 4.1 Mobile Ecosystem: o Platforms o Application frameworks 4.2 Types of Mobile Applications: o Widgets o Applications o Games 4.3 Mobile Information Architecture 4.4 Mobile 2.0 4.5 Mobile Design: o Elements of Mobile Design o Tools 4.1 Mobile Ecosystem: Mobile is an entirely unique ecosystem and like the Internet, it is made up of many different parts that must all work seamlessly together. If the Internet is a cloud, then the mobile ecosystem would be the atmosphere,made up of many clouds. The internet is just one of these clouds, though a very large one. Fig 4.1 Layers of Mobile Ecosystem Each layer is reliant on the others to create a seamless, end-to-end experience. Although not every piece of the puzzle is included in every mobile product and service, for the majority of the time, they seem to add 1 complexity to our work, regardless of whetherwe expressly put them there. The following sections expand on each of these layers and the roles they play in themobile ecosystem. 4.1.1 Platforms A mobile platform’s primary duty is to provide access to the devices. To run software and services on each of these devices, we need a platform, or a core programminglanguage in which all of the software is written. Like all software platforms, these aresplit into three categories: licensed, proprietary, and open source. 4.1.1.1 Licensed Licensed platforms are sold to device makers for nonexclusive distribution on devices. The goal is to create a common platform of development Application Programming Interfaces (APIs) that work similarly across multiple devices with the least possible effort required to adapt for device differences, although this is hardly reality. Following are the licensed platforms: i. Java Micro Edition (Java ME) Formerly known as J2ME, Java ME is by far the most predominant software platform of any kind in the mobile ecosystem. It is a licensed subset of the Java platform and provides a collection of Java APIs for the development of software for resourceconstrained devices such as phones. ii. Binary Runtime Environment for Wireless (BREW) BREW is a licensed platform created by Qualcomm for mobile devices, mostly for the U.S. market. It is an interface-independent platform that runs a variety of application frameworks, such as C/C++, Java, and Flash Lite. iii. Windows Mobile Windows Mobile is a licensable and compact version of the Windows operating system, combined with a suite of basic applications for mobile devices that is based on the Microsoft Win32 API. iv. LiMo LiMo is a Linux-based mobile platform created by the LiMo Foundation. Although Linux is open source, LiMo is a licensed mobile platform used for mobile devices. LiMo includes SDKs for creating Java, native, or mobile web applications using the WebKit browser framework. 4.1.1.2 Proprietary Proprietary platforms are designed and developed by device makers for use on theirdevices. They are not available for use by competing device makers. These include: i. Palm Palm uses three different proprietary platforms. Their first and most recognizable is the Palm OS platform based on the C/C++ programming language; this was initially developed for their Palm Pilot line, but is now used in low-end smartphones such as the Centro line. As Palm moved into higher-end smartphones, they started using the Windows Mobile-based platform for devices like the Treo line. The most recent platform is called webOS, is based on the WebKit browser framework, and is used in the Pre line. ii. BlackBerry Research in Motion maintains their own proprietary Java-based platform, used exclusively by their BlackBerry devices. iii. iPhone Apple uses a proprietary version of Mac OS X as a platform for their iPhone andiPod touch line of devices, which is based on Unix. 4.1.1.3 Open Source Open source platforms are mobile platforms that are freely available for users to download, alter, and edit. 2 They are newer and slightly controversial, but increasingly gaining traction with device makers and developers. Example: Android. It is developed by the Open Handset Alliance, which is spearheaded by Google. The Alliance seeks to develop an open source mobile platform based on the Java programming language. 4.1.2 Application Frameworks Application frameworks are used to create applications, such as a game, a web browser, a camera, or media player. Although the frameworks are well standardized, the devices are not. The largest challenge of deploying applications is knowing the specific device attributes and capabilities. Example: For creating an application using the Java ME application framework, we need to know what version of Java ME the device supports, the screen dimensions, the processor power, the graphics capabilities, the number of buttons it has, and how the buttons are oriented. Multiply that by just a few additional handsets and we have hundreds of variables to consider when building an application. Multiply it by the most popular handsets in a single market and we can easily have a thousand variables, quickly doomingour application’s design or development. Although mobile applications can typically provide an excellent user experience, it almost always comes at a fantastic development cost, making it nearly impossible to create a scalable product that could potentially create a positive return on investment. A common alternative these days is creating applications for only one platform, such as the iPhone or Android. By minimizing the number of platforms the developer has to support and utilizing modern application frameworks, the time and cost of creation go down significantly. The mobile web browser is an application that renders content that is device-, platform-, and operating-systemindependent. The web browser knows its limitations, enabling content to scale gracefully across multiple screen sizes. However, like all applications, mobile web browsers suffer from many of the same device fragmentation problems. When a device is sold to an operator, it is provisioned (customized) to their requirements. This means the operators will often put customized applications on each of the devices sold. With the example of the RAZR, every operator had it and every operator put a different web browser on it. To make matters worse, the RAZRs, like most phones, are not field-refreshable, meaning that you can’t update the software, upgrade the applications, or eliminate bugs. For example, if a device manufacturer makes a device called the MDv1, they must strike a deal with an operator if they want to preload an operator store application, a different web browser, and bowling game. The device is sold as the MDv1.1. The operator sells the devices, or worse, gives them away for free. A couple hundred thousand of them go out into the marketplace before a glitch in the hardware is detected, such as dropped calls. Because the device cannot be upgraded by cable or over the air, the operator stops selling the MDv1.1, but seeing that they have a hit, they quickly replace it with the MDv1.1.1. The whole process is repeated as it is provisioned to each operator. Suddenly, there is an MDv1.2, an MDv1.3, an MDv1.4, and so on. Then we have the next generations—the MDv1.2.1, the MDv1.3.1, the MDv1.4.1, and so on, spreading like a virus. This is essentially what causes device fragmentation, making application development a costly and timely endeavor. 4.2 Types of Mobile Applications: 4.2.1 Mobile Web Widgets Largely in response to the poor experience provided by the mobile web over the years, there has been a growing movement to establish mobile widget frameworks and platforms. 3 For years the mobile web user experience was severely underutilized and failed to gain traction in the market, so several operators, device makers, and publishers began creating widget platforms to counter the mobile web’s weaknesses. According to Webster’s Dictionary “Mobile Widget is a component of a user interface that operates in a particular way”. Wikipedia defines a web widget this way: “A portable chunk of code that can be installed and executed within any separate HTML based web page by an end user without requiring additional compilation”. Between these two definitions is a better answer: “A mobile web widget is a standalone chunk of HTML-based code that is executed by the end user in a particular way”. Basically, mobile web widgets are small web applications that can’t run by themselves; they need to be executed on top of something else. Opera Widgets, Nokia Web RunTime (WRT), Yahoo! Blueprint, and Adobe Flash Lite are all examples of widget platforms that work on a number of mobile handsets. Using a basic knowledge of HTML, we can create compelling user experiences that tap into device features and, in many cases, can run while the device is offline. Widgets, however, are not to be confused with the utility application context, a user experience designed around short, task-based operations. Pros and Cons of Mobile Web Widgets Pros The pros of mobile web widgets are: • They are easy to create, using basic HTML, CSS, and JavaScript knowledge. • They can be simple to deploy across multiple handsets. • They offer an improved user experience and a richer design, tapping into device features and offline use. Cons The cons of mobile web widgets are: • They typically require a compatible widget platform to be installed on the device. • They cannot run in any mobile web browser. • They require learning additional proprietary, non-web-standard techniques. 4.2.2 Mobile Web Applications Mobile web applications are mobile applications that do not need to be installed or compiled on the target device. Using XHTML, CSS, and JavaScript, they are able toprovide an application-like experience to the end user while running in any mobile web browser. Web applications allow users to interact with content in real time, where a click or touch performs an action within the current view. The history of how mobile web applications came to be so commonplace is interesting. Shortly after the explosion of Web 2.0, web applications like Facebook, Flickr, and Google Reader hit desktop browsers, and there was discussion of how to bring those same web applications to mobile devices. The Web 2.0 movement brought user-centered design principles to the desktop web, and those same principles were sorely needed in the mobile web space as well. The challenge, as always, was device fragmentation. The mobile browsers were years behind the desktop browsers, making it nearly impossible for a mobile device to render a comparable experience. While XHTML support had become fairly commonplace across devices, the rendering of CSS2 was wildly inconsistent, and support for JavaScript, necessary or simple DHTML, and Ajax was completely nonexistent. To make matters worse, the perceived market demand for mobile web applications was not seen as a priority with many operators and device makers. It was the classic chicken or-the-egg scenario. What had to come first, market demand to drive browser innovation or optimized content to drive the market? 4 With the introduction of the first iPhone, we saw a change across the board. Using WebKit, the iPhone could render web applications not optimized for mobile devices as perfectly usable, including DHTML- and Ajax-powered content. Developers quickly got on board, creating mobile web applications optimized mostly for the iPhone. The combination of a high-profile device with an incredibly powerful mobile web browser and a quickly increasing catalog of nicely optimized experiences created the perfect storm the community had been waiting for. Pros and Cons of Mobile Web Applications Pros The pros of mobile web applications are: • They are easy to create, using basic HTML, CSS, and JavaScript knowledge. • They are simple to deploy across multiple handsets. • They offer a better user experience and a rich design, tapping into device features and offline use. • Content is accessible on any mobile web browser. Cons The cons of mobile web applications are: • The optimal experience might not be available on all handsets. • They can be challenging (but not impossible) to support across multiple devices. • They don’t always support native application features, like offline mode, locationlookup, filesystem access, camera, and so on. Native Applications The next mobile application medium is the oldest and the most common; it is referredto as native applications. These applications actually should be called “platform applications,” as they have to be developed and compiled for each mobile platform. These native or platform applications are built specifically for devices that run the platform in question. The most common of all platforms is Java ME (formerly J2ME). A device written as a Java ME MIDlet should work on the vast majority of feature phones sold around the world. The reality is that even an application written as a Java ME MIDlet still requires some adaptation and testing for each device it is deployed on. In the smartphone space, the platform SDKs get much more specific. Although many smartphones are also powered by Java, an operating system layer and APIs added to allow developers to more easily offload complex tasks to the API instead of writing methods from scratch. In addition to Java, other smartphone programming languages include versions of C, C++, and Objective-C. Creating a platform application means deciding which devices to target, having a means of testing and certification, and a method to distribute the application to users. Thevast majority of platform applications are certified, sold, and distributed either through an operator portal or an app store. It is possible to create a Java ME MIDlet application and publish it for free on the Web, but it is rarely done. Pros and Cons of Native Applications Pros The pros of native applications include: • They offer a best-in-class user experience, offering a rich design and tapping into device features and offline use. • They are relatively simple to develop for a single platform. • You can charge for applications. 5 Cons The cons of native applications include: • They cannot be easily ported to other mobile platforms. • Developing, testing, and supporting multiple device platforms is incredibly costly. • They require certification and distribution from a third party that you have no control over. • They require you to share revenue with the one or more third parties. 4.2.3 Games The final mobile medium is games, the most popular of all media available to mobile devices. Technically games are really just native applications that use the similar platform. They are different from native applications for two reasons: they cannot be easily duplicated with web technologies, and porting them to multiple mobile platforms is a bit easier than typical platform-based applications. Adobe’s Flash and the SVG (scalable vector graphics) standard are the only way for the mobile games on the Web now, and will likely be how it is done on mobile devices in the future, the primary obstacle being the performance of the device in dealing with vector graphics. The game mechanics are the only thing that needs to adapt to the various platforms. Like in console gaming, there are a great number of mobile game portingshops that can quickly take a game written in one language and port it to another. Mobile games stand apart from all other application genres—their capability to be unique and difficult to duplicate in anotherapplication type, though the game itself is relatively easy to port. Pros and Cons of game applications Pros • They provide a simple and easy way to create an immersive experience. • They can be ported to multiple devices relatively easily. Cons The cons of game applications are: • They can be costly to develop as an original game title. • They cannot easily be ported to the mobile web. 4.3 Mobile Information Architecture: What Is Information Architecture? • The structural design of shared information environments. • The combination of organizations, labeling, search, and navigation systems within websites and intranets. • The art and science of shaping information products and experiences to support usability and findability. • An emerging discipline and community of practice focused on bringing principles of design and architecture to the digital landscape Information architecture, describe several unique disciplines, including the following: i. Information architecture The organization of data within an informational space. In other words, how the user will get to information or perform tasks within a website or application. ii. Interaction design The design of how the user can participate with the information present, either in a direct or indirect way, meaning how the user will interact with the website of application to create a more meaningful experience and accomplish the goals. 6 iii. Information design The visual layout of information or how the user will assess meaning and direction given the information presented to him. iv. Navigation design The words used to describe information spaces; the labels or triggers used to tell the users what something is and to establish the expectation of what they will find. v. Interface design The design of the visual paradigms used to create action or understanding. Information architecture composes the core of the user experience. The role of information architecture is played by a variety of people, from product managers to designers and even developers. Words like intuitive, simple, findable, usable, or the executive favorite—easy to-use—all describe the role that information architects play in creating digital experiences. The visual design of product, what frameworks we use, and how it is developed are integral to the success of any product, but the information architecture stands apart as being the most crucial element of the product. It isthe first line of scrimmage—the user’s first impression of product. Even if we have the best design, the best code, and the best backend service, if the user cannot figure out how to use it, she will fail—and so will the product. When thinking about mobile information architecture, we want to keep it as simple as possible. Support the defined goals. If something doesn’t support the defined goals, lose it. Go back to user goals and needs, and identify the tasks that map to them. Find those needs and fill them. Ask yourself: what need does my application fill? What are people trying to do here? What is their primary goal? Once we understand that, it is a simple process of reverse engineering the path from where they want to be to where they are starting. Cut out everything else—the site or application doesn’t need it. For example, to get some news and information on a mobile device, we need to first ask what the goal is. What is the need we are trying to fill? Then we need to apply context. Where are the users? What are they doing? Are they waiting for the bus? Do they have only a minute to spare? Or, do they have five minutes to spare? With these answers, we get our information architecture. Keep all labels short and descriptive, and never try to be clever with the words we use to evoke action. The worst sin is to introduce branding or marketing into information architecture; this will just serve to confuse and distract users. Executives love to use the words they use internally to external communications on websites and applications, but these words have no meaning outside of company walls. Based on what we know from web design, we should use simple, direct terms for navigating around pages rather than overly clever terms. That latter typically result in confused visitors who struggle to find the content they are looking for. When that happens, they will go elsewhere to look for the information they want. So, if we apply these same mistakes to a constrained device like mobile, then we end up adding confusion to the user experience at a higher magnitude than the Web. 1. Site Maps The first deliverable we use to define mobile information architecture is the site map. Site maps are a classic information architecture deliverable. They visually represent the relationship of content to other content and provide a map for how the user will travel through the informational space, as shown in Figure 4.2.Mobile site maps aren’t that dissimilar from site maps we might use on the Web. But there are a few tips specific to mobile that we want to consider. 7 Fig 4.2 An Example of site map In Figure 4.3, we can see a poorly designed mobile information architecture that too closely mimics its desktop cousin; it was not designed with the mobile user in mind. But in mobile, we cannot make this assumption. In the mobile context, tasks are short and users have limited time to perform them. And with mobile websites, we can’t assume that the users have access to a reliable broadband connection that allows them to quickly go back to the previous page. The users pay cash for viewing the wrong page by mistake, they pay to again download the page they started from: we can’t assume that pages will be cached properly. Therefore, the advice is to limit users’ options. 8 Fig 4.3 An example of a bad Mobile Information Architecture that was designed with desktop users in mind rather than mobile users After the users have selected a path, it isn’t always clear whether they are getting to where they need to be. Information-heavy sites and applications often employ nested or drill-down architectures, forcing the user to select category after category to get to their target. To reduce risking the user’s time and money, we want to make sure we present enough information for the user to wade through information architecture successfully. On the Web, we take these risks very lightly, but with mobile, we must give our users a helping hand. We do this by teasing content within each category— that is, providing at least one content item per category. The challenge with ringtone sites is you have a lot of items, grouped by artist, album, genre, and so on. The user starts with a goal like “I want a new ringtone” and finds an item that suits his taste within a catalog of tens of thousands of items. In order to make sense of a vast inventory of content, we have to group, subgroup, and sometimes even subgroup again, creating a drill-down path for the user to browse. Though on paper this might seem like a decent solution, once you populate an application with content, the dreaded “Page 1 of 157” appears. Users would flip through a few pages of content, then give up or go back and visit another area. We could see a direct relationship to the number of pages viewed to sales—essentially, more pages loaded meant fewer sales. Then we realized we could take the most popular item based on sales and place it as the first item in any list, which is teasing the content. 9 In Figure 4.4, we can see in a constrained screen that teasing the first few items of the page provides the user with a much more intuitive interface, immediately indicating what type of content the user can expect. We immediately saw that users were finding content more quickly, driving up our sales. Fig 4.4 Teasing content to confirm the user’s expectations of the content within 2. Clickstreams Clickstream is a term used for showing the behavior on websites, displaying the orderin which users travel through a site’s information architecture, usually based on data gathered from server logs. Clickstreams are usually historical, used to see the flaws in information architecture, typically using heat-mapping or simple percentages to show where users are going. They are a useful tool for re architecting large websites. However, information architecture in mobile is more like software than it is the Web, meaning that we create clickstreams in the beginning, not the end. This maps the idealpath the user will take to perform common tasks. Being able to visually lay out the path users will take gives a holistic or bird’s-eye view of mobile information architecture, just as a road map does. When we can see all the paths next to each other and take a step back, we start to see shortcuts and how we can get users to their goal faster or easier, as shown in Figure 4.5. 10 Fig 4.5An example clickstream for an iPhone web application The esoteric diagram shown in Figure 4.6, which is made up of boxes and diamonds that look more like circuit board diagrams than an information architecture. A good architect’s job is to create a map of user goals, not map out every technical contingency or edge case. Too often, process flows go down a slippery slope of adding every project requirement, bogging down the user experience with unnecessary distractions, rather than focusing on streamlining the experience. In mobile, our job is to keep it as simple as possible. We need to have an unwavering focus on defining an excellent user experience first and foremost. 11 Fig 4.6 An example process flow diagram 3. Wireframes The next information architecture tool is wireframes. Wireframes are a way to lay out information on the page, also referred to as information design. Site maps show how our content is organized in our informational space; wireframes show how the user will directly interact with it. Wireframes are like the peanut butter to the site map jelly in our information architecture sandwich. Wireframes serve to make information space tangible and useful. But the purpose of wireframes is not just to provide a visual for our site map; they alsoserve to separate layout from visual design, defining how the user will interact with the experience. How do we lay out our navigation? What visual or interaction metaphors will we use to evoke action? What are the best ways to communicate and show information in the assumed context of the user? These questions and many more are answered with wireframes. The challenge is that a diagram on a piece of paper doesn’t go a long way toward describing how the interactions will work. Most common are “in-place” interactions or areas where the user can interact with an element without leaving the page. This can be done withAjax or a little show/hide JavaScript. These interactions can include large amounts of annotation, describing each content area in as much length as we can fit in the margins of the page, as shown in Figure 4.7. 12 Fig 4.7 Using annotations to indicate the desired interactions of the site or application Often we get an idea that we think is brilliant, but once we say it out loud, it just sounds absurd. In mobile, it is this kind of feedback, using wireframes as the key deliverable that turns good ideas into excellent mobile products. 4. Prototyping Wireframes lack the capability to communicate more complex, often in-place, interactions of mobile experiences. This is where prototypes come in. Prototypes might sound like a scary (or costly) step in the process. Some view them as redundant or too timeconsuming, preferring to jump in and start coding things. But as with wireframes, each product we’ve built out some sort of prototype has saved both time and money. The following sections discuss some ways to do some simple and fast mobile prototyping. 5. Paper prototypes The most basic level we have is paper prototyping: taking our printed-out wireframes or even drawings of our interface and putting them in front of people. Create a basic script of tasks (hopefully based on either context or user input) and ask users to perform them, pointing to what they would do. We act as the device, changing the screens for them. For paper prototypes, try using small blank note cards, and for lower-end devices, use business card-sized paper. The size matters and we’ll learn as much from how the user manages working with small media as we will what information is actually on it. 6. Context prototype The next step is creating a context prototype. Take a higher-end device that enables to load full-screen images on it. Take wireframes or sketches and load them onto the device, sized to fill the device screen. Leave the office. 13 Go for a walk down to nearest café. Or get on a bus or a train. As you are traveling about, pull out your device and start looking at the interface in the various contexts you find yourself currently in. Pay particular attention to what you are thinking and your physical behavior while you are using your interface and then write it down. If you are brave and don’t have strict nondisclosure issues, ask the people around you to use it, too. Try to keep an eye on a clock to determine how long the average session is. 7. HTML prototypes The third step is creating a lightweight, semifunctional static prototype using XHTML,CSS, and JavaScript, if available. This is a prototype that we can actually load onto a device and produce the nearest experience to the final product, but with static dummycontent and data. It takes a little extra time, but it is worth the effort. With a static XHTML prototype, you use all the device metaphors of navigation, you see how much content will really be displayed on screen (it is always less than you expect), and you have to deal with slow load times and network latency. In short, you will feel the same pains your user will go through. Whatever route you wish to take, building a mobile prototype takes you one very big leap forward to creating a better mobile experience. Different Information Architecture for Different Devices Depending on which devices you need to support, mobile wireframes can range from the very basic to the complex. On the higher-end devices with larger screens, we might be inclined to add more interactions, buttons, and other clutter to the screen, but this would be a mistake. Just because the user might have a more advanced phone, that doesn’t mean that he is giving you license to pack his screen with as much informationas you can muster. The motivations, goals, and how users will interact with a mobile experience are the same at the low end as they are on a high-end device. For the latter, you just have better tools to express the content. You can learn a lot from designing for the lower end first. The greatest challenge in creating valuable experiences is knowing when to lose what you don’t need. You don’t have a choice on lower-end devices—it must be simple. When designing for both, it is best to try and to keep your information architecture as close to each other as possible without sacrificing the user experience. They say that simple design is the hardest design, and this principle certainly is true when designing information architecture for mobile devices. The Design Myth A little secret about interactive design is that people don’t respond to the visual aestheticas much. What colors you use, whether you use square or rounded corners, or, gradients or flat backgrounds, helps build first impressions, but it doesn’t do too much to improve the user’s experience. Users appreciategood design, but they are quickly indifferent about the visual aesthetic and move almostimmediately to the layout (information design), what things are called (taxonomy), thefindability of content, and how intuitive it is to perform tasks. These are all facets ofinformation architecture. Just look at one of the top-selling iPhone Twitter applications, Tweetie. Many consider Tweetie to be a “well-designed” application, but because it is built from the same API as all other iPhone applications, at first glance there is little that is actually visually distinctive between this and other applications. What makes this application “well designed” is how the content is applied to the context of the user—in other words, the mobile information architecture. The point is great information design is often mistaken for great visual design. Most non–information architects almost always do information architecture in someform or another; often, they don’t even know they are doing it. They might do a few wireframes, or maybe a site map. Sometimes designers will jump in and incorporate information architecture deliverables directly into their designs. By not focusing on the information architecture exclusively from the start, we risk confusing our disciplines, our deliverables, and ultimately our direction. The more time we spend focusing on just information architecture, the faster and less costly the project will be. 4.4 Mobile 2.0: 14 What Is Mobile 2.0? Mobile 2.0, refers to a perceived next generation of mobile internet services that leverage the social web, or what some call Web 2.0. The social web includes social networking sites and wikis that emphasise collaboration and sharing amongst users. Mobile 2.0: The Convergence of the Web and Mobile It is obvious that in the minds of many, Mobile 2.0 is the Web. At this point, the mobile web has always been viewed as a second-class citizen within the mobile ecosystem, for many reasons, as discussed later. Mobile is already a medium, but the consensus is that by leveraging the power of theWeb, integrating web services into the mobile medium is the future of mobile development. When the iPhone exploded onto the scene, it increased the usage of the mobile web by its users to levels never seen before. The spur of new mobile web apps created just for the iPhone doubled the number of mobile websites available in under a year. If Web 2.0 taught us that the Web is the platform, then Mobile 2.0 tells us that mobile will be the primary context in which we leverage the Web in the future. The Mobile Web Browser As the Next Killer App If the future of mobile is the Web, then it only makes sense that the mobile web browser is the next killer app of mobile. This is something we saw confirmed with WebKitin the iPhone and later in Android. However, of particular concern is how device fragmentation factors into mobile browsers. For example, how can we expect developers to support more than 30 different mobile browsers? A fellow panelist from the Mozilla Minimo project offered a potential solution in consolidation—that we will see only a few mobile browsers in the future; specifically, browsers built on Mozilla, Opera, Internet Explorer, and WebKit technologies. The line between smartphone and feature phone seems to be going away, so this prediction is fairly accurate. But the single biggest challenge in mobile remains device fragmentation. The mobile browser enables us to penetrate the problem by not having our content locked so specifically to the device abilities, screen size, and form factor, but device fragmentation still causes old, outdated browsers to remain in the market long after they should be put out to pasture. What appears to be solving browser fragmentation is actually the iPhone. The MobileSafari browser included with the iPhone provided such an excellent web experience on a mobile device that it drove use of the mobile web to huge levels, which means big profits for the operators. This also means that the mobile web is no longer a second class citizen. In the post-iPhone market, all new devices are judged by the quality of their mobile web browser. Operators know it and therefore are demanding better browsers from device makers and browser makers. Mobile Web Applications Are the Future Creating mobile web applications instead of mobile software applications has remained an area of significant motivation and interest. The mobile community is looking at theWeb 2.0 revolution for inspiration, being able to create products and get them to market quickly and at little cost. They see the success of small iterative development cycles and want to apply this to mobile development, something that is not that feasible in the traditional mobile ecosystem. Developers have been keen for years to shift away from the costly mobile applications that are difficult to publish through the mobile service provider, require massive testing cycles and costly porting to multiple devices, and can easily miss the mark with users after loads of money have been dumped into them.The iPhone App Store and the other mobile device marketplaces have made it far easier to publish and sell, but developers still have to face difficult approval processes, dealing with operator and device maker terms and porting challenges. 15 Mobile software has two fundamental problems that mobile web applications solve. The first is forcing users through a single marketplace. We know from years of this model that an app sold through a marketplace can earn huge profits if promoted correctly. Being promoted correctly is the key phrase. The companies that know how to work the system are the ones that get the big prizes, making it increasingly hard for the small developer to see any kind of success. But the mobile web provides any size of developer with the ability to promote and distribute their app on their terms, building a relationship directly with their customers and not by proxy. The second problem is the ability to update your application. It is certainly possible on modern marketplaces like the App Store, but we are still years from that being the norm. Mobile web apps enable you to make sure that you never ship a broken app, or if yourapp breaks in the future due to a new device, to be able to fix it the same day the device hits the street. This flexibility isn’t possible in the downloaded app market. JavaScript Is the Next Frontier If we are going to provide mobile web applications, we have to have a mobile web browser that supports Ajax, or, XMLHttpRequest. It makes a lot of those cool interactions in web browser work via the capability to load content asynchronously into browser view. But it isn’t just Ajax; it is JavaScript, a web technology that has largely been ignored with most mobile web browsers. Ajax is great, but just being able to do a little show/ hide or change a style after click or touch it goes a long way toward improving the user experience. This is probably where mobile web browsers fall behind desktop browsers the most. Because they both support XHTML and CSS relatively well, JavaScript has been a nogoin mobile for years. In order for mobile web apps to rival native applications, wehave to support some JavaScript. Modern mobile browsers have made much progress over the last few years, but there is still plenty of work to be done. For example, accessing the device capabilities like the phone book or filesystem with JavaScript doesn’t work in a consistent way. These problems still need to be solved in order to truly reap the benefits of the Web. Rich interactions kill battery life JavaScript and Ajax have been ignored because using an Ajax-based web application on phone can drain battery at a rate of four to five times normal power consumption.The two most prevalent reasons are: • JavaScript consumes more processor power and therefore more battery life. • Ajax apps fetch more data from the network, meaning more use of the radio and more battery life. Unless we are in the habit of carrying around a bunch of extra batteries, expect to charge phone every hour or two as a penalty for using the modern mobile web. Apple and the open source WebKit browser have made huge strides by releasing aJavaScript engine that is incredibly efficient on mobile devices, though the other big mobile browser technologies aren’t far behind. This problem is going away quickly as the mobile browsers get better, batteries improve efficiency, and devices get more powerful. The Mobile User Experience Is Awful Traditionally, the user experience available in the mobile web has been like using a website from 1995: mostly text-based, difficult to use, and ugly as sin. This isn’t to say that the user experience of mobile applications has been much better, but it used to be that if wanted a good experience, build a native app. Mobile user experience was largely ignored for close to a decade. People in mobile treat the user experience like a chicken-and-egg scenario: bad input/output of the user experience prevents adoption, but designing a shiny user experience with bells and whistles will bring them in droves. Device APIs usually force to use their models of user experience, meaning that wehave to work in the constraints of the API. This is good for creating consistent experiences for that platform, but these experiences don’t 16 translate to others. For example, we cannot take an iPhone app design and put it on an Android device. The user experience for these devices is similar but still remains different. The beauty of the Web, literally, is that we can design whatever experience we want, for better or worse. We are in control of the presentation and can establish own visual metaphors. The problem has been that traditionally complex (which often equates to good) user experiences haven’t been possible on mobile devices. Modern mobile web browsers, as they come closer to their desktop counterparts, remove this distinction, giving us the same canvas on mobile devices that we have for the desktop. This means that creating mobile experiences just got a whole lot easier. It also means we can have a consistent user experience across multiple mediums. Mobile Widgets Are the Next Big Thing The challenges with the mobile web is to create a series of “small webs” targeted at a specific user or task. Though I couldn’t figure out the problem being solved with these widgets, Carrier Is the New “C” Word It is clear that one of the key drivers of Mobile 2.0 and the focus on the mobile web is to find a way to build a business that doesn’t rely on carrier control. Mobile Needs to Check Its Ego The mobile community and the web communityhave treated each other almost like rivals. It is the mobile camp that needs to check their egos at the door and get into the game, before they learn that all the rules have changed. On the mobile side, we have some incredibly intelligent people who have been innovating amazing products under insane constraints for years. On the web side, we have creative amateurs who have helped build a community and ecosystem out of passion and an openness to share information. The web guys want to get into the game and move the medium forward, partly out of desire open up a new market for themselves, but mostly out of passion for all things interactive. But, to the mobile community, they are seen as a threat to expertise. On the other hand, to the web community, the mobile guys come off as overly protective, territorial, selfish, and often snobbish or egotistical, effectively saying, “Go away.” Unless the mobile community comes together with the web community by sharing information, experience, and guidance, one day they will find that their experience has become obsolete. In return, the web guys will share their enthusiasm, willingness to learn, and passion that many in mobile development have forgot. It’s that one principle of Web 2.0 that the mobile community has left out: harnessing collective intelligence. The Web and the mobile community are reaching a point where the two worlds can no longer afford not to be working together, sharing what they know and harnessing the collective intelligence of both media. We Are Creators, Not Consumers The final principle of Mobile 2.0 is recognizing that we are in a new age of consumerism. Yesterday’s consumer does not look anything like today’s consumer. The people of today’s market don’t view themselves as consumers, but rather as creators. The web is about content. Sure, there are programming languages, APIs, and other technical underpinnings, but what we do when we open a web browser? We read. Our primary task online is to read, to gain information. During the early days of the Web, it took tools and know-how in order to publish to the Web. But early in the Web2.0 evolution, we saw a rise in tools that allowed us to publish to the Web easily, giving individuals a voice online, with a massive audience. This democratization of the Web took many forms that some call “social media,” like blogging, social networks, media sharing, microblogging, and lifestreams. Although social media may have many facets, they all share the same goal: to empower normal, everyday people to become creators and publishers of content. It started with the written word, then music, then photos, and more recently video was added. Entire markets have been 17 created to provide today’s consumer with gadgets, software, and web services to record and publish content so that we can share it with our friends and loved ones. At the center of this revolution in publishing is the mobile device. As networked portable devices become more powerful, allowing us to capture, record, and share content in the moment, we are able to add a new kind of context to content—the likes of which we haven’t seen since satellite television. Now you can share any moment with any group of people in real time. Think about how powerful a concept that is! It could change entire cultures. Tony Fish, coauthor of Mobile Web 2.0 (futuretext), says: When everyone has the tools to create content, in addition to zero-cost publishing, we do not consume content, we create it. The early “Web 1.0” days clearly changed how business is done, because businesses are the primary consumer of desktop computers. It probably is no coincidence that Web 2.0 occurred around the same time that laptop computers became affordable for the average person, making the Web a more personal medium. With Mobile 2.0, the personal relevance of the content matches how personal the device is and how personally it applies to our everyday situations or our context. 4.5 Mobile Design: 4.5.1 The Elements of Mobile Design The good design requires three abilities: the first is a natural gift for being able to see visually how something should look that produces a desired emotion with the target audience. The second is the ability to manifest that vision into something for others to see, use, or participate in. The third is knowing how to utilize the medium to achieve our design goals. To think like Mobile designerinvolves knowing the six elements of mobile design that we need to consider, starting with the context and layering in visual elements or laying out content to achieve the design goal. Then, we need to understand how to use the specific tools to create mobile design, and finally, need to understand the specific design considerations of the mobile medium. 1. Context Context is core to the mobile experience. As the designer, it isthe job to make sure that the user can figure out how to address context using app. Some context based questions: • Who are the users? What do you know about them? What type of behavior can you assume or predict about the users? • What is happening? What are the circumstances in which the users will best absorb the content you intend to present? • When will they interact? Are they at home and have large amounts of time? Are they at work where they have short periods of time? Will they have idle periods of time while waiting for a train, for example? • Where are the users? Are they in a public space or a private space? Are they inside or outside? Is it day or is it night? • Why will they use your app? What value will they gain from your content or services in their present situation? • How are they using their mobile device? Is it held in their hand or in their pocket? How are they holding it? Open or closed? Portrait or landscape? The answers to these questions will greatly affect the course of design. Treat these questions as a checklist to the design from start to finish. They can provide not only great inspiration for design challenges, but justification for design decisions later. 2. Message 18 Another design element is message, or what we are trying to say about site or application visually. One might also call it the “branding,”.The message is the overall mental impression we create explicitly through visual design. Branding shouldn’t be confused with messaging. Branding is the impression company name and logo gives— essentially, reputation. Branding serves to reinforce the message with authority, not deliver it. In mobile, the opportunities for branding are limited, but the need for messaging is great. With such limited real estate, the users don’t care about brand, but they will care about the messaging, asking themselves questions like, “What can this do for me?” or “Why is this important to me?” The approach to the design will define that message and create expectations. A sparse, minimalist design with lots of whitespace will tell the user to expect a focus on content. A “heavy” design with use of dark colors and lots of graphics will tell the user to expect something more immersive. 3. Look and Feel The concept of “look and feel” is an odd one, being subjective and hard to define. Typically, look and feel is used to describe appearance, as in “I want a clean look and feel” or “I want a usable look and feel.” The problem is: as a mobile designer, what does it mean? And how is that different than messaging? Look and feel in a literal sense, as something real and tactile that the users can “look” at, then “feel”—something they can touch or interact with. Look and feel is used to evoke action—how the user will use an interface. Messaging is holistic, as the expectation the users will have about how you will address their context. It is easy to confuse the two, because “feel” can be interpreted to mean our emotional reaction to design and the role of messaging. Establishing a look and feel usually comes from wherever design inspiration comes from. However, personal inspiration can be a hard thing to justify. Therefore we have “design patterns,” or documented solutions to design problems, sometimes referred to as style guides. On large mobile projects or in companies with multiple designers, a style guide or pattern library is crucial, maintaining consistency in the look and feel and reducing the need for each design decision to be justified. Although a lot of elements go into making Apple’s App Store successful, the most important design element is how it looks and feels. Apple includes a robust user interface tool that enables developers to use prebuilt components, supported with detailedHuman Interface Guidelines (or HIG) of how to use them, similar to a pattern library. This means that a developer can just sit down and create an iPhone application that looks like it came from Apple in a matter of minutes. During the App Store submission process, Apple then ensures that the developer uses these tools correctly according to the HIG. The look and feel can either be consistent with the stock user interface elements thatApple provides; they can be customized, often retaining the “spirit” of Apple’s original design; or an entirely new look and feel can be defined—this approach is often used for immersive experiences. The stock user experience that Apple provides is a great example of how look and feel works to supporting messaging. For the end user, the design sends a clear message: by using the same visual interface metaphors that Apple uses throughout the iPhone, we can expect the action, or how this control will behave, but we can also expect the same level of quality. This invokes the message of trust and quality in the application and in the platform as a whole. Apple isn’t the first to use this shared look and feel model in mobile—in fact, it is incredibly common with most smartphone platforms—but they are surely making it incredibly successful, with a massive catalog of apps and the sales to support it. Mobile designers haveto be creative and remember the context. Like in the early days of the Web, people tend to be skeptical about mobile experiences. The modal context of the user—in this case, what device he is using —should be considered during the design, as it will help to establish the user’s expectations of the experience. 4. Layout 19 Layout is an important design element, because it is how the user will visually processthe page, but the structural and visual components of layout often get merged together, creating confusion and making design more difficult to produce. The first time layout should rear its head is during information architecture. Why define the layout before the mobile design? Design is just too subjective of anissue. If we are creating a design for anyone but our self, chances are good that there will be multiple loosely-based-on-experience opinions that will be offered and debated. There is no right answer—only opinions and gut instincts. Plus, in corporate environments we have internal politics we have to consider, where the design opinions of theCEO or Chief Marketing Officer (CMO) might influence a design direction more than, say, the Creative Director or Design Director. By defining design elements like layout prior to actually applying the look and feel, wecan separate the discussion. The majority of comments that reviewers would make were about the layout. They focus on the headers, the navigation, the footer, or how content blocks are laid out, and so on. But their feedback will get muddied with the “look and feel, the colors, and other design elements.” Reviewers do make remarks like “I like the navigation list, but can you make it look more raised?” Most designers don’t hear that; they hear “The navigation isn’t right, do it again.” But, with this kind of feedback, there are two important pieces of information about different types of design. First, there is confirmation that the navigation and layout are correct. Second, there is a question about the “look and feel.” Because designers hear “Do it again,” they typically redo the layout, even though it was actually fine. Creating mobile designs in an environment with multiple reviewers is all about gettingthe right feedback at the right time. Our job is to create a manifestation of a shared vision. Layout is one of the elements we can present early on and discuss independently. People confuse the quality and fidelity of deliverables as design. By keeping it basic, we don’t risk having reviewers confuse professionalism with design. Different layouts for different devices The second part of layout design is how to visually represent content. In mobile design, the primary content element we deal with is the navigation. Whether we are designing a site or app, you need to provide users with methods of performing tasks, navigating to other pages, or reading and interacting with content. This can vary, depending on the devices we support. There are two distinct types of navigation layouts for mobile devices: touch and scroll. With touch, we literally point to where we want to go; therefore, navigation can beanywhere on the screen. But we tend to see most of the primary actions or navigation areas living at the bottom of the screen and secondary actions living at the top of thescreen, with the area in between serving as the content area, like what is shown inFigure 4.8. 20 Fig 4.8 iPhone HIG, showing the layout dimensions of Safari on the iPhone This is the opposite of the scroll navigation type, where the device’s D-pad is used togo left, right, up, or down. When designing for this type of device, the primary and often the secondary actions should live at the top of the screen. This is so the user doesn’t have to press down dozens of times to get to the important stuff. In Figure 4.9, we can actually see by the bold outline that the first item selected on the screen is the link around the logo. Fig 4.9 Example layout of a scroll based application, where the user had to press the D-pad past each link to scroll the page When dealing with scroll navigation, we also have to make the choice of whether to display navigation horizontally or vertically. Visually, horizontally makes a bit more sense, but when we consider that it forces the user to awkwardly move left and right, it can quickly become a bit cumbersome for the user to deal with. There is no right or wrong way to do it, but try and keep it as simple as possible. Fixed versus fluid Another layout consideration is how design will scale as the device orientation changes, for example if the device is rotated from portrait mode to landscape and vice versa. This is typically described as either being fixed (a 21 set number of pixels wide), or fluid (having the ability to scale to the full width of the screen regardless of the device orientation). Orientation switching has become commonplace in mobile devices, and designshould always provide the user with a means to scale the interface to take full advantage of screen real estate. 5. Color The fifth design element, color, is hard to talk about in a black-and-white book. Maybe it is fitting, because it wasn’t that long ago those mobile screens were available only in black and white (well, technically, it was black on a green screen). These days, we havenearly the entire spectrum of colors to choose from for mobile designs. The most common obstacle we encounter when dealing with color is mobile screens,which come in a number of different color or bit depths, meaning the number of bits (binary digits) used to represent the color of a single pixel in a bitmapped image. When complex designs are displayed on different mobile devices, the limited color depth on one device can cause banding, or unwanted posterization in the image. Different devices have different color depths. In Table 4.1, we can see the supported colors and a few example devices. Table 4.1 Supported Colors and Example Devices The psychology of color People respond to different colors differently. It is fairly well known that different colors produce different emotions in people, but surprisingly few talk about it outside of art school. Thinking about the emotions that colors evoke in people is an important aspect of mobile design, which is such a personal medium that tends to be used in personalways. Using the right colors can be useful for delivering the right message and setting expectations. One of the examples used earlier was the ESPN mobile site, which uses a bold red header to create a stark and prominent tone to the design. But what does that say about ESPN? What does it tell the user about the experience? For the purposes of reference, Table 4.2provides some of the characteristics of various colors that naturally evoke certain emotions in people. 22 Table 4.2 Color Characteristics Note what some of the different colors can mean in different cultures. In some cases,the color we use can have opposing meanings in different cultures. This is something to consider when thinking of deploying mobile experience to countries with the highest number of mobile devices, such as China or India. Color palettes Defining color palettes can be useful for maintaining a consistent use of color in mobile design. Color palettes typically consist of a predefined number of colors to use throughout the design. Selecting what colors to use varies from designer to designer, each having different techniques and strategies for deciding on the colors. Three basic ways to define a color palette: i. Sequential 23 In this case, there are primary, secondary, and tertiary colors. Often the primary color is reserved as the “brand” color or the color that most closely resembles the brand’s meaning. The secondary and tertiary colors are often complementary colors that can be selected using a color wheel. ii. Adaptive An adaptive palette is one in which we leverage the most common colors present in a supporting graphic or image. When creating a design that is meant to look native on the device, an adaptive palette can be usedto make sure that colors are consistent with the target mobile platform. iii. Inspired This is a design that is created from the great pieces of design we might see onlineor offline, in which a picture of the design might inspire us. This could be anything from an old poster in an alley, a business card, or some packaging. Like with the adaptive palette, we actually extract the colors from the source image, though we should never ever use the source material in a design. Typography The sixth element of mobile design is typography, which in the past would bring tomind the famous statement by Henry Ford: Any customer can have a car painted any color that he wants so long as it is black. Traditionally in mobile design, you had only one typeface that we could use, and that was the device font. The only control over the presentation was thesize. As devices improved, so did their fonts. Higher-resolution screens allowed for a morerobust catalog of fonts than just the device font. First, let’s understand how mobile screens work. Subpixels and pixel density There seem to be two basic approaches to how type is rendered on mobile screens: using subpixel-based screens or having a greater pixel density or pixels per inch (PPI). A subpixel is the division of each pixel into a red, green, and blue (or RGB) unit at a microscopic level, enabling a greater level of antialiasing for each font character or glyph. The addition of these RGB subpixels enables the eye to see greater variations of gray, creating sharper antialiasing and crisp text. In Figure 4.10, you can see three examples of text rendering. The first line shows a simple black and white example, the second shows text with grayscale antialiasing, and the third line shows how text on a subpixel display would render. Fig 4.10 Different ways text can render on mobile screens The Microsoft Windows Mobile platform uses the subpixel technique with its Clear-Type technology, as shown in Figure 4.11. 24 Fig 4.11 Microsoft Clear-Type using sub pixels to sharp text The second approach is to use a great pixel density, or pixels per inch. We often refer to screens by either their actual physical dimensions or their pixel dimensions, or resolution. The pixel density is determined by dividing the width of the display area in pixels by the width of the display area in inches. So the pixel density for15.4-inchlaptop would be 110 PPI. In comparison, a 1080p HD television has a PPI of 52. As this applies to mobile devices, the higher the density of pixels, the sharper the screenappears to the naked eye. This guideline especially applies to type, meaning that as text is antialiased on a screen with a high density of tiny pixels, the glyph appears sharper to the eye. Some mobile screens have both a high PPI and subpixel technology, though these are unnecessary together. Table 4.3 provides the dimensions and PPI for a few mobile devices. Table 4.3 Dimensions and PPI for some Mobile Devices Type options Fortunately, today’s mobile devices have a few more options than a single typeface, butthe options are still fairly limited. Coming from web design, where we have a dozen or so type options, the limited choices available in mobile design won’t come as a big surprise. Essentially, we have a few variations of serif, sans-serif, and monospace fonts, and depending on the platform, maybe a few custom fonts (Figure). 25 Fig 4.12 Options in Typography increase as the device become more sophisticated Therefore, when creating mobile designs for either web or native experiences, stick with either the default device font, or web-safe fonts—the basic serif variants like Times New Roman and Georgia or sans-serif typefaces like Helvetica, Arial, orVerdana. Font replacement The ability to use typefaces that are not already loaded on the device varies from model to model and chosen platform. Some device APIs will allow to load a typeface into native application. Some mobile web browsers support various forms of fontreplacement; the two most common are sIFR and Cufon. sIFR uses Flash to replaceHTML text with a Flash representation of the text, but the device ofcourse has to support Flash. Cufon uses JavaScript and the canvas element draws the glyphs in the browser, but the device of course needs to support both JavaScript and the canvas element. In addition, the @font-face CSS rule allows for a typeface file to be referenced and loaded into the browser, but a license for web use is usually not granted by type foundries. Readability The most important role of typography in mobile design is to provide the user with excellent readability, or the ability to clearly follow lines of text with the eye and not lose one’s place or become disoriented, as shown in Figure 4.13. This can be done byfollowing these six simple rules: Fig 4.13 Classics, an iPhone application designed with readability and typography in mind i. Use a high-contrast typeface Remember that mobile devices are usually used outside. Having a high-contrast typeface with regard to the background will increase visibility and readability. ii. Use the right typeface The type of typeface you use tells the user what to expect. For example, a sans-serif font is common in navigation or compact areas, whereas serif typefaces come in handy for lengthy or dense content areas. iii. Provide decent leading (rhymes with “heading”) or line spacing Mobile screens are often held 10–12" away from the eye, which can make tracking each line difficult. Increase the leading to avoid having the users lose their place. iv. Leave space on the right and left of each line; don’t crowd the screen 26 Most mobile frameworks give you full access to the screen, meaning that you normally need to provide some spacing between the right and left side of the screen’s edge and text—not much, typically about three to four character widths. v. Generously utilize headings Break the content up in the screen, using text-based headings to indicate to the user what is to come. Using different typefaces, color, and emphasis in headings can also help create a readable page. vi. Use short paragraphs Like on the Web, keep paragraphs short, using no more than two to three sentences per paragraph. 6. Graphics The final design element is graphics, or the images that are used to establish or aid avisual experience. Graphics can be used to supplement the look and feel, or as content displayed in line with the text. For example, in Figure 4.14, we can see Ribot’s Little Spender application for the iPhone and the S60 platform. The use of graphical icons in the iPhone experience helps to establish a visual language for the user to interact with to quickly categorize entries. On the S60 application, the wallet photo in the upper-right corner helps communicate the message of the application to the user. Fig 4.14 Ribot’s Little Splendor application uses graphics to define the experience Iconography The most common form of graphics used in mobile design is icons. Iconography is useful to communicate ideas and actions to users in a constrained visual space. The challenge is making sure that the meaning of the icon is clear to the user. For example, looking at Figure 4.15, we can see some helpful icons that clearly communicate an idea and some perplexing icons that leave us scratching our head. 27 Fig 4.15 Glyphish provides free iPhone icons Photos and images Photos and images are used to add meaning to content, often by showing a visual display of a concept, or to add meaning to a design. Using photos and images isn’t as common in mobile design as we might think. Because images have a defined height and width, they need to be scaled to the appropriate device size, either by the server, using a content adaptation model, or using the resizing properties of the device. In the latter approach, this can have a cost in performance. Loading larger images takes longer and therefore costs the user more. Using graphics to add meaning to a design can be a useful visual, but we can encounter issues regarding how that image will display in a flexible UI—for example, when the device orientation is changed. In Figure 4.16, we can see how the pig graphic is designedto be positioned to the right regardless of the device orientation. Fig 4.16 Using Graphics in Multiple Design Orientations 4.5.2 Mobile Design Tools 28 Mobile design requires understanding the design elements and specific tools. The closest thing to a common design tool is Adobe Photoshop, though each framework has a different method of implementing the design into the application. Some frameworks provide a complete interface toolkit, allowing designers or developers to simply piece together the interface, while others leave it to the designer to define from scratch. In Table 4.4, we can see each of the design tools and what interface toolkits are available for it. Table 4.4 Design Tools and Interface Toolkits All the Best! 29