ISSN 1754-3657 (Print) ISSN 1754-3665 (Online)

Computer Science for Fun

I ssue 1 5

Does your computer understand you?

Making computers that treat you right
How user-friendly is a baby? Can emotions spread through Facebook? Will your next phone be a banana?

W elcome to cs4fn

Making computers that treat you right

Computing is about people. Sure, you need technical knowledge, but in order to make a successful gadget or design a bestselling app, you have to know how people work too. Your users have to enjoy using your invention. Fortunately, theres an entire field of computer science dedicated to improving the experience of using a computer. Human-computer interaction combines technical know-how with psychology, inventiveness and artistry. In this issue were exploring how designers make computers that feel good to use. Youll read about eating cookies in virtual reality, controlling computers by harnessing invisible airborne electricity and falling in love over video chat. Dive in!

Delicious computing
Imagine being able to pick up an ordinary banana and use it as a phone.
That's part of the vision of 'invoked computing', which is being developed by Japanese researchers. A lot of the computers in our lives are camouflaged smartphones are more like computers than phones, after all but invoked computing would mean that computers would be everywhere and nowhere at the same time. The idea is that in the future, computer systems could monitor an entire environment, watching your movements. Whenever you wanted to interact with a computer, you would just need to make a gesture. For example, if you picked up a banana and held one end to your ear and the other to your mouth, the computer would guess that you wanted to use the phone. It would then use a fancy speaker system to direct the sound, so you would even hear the phone call as though it were coming from the banana. Sometimes you might find yourself needing a bit more computing power, though, right? Not to worry. You can make yourself a laptop if you just find an old pizza box. Lift the lid and the system will project the video and sound straight on to the box. At the moment the banana phone and pizza box laptop are the only ways that you can use invoked computing in the researchers' system, but they hope to expand it so that you can use other objects. Then, rather than having to learn how to use your computers, your computers will have to learn how you would like to use them. And when you are finished using your phone, you could eat it.

www.cs4fn.org

The A formula for confusion robot

If you were a Formula 1 driver twenty years ago, you only had one button on your steering wheel: for controlling the radio. Now, you would have more than twenty buttons controlling the radio, your speed, engine revs, your differential, and the fuelair mixture your car runs on. You might even have a button that controls a drinks bottle. As F1 drivers whiz around a track at over 200 mph, it's surprising that they would have steering wheels that are so complicated. Drivers agree it's distracting. It's a lot of extra stuff to think about, and it keeps their eyes off the road. Although steering wheels are custom-made for each driver, they can really only change the grips
Image: philki, flickr

to fit their hands, rather than change the buttons to suit their user preferences. It will be interesting to see whether F1 steering wheels become simpler as car designers discover human-computer interaction, or whether drivers will just have to get used to having as much to look at inside the car as outside.

always wins
Researchers in Japan have made a robot arm that always wins at rock, paper, scissors. Not with ultra-clever psychology, but with old-fashioned cheating. The robot uses high-speed motors and precise computer vision systems to recognise whether its human opponent is making the sign for rock, paper or scissors. One millisecond later, it can play the sign that beats whatever the human chooses. Because the whole process is so quick, it looks to humans like the robot is playing at the same time. See for yourself by going to the magazine+ section of www.cs4fn.org and watching the video of this amazing cheating robot.

A fishy format that formally fixes folk to focus

The setup for the transfer of data is called handshaking. On the Internet, for example, there are well defined rules about how computers know to get ready to receive data. This is called IP, for Internet Protocol. There are, however, other important occasions when we need to ensure that recipients are ready to receive the information we are sending. The BBC Radio 4 shipping forecast is an interesting example. The shipping forecast is a special weather bulletin transmitted at 12.48am precisely that gives detailed information about the current weather conditions around the British Isles, vital for fishing boats and other seafarers. To ensure that boats at sea are able to get ready to hear this important information, the BBC introduced a piece of music to precede the broadcast. Sailing By is a slow waltz composed in 1963 by Ronald Binge. The music uses a repetitive ABABC structure with distinctive rising and falling woodwind sounds. This set of repeating patterns helps sailors tuning in, often with bad reception, to be able to recognise the music and know that the forecast is on its way. In the early days it also helped act as a buffer in case the earlier news broadcast ran shorter or longer than usual.

cs4fn

eecs.qmul.ac.uk

Bringing people closer when theyre far away

Living far away from the person you love is tough. You spend every day missing their presence. The Internet can help, and many couples in long-distance relationships use video chat to see more of each other. Its not the same as being right there with someone else, but couples find ways to get as much connection as they can out of their video chats. Some researchers in Canada, at the University of Calgary and Simon Fraser University, interviewed couples in long-distance relationships to find out how they use video chat to stay connected.
Nice to see you The first thing that the researchers found is perhaps what you might expect. Couples use video chat when its important to see each other. You can text little messages like I love you to each other, or send longer stories in an email, and thats fine. But seeing someones face when theyre talking to you feels much more emotionally close. One member of a couple said, The voice is not enough. The relationship is so physical and visual. Its not just about hearing and talking. Others reported that seeing each others face helped them know what the other person was feeling. For one person, just seeing his partners face when she was feeling worn out helped him understand her state of mind. In other relationships, seeing one another helped avoid misunderstandings that come from trying to interpret tone of voice. Plus, having video helped couples show off new haircuts or clothes, or give each other tours of their surroundings. Hanging out on video The couples in the study didnt use video chat just to have conversations. They also used it in a more casual way: to hang out with each other while they went about their lives. Their video connections might stay open for hours at a time while they did chores, worked, read, ate or played games. Long silences might pass. Couples might not even be visible to each other all the time. But each partner would, every once in a while, check back at the video screen to see what the other was up to. This kind of hanging out helped couples feel the presence of the other person, even if they werent having a conversation. One participant said of her partner, At home, a lot of times at night, he likes to put on his PJs and turn out all the lights and sit there with a snack and, you know, watch TV As long as you can see the form of somebody thats a nice thing. I think its just the comfort of knowing that theyre there. Some couples felt connected by doing the same things together in different places. They shared evenings together in living rooms far away from each other, watching the same thing on television or even getting the same movie to watch and starting it at the same time. Some couples had dinner dates where they ordered the same kind of takeaway and ate it with each other through their video connection. Designing to connect This might not sound like research about human-computer interaction. Its about the deepest kind of human interaction. But good computer design can help couples feel as connected as possible. The researchers also wanted to find out how they could help couples make their video chats better. Designers of the future might think about how to make gadgets that make video chat easier to do while getting on with other chores. Its difficult to talk, film yourself, cook and move through the house all at the same time. Whats more, todays gadgets arent really built to go everywhere in the house. Putting a laptop in a kitchen or propping one up in a bed doesnt always work so well. The designers of operating systems need to work out how to do other stuff at the same time as video. If couples want to have a video chat connection open for hours, sometimes they might need to browse the web or write a text message at the same time. And what about couples who like to fall asleep next to one another? They might need night-vision cameras so they can see their partner without disturbing their sleep. Were probably going to have more longdistance relationships in the future. Easy, cheap travel makes it easier to move to faraway places. You can go to university abroad, and join a company with offices on every continent. Its an awfully good thing that technology is making it easier to stay connected with the people who are important too. Video chat is not nearly as good as feeling your lovers touch, but when you really miss someone, even watching them do chores helps.

www.cs4fn.org

cs4fn

eecs.qmul.ac.uk

Proper pumping
Humans are imperfect. There are some things we always find difficult, and theres no way to change it. Interaction designers need to accept those things, and then design ways around them. For example, humans are not that great at doing many jobs at the same time. Sure, we can multitask if we really have to, but it often takes longer and we are prone to making more mistakes. One place that mistakes can be really dangerous is a hospital. Which means that its probably better for doctors and nurses to do one thing at a time.
One thing at a time How do you make that happen? This is where designers can almost be like clever tricksters. People will often choose the path of least resistance, so if you can find a way to make your desired method the easiest method, youre ahead of the game. That was the idea behind a study by Jonathan Back, Anna Cox and Duncan Brumby at University College London. They work on a project called CHI+MED, which is trying to find ways to design safer medical devices. They were working on the pumps that deliver medicine into patients bodies through an intravenous drip. They believed that it was safer for nurses to program those pumps one at a time: they make fewer mistakes that way, and mistakes with medicine can be fatal. But could the researchers find a way to coax nurses into programming them in sequence rather than all together? Design detectives They found a clue in the way nurses get their dosage information. They read the dosage off of a prescription form, then program it into the machine. Another clue came from the design of many pumps in hospitals. They are designed to stack one on top of another, to save space. What would happen is that nurses would bring their prescription form right next to the stack of pumps, and program the machines all together. Once they did a bit of programming on the first pump they would have to wait for it to do some work before the second step. So the nurses would go on to program a bit of the second pump, then come back to finish the first pump later. It seems more efficient, but it meant more mistakes. Sometimes nurses would get mixed up and forget steps, like opening the clamps that allow the medication to flow. Improving nurses form The CHI+MED team wondered if the solution was to do what seems less efficient and move the prescription form away from the stack of pumps. That way, the nurses would have to remember the information they were reading from the forms. Suddenly it was easier to program one pump at a time than to do them together. Their test subjects looked at the form, memorised the information, programmed the pump, released the clamp, and went back to the form to memorise the information for the next pump. Fewer mistakes! In the real world, you cant really force nurses to keep their forms on the other side of the room from the pumps. But the experiment gave designers an important clue to reducing mistakes. If they make it easier to program one pump at a time, patients will be safer. And it goes to show that good design is about knowing how people work especially when were not perfect.

www.cs4fn.org

A game set by sweat

At the very beginning of a video game, you have to decide how difficult you want the game to be. Sometimes, though, you choose wrongly, and get overwhelmed by tough parts of a level. The problem is, not many games let you go back and choose a different level of difficulty without restarting the entire game. It would be great if you could switch in the middle without losing all your hard work. Some human-computer interaction researchers in Texas thought that they could do even better than that. What if the computer could sense when you were having trouble, and adjust the level for you? They began by hooking up a heatsensing camera to the game and training it on players' foreheads. The idea was that as you become more stressed, your forehead heats up. If the researchers could tell what temperature signals that players have stopped having fun and have started feeling stressed, the game could be programmed to turn down the difficulty automatically. That's just what they did. Over a series of experiments, they had people come into their lab and play a game. It started out easy, so that the players could get to know the game, and then the heatsensing camera was turned on. The game would automatically adjust its difficulty if their forehead temperature suggested they were getting overwhelmed. The game also works the other way: if the computer thinks you might be bored, the difficulty level goes up. That way, with the help of a bit of fancy equipment and computer psychology, game players are never too bored and never too stressed. Just right.

The invisible dice mystery

The ancient Egyptians, Romans and Greeks used dice with various shapes and markings. Some even believed they could use dice to predict the future. Using just a few invisible dice, which you can easily make at home, you can amaze your friends with a transparent feat of magical prediction. Find out how in the magazine+ section of our website, www.cs4fn.org.

cs4fn

eecs.qmul.ac.uk

T asty behaviour
Its hard to be good, especially when it comes to eating right. Sadly whats tasty isnt always whats healthiest. Sometimes it can help to have a little nudge in the right direction. Thats what computer scientists are trying to do in three new systems. By training you, enticing you or downright fooling you, each invention on these two pages uses computer interaction to help users eat better.

The power of chocolate

Be honest: theres some food you find tough to resist. It could be chips, fizzy drinks or cake. For a lot of people, the irresistible temptation is chocolate. We know its not good to overindulge in anything, but when that sweet, dark dessert is put in front of us we just have to take a bite. It takes a lot of willpower to resist. The problem is, scientists think willpower may be something we run out of. The strength of our will can be tired out like a muscle, and making the right choice gets more difficult the longer you hold out. Some researchers in Germany wanted to test whether willpower could be like a muscle in another way: maybe over time you could train it. They even designed a machine to work out your willpower, and called it the chocolate machine. The chocolate machine is a dispenser filled with round chocolates. It sits on a students study desk, and every 40 to 60 minutes it releases one of its little sweeties. Users were free to eat one of the chocolates, but the researchers told them that to practise self-control, it would be good to put the chocolates back in the machine. Over time, seven out of ten chocolate machine users found that their selfcontrol got better. Most people developed strategies to help them with their willpower. Some imagined that the chocolate balls were actually just balls of wood, while others pretended that the chocolates belonged to somebody else. A few users imagined that the machine had feelings, and that by putting the chocolates back they were helping it. Overall, it seems like whenever you face a really strong temptation, it helps to imagine ways to make the temptation seem weaker. The researchers think that their chocolate machine could be a better way to help people behave. A lot of products that try to change peoples behaviour depend on monitoring someone closely. A bit like having a parent over your shoulder, making sure youre doing your homework. The chocolate machines playful design might be a better way to help someone develop their own good strategies. But now that youve read this far, can you resist having a chocolate?

www.cs4fn.org

Making students life brighter (and wetter)

A lonely, unused drinking fountain in a busy university corridor inspired a group of interaction researchers based in Spain and America. Drinking water is good for you, and so is a few seconds rest from rushing to class. What could encourage people to stop and take a break? They needed to find a design that would work in a crowded hallway, and hopefully one that would make the whole corridor seem a little friendlier. They came up with a concept they called Gurgle. Gurgle is an interactive light and sound display around the drinking fountain. The researchers found a light called an aquasplash, which is normally used in aquariums. It gives off a blue light that looks like wavy water. They pointed the aquasplash up from the fountain, filling the nook around the fountain with pleasant blue ripples. When someone walks past the fountain, their movement triggers the lights for just a moment, but if they then stop to take a drink the lights stay on. To add to the relaxing effect, while the person drinks they are treated to the sound of a gurgling stream coming from a speaker. The researchers may have been proud of their setup, but they needed to find out whether it was having any effect. Did more people actually use the fountain? To find out they turned Gurgle on and off at certain times over one term, and counted how many people used the fountain. Gurgle worked: the number of people using the fountain went up by around 50% when it was turned on. They also found some buzz around campus for Gurgle. Some people went out of their way to use the fountain, and told their friends to use it too. Its nice to know that a simple design can liven up some dull surroundings, and get people to drink more water too.

Virtually full
Heres a surprising trick that makes you feel like youve had your fill of food: make the food look bigger. Researchers in Japan have developed a virtual reality system that makes you see your food as though its a larger portion. Once youve eaten it, you feel as though youve eaten more than you actually have. The researchers think they could use their system to help people eat less. Humans use clues to guess how much food to eat. Psychologists have found that one of the clues we use is size. If the food portion looks large compared to the objects around it, we assume the portion is bigger. For example, experiments have found that the size of a bowl can affect how much soup it takes to make someone feel full. The researchers wondered if people would eat fewer cookies if they thought the cookies were bigger than they actually were. To get the effect they used a virtual reality system. Volunteers wore goggles with a video camera on them. The camera was connected to little computer screens inside the goggles, showing a view of what was in front of them. The goggles were also connected to a computer that could control what the volunteer saw. Researchers gave the volunteers a plate of Oreo cookies and told them they could have as many as they liked. What the volunteers didnt know was that sometimes the virtual reality changed the size of the cookie so that it looked like they were holding a bigger (or sometimes a smaller) Oreo. The researchers then kept track of how many cookies each volunteer ate before feeling full. The researchers found that their system worked. By changing the apparent size of the cookies, they could indeed affect how many cookies the volunteers had to eat before they felt full. And the virtual reality system did its trick flawlessly: no one suspected that the size of the food was being changed. One problem, though, was that in order to keep the illusion going, you cant change the size of the cookie very much. If you think about it, imagine if you tried to make the cookie really huge like the size of a pizza. You would have to deform the cookie (and the hand holding it) so much that it would be obviously fake. So computer science will have to be happy with just making our cookie binges a little bit smaller.

cs4fn

eecs.qmul.ac.uk

Designer baby
How user-friendly is a baby? Paul Curzon, father to a newborn son and computer scientist at Queen Mary, University of London, investigates.
I recently acquired a new interactive gadget, better known as a newborn baby. How well-designed is he, I wondered. In particular how usable is he for a novice like me? Time to do a usability analysis. Heuristic evaluation is one of the simplest and most commonly used methods to evaluate the usability of a new system. It involves checking the system against a set of design principles: things like the system should always keep users informed of what is going on. The usability expert judges the system under evaluation against each principle making a list of potential problems and suggests improvements to fix each. Here goes for our baby... Whats going on? The first principle to check, 'visibility of system status', is about whether you can tell what is going on inside a gadget as you interact with it. It's all about 'feedback'. Every interactive gadget has a hidden internal state to keep track of what it is doing, and users need to be able to tell what that state is when it matters. The feedback should not only be understandable but also be timely. It's likely to be useless if it comes too late. So how does the baby do for visibility? Well there are five or six main internal states that seem to matter: happy, tired, hungry, in pain and needing his nappy changing. He certainly gives feedback about all of them. It's clear when he is happy. Its all smiles, gurgling and cooing. When he enters one of the other states you definitely know it. Speak the lingo The second design principle to check is about the match between the system and the real world. This is a convoluted way of saying it must be easy to understand. Does it use the same language and concepts as you or do you have to learn what everything means first to use it? Does it follow standard conventions? Hmm! The baby doesnt do so well here. He uses screams. It certainly would help if there was a bit of English there from the start. Just a few words like hungry, poo, and so on pre-installed when it came out of the box would be a big improvement. He screams his head off. The feedback is timely too! The moment he leaves the happy state he starts to scream and he wont stop unless you fix the problem, however long it takes. So far so good! There is a problem though: for a novice at least, there seems to be only one form of feedback for all non-happy states. It seems to be impossible to tell the states apart, and so to know what the right thing is to do: feed him? Change his nappy? Rock him to sleep? Kiss him better? All you can do is guess. Our designer baby definitely needs something more. A screen on the forehead might be one option. It could flash a message that just says what the problem actually is. Simple. Control! The third principle is about user control and freedom. The user of the system should feel they have control. They should feel things happen when they want them to, rather than the system just doing things behind their back without them being able to stop it. There should be simple controls to get the state to where you want it, too like an undo button for when something goes wrong and a home button that takes you back to the start rather than you having to navigate there. What can I say? This doesnt seem to have been taken into account at all in the babys design. In fact, since we had the baby, my whole life is out of control, never mind the baby! Focusing on the baby though, I seem to have little control of it at all. Things just happen and I have to react, with no easy way of gaining back control. Take poo! You try and gain control by changing the baby before it leaks and what happens? Five minutes later, when its most inconvenient, he goes and poos. You have to change him all over again. Some of the help books suggest you can gain control by fixing feeding times and sleep times, ignoring the screams of anger that result. After a while (and we are talking months of frazzled nerves rather than hours here) the baby will fall in to the routine and all will be fine. Why on earth didnt the designer just build that behaviour in as standard? There definitely should be a home button. It would just put the baby back in the happy state it was in when it first woke up, allowing you to start the day again. A poo now button would also be good. In fact that is so obvious I cant imagine why the designer never thought of it. He or she must have never had a baby themselves.

10

www.cs4fn.org

Same again? The next principle is about consistency and standards. The system shouldnt do different things at different times that mean the same thing. At first sight this seems to be ok. The baby just screams whenever he wants something. Thats pretty consistent! However, with more detailed observation its clear there is more going on. When he is tired, sometimes he screams. At other times he starts to angrily throw his toys about. At calmer times he starts to twist his hair with his fingers, or just reaches up for a cuddle, all meaning I need to sleep.

We want consistency for our designer baby. The designer should pick one way to communicate the message and stick to it. My preference would be for using the reaching up for a cuddle signal. We are focusing on usability here rather than user experience. However, user experience is important too. If the baby always used the want cuddle signal rather than the scream signal whenever he wanted to communicate Im tired it would certainly improve my experience! User experience is good for business if you want repeat sales. If the designer had focused a bit more on experience it would have increased the likelihood I would get another.

To read more on the usability of Pauls baby, look in the magazine+ section of our website, www.cs4fn.org.

Our designer baby definitely needs something more. A screen on the forehead might be one option. It could flash a message that just says what the problem actually is. Simple.

cs4fn

eecs.qmul.ac.uk

11

Cooking up computer style

Using clever computer vision techniques its now possible for your ingredients to tell you how they should be cooked in a kitchen. The system uses cameras and projectors to first recognise the ingredients on the chopping board, for example the size, shape and species of fish you are using. Then the system projects a cutting line on the fish to show you how to prepare it, and a speech bubble telling you how long it should be cooked for and suggesting ways it can be served. In the future these cooking support systems could take some of the strain from mealtimes. At least it will help to make us all better cooks, and perhaps with an added pinch of artificial intelligence we can all become more like Jamie Oliver.

12

www.cs4fn.org

Ideas in a flash
Not long ago, some computer scientists stood up and spoke out. They said that a particular bit of gadget design had become unimaginative. They were bored with the way things were and they werent going to take it anymore. They gave themselves a mission: to reignite peoples imagination about this particular bit of design. What were they talking about? A feature that almost every electronic device shares, which we take completely for granted and accept is small and kind of dull. The researchers, from Carnegie Mellon and Michigan State universities in the US, want to start a revolution in those little lights that tell us whether something is on or off.
Not saying very much Those lights (called LEDs for light emitting diodes) started appearing on gadgets in the 1970s, when they first became cheap enough for manufacturers to stick on almost everything they made. Fast forward to the 21st century, and theyre everywhere. LEDs are on computers, televisions, smoke alarms, speakers, even toothbrushes. But what do they really tell us? Not a whole lot, when you think about it. They can be on, which usually means that the appliance is on, or off, which means off. But then, sometimes a light thats on means the gadget is standing by. Your TV probably has an LED thats on when the television itself is off. Thats one problem with the little lights they can mean different things on different gadgets. Another problem is that its not always clear what the lights mean anyway. The researchers found a toaster that had a blinking LED on it. What is that supposed to mean? That the toast is done? Or that its still cooking? Or that the toaster is jammed? Who knows. On, off and in between To start to get the creative wheels turning, the researchers brought in a team of designers for a brainstorming session. How many different things could they think of for a single, solid-colour light to do? A lot, it turns out. They came up with 24 ways for a light to be on or off, ranging from a heartbeat-like pulse to a gentle flickering like a candle. The next step was to try and connect those behaviours to something that your gadget might like to tell you. Things like incoming call, booting up, or unable to connect. Next came an experiment. The computer scientists randomly partnered up combinations of the flickery light behaviours and the possible messages. Then they got hundreds of testers to look at the pairings and rate how appropriate they were. Would a heartbeat go better with youve received a message or Im thinking? It was up to the testers to say. Which light is right? The results showed that certain types of behaviour connected better with certain messages. For the messages that were notifications like incoming call or scheduled event, bright flashes seemed to make sense. On the other hand, slow pulses were better to get a low-energy message across. For turning on, a light that got brighter in steps, like a staircase, was best. You can sort of see the comparisons that are at work in each of these messages the slowly building light is kind of like the machine activating all the parts within itself. One problem the experiment found was that there wasnt a clear way to get across messages about being unable to do something. There was no agreement among the testers about whether any of the light behaviours looked right. Maybe they just didnt find the right comparison this time. There will be a next time. After all, the researchers want to start a revolution. They want our little lights to say more, and say it with more imagination. Others will try again, and build a more expressive language of tiny little lights. And just wait until they start changing colours.

cs4fn

eecs.qmul.ac.uk

13

Y our own electrical sea

You cant see them, but there are waves of electricity flowing around you right now. Electricity leaks out of power lines, lights, computers and every other gadget nearby. Soon a computer may be able to track your movements by following the ripples you make in your own electromagnetic sea. Scientists at Microsoft Research in the US have figured out a way to sense the position of someones body by using it as an antenna.
Why would you want a computer to do this? So that you could control it just by moving your body. This is already possible with systems like the Xbox Kinect, but that works by tracking you with a camera, so you have to stay in front of it or it loses you. A system that uses your body as an electric antenna could follow you throughout a room, or even a whole building. First you need an instrument that can sense the changes you make in your own electrical field as you move around. In the future, the researchers would like this to be a little gadget you could carry in your pocket, but the technology isnt quite small enough yet. For this experiment, they used a wireless data sensor thats about twice the size of a mobile phone. The volunteers wore it in a little backpack. All the electrical data it picked up were transmitted to a computer that would run the calculations to figure out how the user was moving. Get moving In their first experiment, the researchers wanted to find out whether their gadget could sense what movements their volunteers made. To do this, they had the volunteers take their sensing devices home and use them in two different rooms: the kitchen and the living room. Those two rooms are usually different from one another in interesting ways. Living rooms are usually big open spaces with only a few small appliances in them. Kitchens, though, are often small, and cram lots of big electricals in the same room. The electrical sensors would really have to work hard to make sense through the interference. Once the experiment was ready to go, each volunteer ran through a series of twelve movements. Their exercises included waving, bending over, stepping to the right or left, and even a bit of kicking and punching. The sensor would collect the electrical readings and then send them to a laptop. What happened after that was a bit of artificial intelligence. The researchers used the first few rounds of movements to train the computer to recognise the electrical signatures of each movement. Later on, it was the computers job to match up the readings it got through the sensor to the gestures it already knew. Thats a technique called machine learning. One of the surprising things that made the sensors job tougher was that electrical appliances change what they are doing more often than you think. Maybe a refrigerator switches its cooling on and off, or a computer starts up its hard disk. Each of these changes means a change in the electrical waves flowing through the room, and the computer had to recognise each gesture through the changing noise.

14

www.cs4fn.org

Where d you go? The next step for the system was to see if it could recognise which room someone was standing in when they performed the movements. There were now eight locations to keep straight two locations in one large room and six more scattered throughout the house. It was up to the system to learn the electrical signature for each room, as well as the signature for each movement. Thats pretty tough work. But it worked well really well. The system was able to guess the room almost 100% of the time. Whats more, they found that the location tracking even worked on the data from the first experiment, when they were only supposed to be looking at movements. But the electrical signatures of each room were built into that data too, and the system was expert enough to pick them out.

Putting it all together In the future the researchers are hoping that their gadgets will become small enough to carry around with you wherever you are in a building. This could allow you to control computers within your house, or switch things on and off just by making certain movements. The fact that the system can sense your location might mean that you could use the same gestures to do different things. Maybe in the living room a punch would turn on the television, but in the kitchen it would start the microwave. Whatever the case, its a great way to use the invisible flow of energy all around us.

The electrical sensors would really have to work hard to make sense through the interference.

cs4fn

eecs.qmul.ac.uk

15

Robot Road Run

Here's a fun, free iPhone app in which you are a robot racing down a road. In Robot Road Run it's up to you to stay away from objects and people as you zoom along, collecting bolts and batteries.
You play FLASH, an emotional robot. You need supplies to keep yourself running, so you go down the road in search of building materials and power. As long as you stay clear of obstacles and play nicely with the humans you meet, you do well. Each level gets a little bit harder. This game is brought to you by the LIREC project, which is developing emotional robots in real life. In fact, the robot you play in the game was built for real by researchers at the Wroclaw University of Technology in Poland. They hope their real-life FLASH will be so good that it could be your friend in the future. But in the meantime, you can have lots of fun playing the game. You'll find Robot Road Run in the App Store on your iPhone. But there is more information on the web too! Look for a link in the magazine+ section of www.cs4fn.org.

16

www.cs4fn.org

Shh! Can you hear that diagram?

What does a diagram sound like? What does the shape of a sound feel like? Researchers at Queen Mary, University of London are finding out.
Listening to diagrams and feeling sounds might sound like nonsense, but for people who are visually impaired it is a practical issue. Even if you can't see them, you can still listen to words, after all. Spoken books were originally intended for partially-sighted people, before we all realised how useful they were. Screen readers similarly read out the words on a computer screen, making the web and other programs accessible. Blind people can also use touch to read. That is essentially all Braille is replacing letters with raised patterns you can feel. Getting the picture The written world is full of more than just words though. There are tables and diagrams, pictures and charts. How does a partially-sighted person deal with them? Is there a way to allow them to work with others to create or manipulate diagrams, even when each person is using a different sense? That's what the Queen Mary researchers, working with the Royal National Institute of Blind People and the British Computer Association of the Blind, have been exploring. Their solution is a diagram editor with a difference. It allows people to edit 'node-and-link' diagrams, like the London Underground map, for example, where the stations are the nodes (meeting points) and the links show the lines between them. The diagram editor converts the graphical part of a diagram, such as shapes and positions, into sounds you can listen to and textured surfaces you can feel. It allows people to work together to explore and edit a variety of diagrams including flowcharts, circuit diagrams, tube maps, mind maps, organisation charts and software engineering diagrams. Each person, whether fully sighted or not, 'views' the diagram in the way that works for them. What does a circle sound like? The tool combines speech and non-speech sounds to display a diagram. For example, when the label of a node is spoken, it is accompanied by a bubble bursting sound if it's a circle, and a wooden sound if it's a square. The labels of highlighted nodes are spoken with a higher pitched voice to show that they are highlighted. Different types of links are also displayed using different sounds to match their line style. For example, the sound of a straight line is smoother than that of a dashed line. The idea for arrows came from listening to one being drawn on a chalkboard. They are displayed using a short and a long sound where the short sound represents the arrowhead, and the long sound represents its tail. Changing the order they are presented changes the direction of the arrow: either pointing towards or away from the node. For the touch part, the team uses a PHANTOM Omni haptic device, which is a robotic arm attached to a stylus that can be programmed to simulate feeling 3D shapes, textures and forces. For example, in the diagram editor nodes have a magnetic effect: if you move the stylus close to a node the stylus gets pulled towards it. You can grab a node and move it to another location, and when you do, a spring-like effect is applied to simulate dragging. If you let it go, the node springs back to its original location. Sound and touch are also integrated to reinforce each other. As you drag a node, you hear a chain-like sound, like dragging a metal ball chained to a prisoner! When you drop it in a new location, you hear the sound of a dart hitting a dartboard. The Queen Mary research team has tried out the editor in a variety of schools and work environments where visually impaired and sighted people use diagrams as part of their everyday activities and it seems to work well. It's free to download, so why not try it yourself? You might see diagrams in a whole new light. Just go to the magazine+ section of our website, www.cs4fn.org, for the link.

cs4fn

eecs.qmul.ac.uk

17

Contagious emotion
When you express yourself on Facebook, do you ever think that it can change what your friends write? You might not be writing for anyone in particular. You might even feel like youre just shouting into the wind. But the science of interaction shows the things you write have a small effect. Posts you write about your own emotions could end up being reflected in the emotions others write about. Research has found that your own Facebook status updates could affect your friends' posts, even days later.
The power of words? Psychologists have studied how people transmit emotions to one another, almost like viruses. Its useful to be able to share emotions; it helps us bond with someone else. But most research so far has been on people sharing emotions in person. Some psychologists even think that most emotions are shared through physical cues rather than words. But now that we communicate more in text, research is beginning to show its possible to affect someone elses emotions even if youre not with them. Adam Kramer, a researcher at Facebook, wanted to see whether emotional status updates led to friends posting about similar emotions. No need to be nice There is an interesting advantage to studying Facebook status updates. They are written for a large group rather than to any particular person, which means any shared feelings are more likely to be true. If someone sent you an emotional text, it would be rude not to respond, and sometimes you might pretend to share a certain emotion just to be nice. No one in particular needs to reply to a status update, so if you dont share the emotion you dont have to say anything similar. If Facebook statuses seemed to mimic emotion, its a clue that it really is possible for your posts to influence the emotions that others post about. Collecting feelings Adam started by selecting about 60,000 people on Facebook a group of users and all their friends who had posted something three days in a row. Then he gathered all the status updates they wrote over those three days. He analysed all the words they used with the help of a special dictionary for emotion research, which rates words according to whether they have positive or negative meanings. His prediction was that he would see users and their friends using words associated with the same sort of emotion. He did. Not only were users likely to post about similar emotions on the same day, the effect seemed to stick around for at least a couple of days afterward. Imagine you post about being happy. What Adam found was that if he compared your friends posts tomorrow to your posts today, they would tend to be happy posts too. Your friends posts would even show similar emotions to your original posts after two days. They even found that if one person posted about a certain emotion, it could stop others from posting an opposite emotion. For example, when people expressed positive emotions in their status updates, their friends held back from posting about negative emotions. Y oure influential So it turns out that the words you use on Facebook influence your friends choice of words too. Adam points out that you may not be affecting your friends themselves. Others will have to do more research about whether your Facebook posts can directly give other people the same emotions as you. But just by posting on Facebook, you change what your friends post about. Thats not to say that you have to follow the crowd. In fact, its probably better that you say what youre thinking, without worrying whether others are saying it too. Your friends can feel good for you when things are going well, and they can help if things are troubling you. But the next time you write a post, its interesting to think that you might be helping in a small way to bring thousands of other posts into existence too.

18

www.cs4fn.org

Let the brain take the strain

Whenever humans have complicated, repetitive jobs to do, designers set to work making computer systems that do those jobs automatically. Autopilot systems in airplanes are a good example. Flying a commercial airliner is incredibly complex, so a computer system helps the pilots by doing a lot of the boring, repetitive stuff automatically. But in any automated system, there has to be a balance between human and computer so that the human still has ultimate control. It's a strange characteristic of human-computer interaction: the better an automated program, the more its users rely on it, and the more dangerous it can be.
The problem is that the unpredictable always happens. Automated systems run into situations the designers haven't anticipated, and humans are still much better at dealing with the unexpected. If humans can't take back control from the system, accidents can happen. For example, some airplanes used to have autopilots that took control of a landing until the wheels touched the ground. But then, one rainy night, a runway in Warsaw was so wet that the plane began skidding along the runway when it touched down. The skid was so severe that the sensors never registered the touchdown of the plane, and so the pilots couldn't control the brakes. The airplane only stopped when it had overshot the runway. The designers had relied so much on the automation that the humans couldn't fix the problem. Many designers now think it's better to give some control back to the operators of any automated system. Instead of doing everything, the computer helps the user by giving them feedback. For example, if a smart car detects that it's too close to the car ahead of it, the accelerator becomes more difficult to press. The human brain is still much better than any computer system at coming up with solutions to unexpected situations. Computers are much better off letting our brains do the tricky thinking.

cs4fn

eecs.qmul.ac.uk

19

Back (page) to full health

We are all living longer, and behind the amazing feats of medical science are an army of computer scientists helping to make it possible. The doctor will see you now as we explore the computer science behind medical marvels.
Designed to get you going People do a lot of walking and running throughout their lives, and as we live longer our joints can wear out. Particularly in the hips and knees, disease, gravity and friction cause the normally smooth surfaces in the joint to deteriorate. This causes a great deal of pain and a loss of mobility. These days we can replace failed joints fairly easily using surgical procedures. Surgeons exchange knackered joints with artificial ones made from new types of clever bio-compatible materials (in the past joints were sometimes made using sculptured ivory). Computer scientists and medics have created computer-aided design and manufacturing techniques that use information from X-ray scans. A precise scan makes sure the fit of a new artificial joint is as close as possible to the shape of the joint it is replacing, in effect building a custom body part replacement. Diagnosis: a step up to help stepping out Sounds like a good idea Being able to see whats happening inside the human body without cutting it open sounds like a good idea, and sound allows us to do this. Ultrasound, a particularly high frequency noise, can be generated by vibrating a special probe in contact with the body. These sound waves penetrate the body and are reflected from internal organs. The echoes are detected in much the same way as a bats sonar works. With the help of clever software its possible to take the ultrasound echoes and create accurate 3D models of whats happening inside the body. We can safely take pictures of babies in the womb, and odd lumps and bumps on organs. In addition, the Doppler effect, which changes the pitch of a sound as its source moves, allows us to be able to measure the speed of blood flow in the chambers of the heart. Diagnosis: on reflection, sound can be good for your health Sick of tweeting Twitter, the popular social media platform, lets people tell the world what they are up to, from buying beans in the supermarket to feeling a bit ill. Computer scientists are working to develop ways to extract key words from tweets that indicate the onset of particular diseases, so that preparations can be made to treat them. While still in its early days this way of crowdsourcing information on the spread of disease, even at early stages, could prove useful in the future. Diagnosis: saying youre sick doesnt make you a tweet New tips for surgeons Researchers have developed new stretchy wearable electronic sensors that may find their way onto surgeons gloves. Using nanotechnology techniques they have been able to create ultra-thin electronics that can be cut into strips and attached to a rubbery layer, making it possible for them to stretch and bend round curvy things like fingertips without breaking. They plan to build these into surgical gloves, opening the possibility of enhancing surgeons' fingertips. Future surgeons could detect accurate temperatures, electrical activity and even to use their fingers to cut human tissue with a touch. Diagnosis: new healing hands that fit like a glove

cs4fn is supported by Google.

For a full list of our university partners see www.cs4fn.org

www.cs4fn.org