#006 Still Untitled

The dangers of data-driven UX

“The frequency of this type of occurrence on bus journeys would at times make me prefer cycling even in the snow, when I reflected on the possibility of being dumped along the way. I’d get on the bike instead, it was bound to be less painful.”

10 September 2021

Double decker bus speeding past on Westminster Bridge, London. Think it’s fast? It might be just an optical illusion. Photo © Piccia Neri

Contents of this issue:
    Add a header to begin generating the table of contents
    Piccia Neri's face looking up to the left, with a smile and the sunlight behind her

    By Piccia Neri

    Reading time:
    10-12 minutes

    Like this article? Sign up to the “Still Untitled” newsletter to read it before anyone else, and to get exclusive news and offers.
    By Piccia Neri.

    The UX of double-decker buses

    Have you ever been on a double-decker London bus?

    Even if they’re not as cool as the old-fashioned Routemasters these days, they still are the best way to explore the Big Smoke. If you get on at the first stop, you can easily be the one to win the race upstairs and conquer the best seat, on the front of the top deck, where you can heave a sigh of relief, relax and enjoy the view. Human traffic on the streets of London is as interesting as its buildings, monuments and parks, so you can count on endless entertainment.

    I always much preferred buses to the Tube when I lived in London. It was truly time out for me, when I was lucky enough to get a good window seat on the top deck. I’d either let my mind wander looking out, or read a good book, or listen to music or a podcast.

    It’s also true that I would usually only ever use public transport when it poured down with torrential rain or the elements made it otherwise impossible for me to be on my bicycle. I would only ever cycle otherwise, and one of the reasons will come to light as you read on.

    So if I was on a bus, you can rest assured that the weather conditions were almost certainly inclement.

    Imagine the situation. It’s dark out, on an extremely wet and cold winter evening. You’re on your way home from a long day at work, exhausted, looking forward to making it back and enjoying your evening in the warmth of your home. It’s late, you’re hungry too, you plan dinner in your head. The peak hour traffic in central London is horrendous, most roads are gridlocked to a halt.

    But you feel safe in the knowledge that your bus is taking you home, you checked the destination on the front (essential bit of London savvy, this one: always check the front of the bus. Not that it actually provides conclusive evidence, as you’ll see, but it does help – read on).

    So there you are, thinking yourself so lucky for bagging a good seat, comfortably settled by a window on the top deck, smug as a bug in a rug, looking out at the people still in the street rushing home, fighting the wind and the rain under their umbrellas, while the bus hops along in the preferential bus lane, passing the cars stuck in traffic.

    The progress is slow, sometimes painfully so, but steady. You check the time, not long now till the comfort of home. You’re a few stops away, just a mile or two. You powerfully visualise your living room, like a beacon in the stormy night.

    You trust. You believe. You have faith.

    When visualisation doesn’t work

    But suddenly, at the following stop, the bus comes to an abrupt halt.

    OH NO! You cry inside.

    Surely not? It cannot be…

    And yet, hope is still alive inside you. You think, the driver just braked to avoid an obstacle. Surely we’ll start again soon.

    Or perhaps it’s just a driver change? It often happens on route.

    Then – the lights start flashing.

    And all hope dies within you.

    The moment you dreaded has come.

    You wail loudly inside, a silent, piercing scream full of impotent rage.

    Flashing overhead lights on a stationary London bus mean one thing, and one thing only.

    You have to get off it.

    ALL OUT! shouts the driver. ALL OUT!

    😭

    🤬

    Light trails of double decker bus speeding past St Paul's Cathedral, London, visible in the background, at night. A red phone booth is on the foreground on the right.
    Double decker bus speeding past St Paul’s cathedral, probably after kicking out all passengers. Photo © Piccia Neri

    Yes. On a wet, cold, dark, miserable winter evening, every single last passenger on this bus has to be disgorged out into the street. A furious mob of people is now stranded, too far from home still to walk it back in the rain, and just hoping the next bus along has room for them (yeah right…).

    Even though the final destination on the front was the correct one – the bus dumps you unceremoniously on route, way before the end of your journey.

    [but the driver has to provide you with the fare for the second leg of your journey, because you checked the destination on the front – so, demand it!]

    But why you ask, WHY??? Isn’t a bus’s primary job supposed to be TAKING THE PASSENGERS HOME???

    Well, you would think so. But as it turns out, it actually isn’t.

    One of the most important KPIs (Key Performance Indicators) for a London bus driver is – the time it takes to complete the journey. This factor does not take into account at all the perfidy of London traffic on a rainy winter night. And it doesn’t care at all about whether the passengers were taken home or not.

    So when a bus driver realises they’re late for their utterly science-fictional expected time back to the depot, they have no choice but drop their passengers off: stopping at every bus stop along the way to get them on and off would result in them getting heavily fined for tardiness.

    A total UX fail

    This is what happens when your UX and design rely too heavily on unreasonable KPIs and other quantitative data, without taking quality into account: a total UX fail.

    This musing on London buses was inspired by a conversation I had this week with my friend Mike Sale, a data expert. I interviewed Mike for the UX Blueprint course: he delivered a super-useful lesson on how analytics can help UX.

    Mike explained  to me the concept of the “QQP Triangle”: Quality, Quantity and Perception. When you use this triangle to evaluate UX, pure metrics on their own simply don’t count enough to make any significant decision.

    Screenshot of the video on how analytics can help UX, featuring Mike Sale.

    If London Transport were using the QQP triangle to evaluate their drivers’ performance, they certainly would NOT insist on judging them solely on the length of their journey – an insane metric anyway in any central city traffic.

    They would also consider quality (not achieved: an unfinished journey is a bad quality product) and perception (abysmal. I am still screaming inside).

    The frequency of this type of occurrence on bus journeys would at times make me prefer cycling even in the snow, when I reflected on the possibility of being dumped along the way. I’d get on the bike instead, it was bound to be less painful.

    Joe Natoli confirms: KPIs are bullshit

    Interestingly, this week my mentor and all-around UX guru Joe Natoli also posted a bit of a rant on KPIs and other “wrong” data and numbers that companies tend to base way too many crucial decisions on.

    If you don’t follow Joe on Twitter yet, I strongly recommend that you do. His content is invariably on point, and he is very vocal on the issues that matter. 

    In this super-interesting thread, Joe points out how the main problem is that the metric becomes the goal. London bus drivers’ KPIs providing an excellent example of that. How can the goal be the speed, and not the welfare of the passengers?

    In fact, Joe uses the metrics used in hospitals as an even more powerful example. When metrics such as speed of care are not adjusted to their context, corners are cut, and everybody suffers. Cutting corners in a hospital is worse than not completing a journey: it can mean the difference between life and death.

    In Joe’s words: “What’s ironic about KPIs is that they’re often adopted to improve safety, service or quality; increase profit, reduce loss. In far too many instances I’ve seen, they do the exact opposite.”

    However, there ARE indeed metrics that help, and Joe has a course that shows you which ones they are. This is not an affiliate link. I have taken this course and I think it gives you an absolutely golden ticket to the best audits you can possibly conduct – as well as a new product to offer your clients.

    Content-first Design with Wireframes: free Balsamiq webinar

    On Thursday, 30 September 2021 at 7pm CEST, my friend Billy Carlson from Balsamiq will hold a free webinar on the power of the content-first approach to design, and how wireframes can help us with that.

    I met Billy because Balsamiq were generous enough to sponsor the Design for Conversions conference. I interviewed Billy on the conference day dedicated to content, and we had a great conversation about the content-first approach and how it’s kind of the only one you should ever use… (fight me).

    I have already signed up to this webinar of course, I hope to see you there.

    In case you missed it: How the UI can affect the UX


    As exemplified by this series of restaurant menus that are indeed remarkable – in good ways and bad, all directly affecting the UX.

    Like this article? Sign up to the “Still Untitled” newsletter to read it before anyone else, and to get exclusive news and offers.
    By Piccia Neri.

    The Design for Conversions Newsletter

    Scroll to Top