FiftyThree interview: From Microsoft Courier roots to Paper and Pencil

FiftyThree has established itself as a major player in the App Store with its Paper sketchbook app. It launched with rave reviews and an award from Apple in 2012. This is why when FiftyThree this week announced a venture into hardware with the $49.95 Pencil stylus, we were instantly intrigued. Has the company responded to the cheap stylus question and cracked the code for a stylus to work with what users call the best sketchbook app available? 

Pocket-lint sat down to chat with FiftyThree co-founder and design lead Andrew Allen to discuss the Paper beginnings, how Pencil came to be, and what's next for FiftyThree. Allen takes us from the FiftyThree founders' Microsoft roots, how the hardware came to be, to FiftyThree's plans to jump into services. 

"We're trying to combine beautiful design and really smart engineering," he told us. "We’re trying to excel on all those fronts. The tech behind it all is just so much more sophisticated than what others out there can do."

What do you do at Paper and FiftyThree? 

"FiftyThree was founded by myself and three others: Georg Petschnigg who is the CEO, Jon Harris who heads up hardware design, Julian Walker who heads up software engineering, and I in software design. 

"Paper has passed 8 million downloads, and we actually just crossed 100 million pages created within the app. That means there have been over 100 million ideas people have captured. It’s done extremely well. 

"Of course, the one thing people always ask us once they download Paper was, ‘What stylus do I use?’ For the longest time we just had to say, ‘Well you know, pick one out there’. Even before we launched Paper, we were working on Pencil. It’s been well over two years in development." 

What was it like working on the Courier project at Microsoft?

"I think the biggest thing Courier brought was the founders of Paper together, to be honest. We didn’t know each other before then, and that’s how we came together. We just loved working together. We would find that it would be late at night - 10 o’clock - and we’d look around the office and it would just be us still there working. We just had a great respect for each other and how we worked. 

"Courier was born out of the idea that creativity is the key piece to solving today’s problems. It was sort of a look at how could computing work to enable creativity rather then obstruct it the way it typically has. I think that’s why Courier struck a chord with so many people, because it was a re-envisioning of technology and something that put technology as more of an empowering tool for people instead of an obstacle. 

"For example, I think people see Adobe products as complex video editing applications. They see these tools people typically create with as just way too complex and intimidating. Courier was the first time people felt like, ‘Hey maybe I can be creative too. It’s not my fault it’s the tool’s fault.’ When Courier went away, we just all went our separate ways and then about a year later came back together and felt this was just too good of an idea to drop."

Did you expect Courier to come to market, or in the back of your mind were you thinking, 'This is all just fantasy to see what we can come up with'?

"The goal was to make something real we could bring to market. The videos that were leaked were early interaction studies, but that was always the goal to bring something to market. That’s something we were always tasked with doing. 

"I say yeah the goal was to bring it to market, but honestly every R&D effort there’s a hope you bring it to market. I’m not saying Courier was ready for market, but it was just like every other R&D project out there."

What were some problems you saw with the stylus market and what prompted you to jump in?

"It totally transforms the experience, even with a regular passive stylus. That’s why we’ve been recommending since day one people use at least something. It’s just... we grow and evolve using tools. It’s a natural extension of what we can do. Touch is great for consumption interfaces like navigating, flipping through magazines, browsing websites. When it comes time to make something, touch falls short there.

"When we looked at the stylus market and what we wanted to do with Pencil, you can look at two ends of the spectrum. On one end you could build the ballpoint pen, the functional note-taking, precise type of stylus. Or on the other end, something much more masterful, much more expressive like the paintbrush. But then fewer people would be able to use it. Similar to the way we looked at the space for Paper, we did the same thing with Pencil and found that the best thing was not an extreme, but something right in the middle. Something that had that simple functionality... that offered a lot of expressiveness. 

"Then we looked at that feature set and tried to make it like the pencil you had on your desk right now. It had to be able to connect really quickly and really seamlessly. Most styluses that connect usually have this crazy Bluetooth dance where you have to connect, with a few switches on the iPad and switches on the pen itself and then hope things connect. 

"We spent a lot of time developing a new Bluetooth-pairing gesture that we call 'kiss to pair'. What you do there is just tap and hold the screen with the Pencil and two to three seconds later it connects. Another big thing was palm rejection. You’ll see as soon as you hand a stylus and iPad to someone, someone hasn’t really used one before, they’ll put their hand down and screw it up. Because the iPad just sees every touch the same way, there’s no distinction between stylus and touch input. We wanted to build the best palm recognition out there and we really took it up a notch."

How does the palm rejection work?

"Basically there’s a set of heuristics we look at and run through a bayesian model to look to predict what you’re doing. 

"There’s a switch in the tip of the pencil, so when you touch down it activates the switch and communicates with the iPad the touch you’re seeing should be a pencil. That works most of the time, but there’s also other aspects we look at - like what we think your hand is. We can determine if you’re a lefty or righty. You'll notice when you use Paper and Pencil there are no settings. You don’t have to go in and collaborate, it just works. You can flip the iPad like you would just like a piece of paper. We had two PhDs working on Pencil, and it’s by far the most advanced palm rejection out there."

Who’s behind the tech?

"Our hardware team is super experienced; people that have spent decades in the hardware business. Jon Harris is one of the founders who heads up the design division, and he worked on Zune and Xbox at Microsoft, and worked at HP. He’s been in the business for a long time. John Ikeda worked on the Xbox One, Kinect, and all the Xbox accessories at Microsoft. These guys know hardware inside and out. It was just a matter of finding the right partners to help us."

You guys sound like the second-coming of Microsoft, honestly. Is that what you’re going for?

*Laughs* "I don’t think so... But you know… It sounds good in a headline. I don’t know, Microsoft seems like it’s having a second coming soon."

How important was the price point for Pencil? Is it aimed at professionals or are you going for people who can barely draw at all, but want to mess around on their iPad?

"Yeah, I think we’re going for [the broader audience]. Paper is built for that. Pencil is built for that. It’s not just the price point but the simplicity. The design and aesthetics. It’s the functionality. It’s everything you can package up together into one beautiful product. It’s trying to open up creative to the boarder audience. There’s really a big opportunity to build hardware, software, and service experiences that are really integrated, that can innovative on the interaction, take it to a new level, and allow these really different type of creative input for everyday people. Everyone’s creative. We all have our creative minds." 

Why do you think the market is drawn to styluses? What’s the big deal? Is it taking you back to the Paper and Pencil past? 

"I think we have these amazing mobile devices now. It has been great for consumption, but now we’re actually wanting to start creating something. Once you start creating something meaningful and not just using the technology to browse Facebook or play Angry Birds, once you start making something, you have to change the input. I think the door is opening now for all different types of hardware for these mobile devices. It’s really driven by that need to want to make.

"If you’re on a train somewhere and have a great idea, you can capture it and create it."

What’s next for FiftyThree? 

"The name FiftyThree really comes from the 53centime ters that is your arm’s reach. The products we create are the things that live within your arm’s reach. It’s various digital tools that connect things with the heart and the hand. We’ve always said we’re very interested in building hardware, software and services. We’ve seen software with Paper, we’ve seen hardware with Pencil, the next piece for us is around web services. 

Image credit: Flickr

"The way we’re thinking about services is that Paper is this really personal place where you capture your own thoughts, but for an idea to really have impact it has to go out there. It has to be shared, and so that’s the piece we’re looking at. A web service as a way of bringing ideas together."

A social network for creatives? 

"Something… along those lines. Somehow bringing those ideas together… To be determined."