Design and AI: Fret or Not?

I came across this tweet thread today.

Screen Shot 2020-10-28 at 11.06.50 am.png

The thread basically questions the idea of Bohemian Coding’s Sketch having good reasons to be a native, rather than Mac app. Yes, the web is more accessible. Yes Figma (the competition) does a much better job in some areas, especially anything to do with collaboration. It’s faster in some ways, and has some fantastic vector tools, and other unique things.

On the plus-side for Sketch, I love the fact that Sketch not only fits, but enables all kinds of flow in my Mac experience. I can drag and drop stuff in and out with zero friction. But I don’t think that’s the killer reason for Sketch to double-down on the native experience.

And it has to do with Rock’n’Roll, baby. Let’s get started.

Clippit, or Clippy, the annoying Microsoft Helper.

Clippit, or Clippy, the annoying Microsoft Helper.

It’s looking pretty inevitable that Machine Learning/AI is going to play a big part in all of our computing life. In my opinion, I’d like to think I know what this doesn’t look like. It doesn’t look like Clippit, the unhelpful Microsoft paperclip. It’s not a helper, and it’s not a one-click solution.

Have you ever hit the DEMO button on a small Casio music keyboard? Yes, you can press it, and play music. But it’s not the experience we’re hoping for. Instead, I think of great ML being like a fret on a guitar. If you don’t know it, the frets are the vertical lines between where you put your fingers on a guitar’s neck. They make it easy to choose notes on each string. There are many stringed instruments that don’t have frets—like violins, cellos, double basses. There are still fretless guitars, but they’re very much the niche. The fretted guitar is the winning instrument in terms of popularity for most people.

Frets on a guitar neck

Frets on a guitar neck

A fret is very much a constraint that you work with as you play. It helps guide where to put your fingers, and once you hold down your fingers, it guarantees that the notes will be in tune. In this way, playing a guitar is much easier than playing a violin, since you can start producing pleasant (if not simple) music in just days, rather than months or even years.

Note that the fret doesn’t play the music for you. You don’t interact with a fret and wait a second for the result to come back. And it’s not simple an on/off button. It’s a set of edges, shapes that you can interact with in infinite ways.

And this unlocks whole other art forms. Because you can play full, rich chords reliably, it’s possible to sing songs with rich harmony. The guitar is a portable instrument that goes back to the Renaissance lute, and shares that long history of popular songwriters and singers.

A lute and player.

A lute and player.

And without the ability to play guitar and sing, you wouldn’t have Rock’n’Roll. And about a zillion other popular music styes. The fret, because it makes a clean strong connection with the string, also enables instruments like the electric guitar to have long sustained notes (think Jimmy Hendrix’s Star Spangled Banner), and complex tapping work.

Bringing this back to design, I see the coming age of integrated ML best implemented likes frets on a guitar. It is not a Casio Keyboard’s demo button, instead a rich interaction that the creator works intimately with to enable faster creation, and whole new art forms. And we are seeing Apple, in particular, invest incredibly heavily in building powerful Neural engines into each and every iPhone and iPad. And soon, I expect, the Mac.

This is where I think Sketch can shine. This is where you want the lowest latency possible, and the biggest bandwidth between you and the tool.

The web is an amazing platform. Figma is a fantastic tool. And I love competition between design tools. Trust me, doing UI design in Photoshop sucked in comparison.

What I want is my design tools to be tactile, rich, solid experiences. And I think there’s a fantastic opportunity for native, close-to-the-metal apps to deliver this.