Robotic front-end web jockeys

We’re starting to see some more experiments in using automation and AI in the web design process. Microsoft have open sourced a system that takes hand drawn sketches and generates working code. This is clearly aimed at demonstrating the Azure platform’s ML and AI capabilities (The Register wrote about it here - note the use of the term front-end web jockeys).

A similar experiment by Airbnb works by translating sketches into components from their already well-defined design system. The approach is very different to Microsoft’s, because of the strong constraints imposed by the design system.

Translating roughly drawn sketches to a limited number of well-defined components with constrained property values will produce far more useful prototypes than training a system to author HTML and CSS from scratch. The authoring language is just far too broad and low-level.

More importantly, Airbnb’s design system embodies and abstracts a huge amount of the hard design decisions, user research, accessibility work and implementation expertise necessary to create a great experience.

Without this, early design automation and AI tools are likely to be more like the worst imaginable PSD2HTML service. Humans are also pretty bad at producing good work if we’re only given a sketch hurriedly drawn with a black Sharpie. We need to do more work to interpret the sketch: conversations, collaboration, research, formal design constraints, component libraries, platform expertise and the application of hard-won experience.

There’s no need to be worried that your job is under threat just yet. But it is worthwhile examining the parts of our work that are good early targets for automation, to see where the industry may be heading, and to understand where the real value is in the work we do.

What part of your job are you most worried that the robots will steal?

All the best,

– Jim