Skip to content. | Skip to navigation

Personal tools
You are here: Home Publications UISketch – a large-scale dataset of UI element sketches


Prof. Dr. S. Decker
RWTH Aachen
Informatik 5
Ahornstr. 55
D-52056 Aachen
Tel +49/241/8021501
Fax +49/241/8022321

How to find us

Annual Reports





UISketch – a large-scale dataset of UI element sketches

Year 2021
Abstract URL view
ISBN-13 9781450380966

This paper contributes the first large-scale dataset of 17,979 hand-drawn sketches of 21 UI element categories collected from 967 participants, including UI/UX designers, front-end developers, HCI, and CS grad students, from 10 different countries. We performed a perceptual study with this dataset and found out that UI/UX designers can recognize the UI element sketches with ~96% accuracy. To compare human performance against computational recognition methods, we trained the state-of-the-art DNN-based image classification models to recognize the UI elements sketches. This study revealed that the ResNet-152 model outperforms other classification networks and detects unknown UI element sketches with 91.77% accuracy (chance is 4.76%). We have open-sourced the entire dataset of UI element sketches to the community intending to pave the way for further research in utilizing AI to assist the conversion of lo-fi UI sketches to higher fidelities.


CHI '21: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (Yokohama 2021)


Presented at

CHI, 2021 , Yokohama , JP.

Published in

Proceedings ACM-SIGCHI Conference on Human Factors in Computer Systems , p. 1-14 ; ACM Press , New York , US .

Document Actions