Skip to content. | Skip to navigation

Sections
Personal tools
You are here: Home Publications SynZ: Enhanced Synthetic Dataset for Training UI Element Detectors

Contact

Prof. Dr. S. Decker
RWTH Aachen
Informatik 5
Ahornstr. 55
D-52056 Aachen
Tel +49/241/8021501
Fax +49/241/8022321

How to find us

Annual Reports

Disclaimer

Webmaster

 

 

SynZ: Enhanced Synthetic Dataset for Training UI Element Detectors

Year 2021
Abstract URL view
ISBN-13 9781450380188

User Interface (UI) prototyping is an iterative process where designers initially sketch UIs before transforming them into interactive digital designs. Recent research applies Deep Neural Networks (DNNs) to identify the constituent UI elements of these UI sketches and transform these sketches into front-end code. Training such DNN models requires a large-scale dataset of UI sketches, which is time-consuming and expensive to collect. Therefore, we earlier proposed Syn to generate UI sketches synthetically by random allocation of UI element sketches. However, these UI sketches are not statistically similar to real-life UI screens. To bridge this gap, in this paper, we introduce the SynZ dataset, which contains 175,377 synthetically generated UI sketches statistically similar to real-life UI screens. To generate SynZ, we analyzed, enhanced, and extracted annotations from the RICO dataset and used 17,979 hand-drawn UI element sketches from the UISketch dataset. Further, we fine-tuned a UI element detector with SynZ and observed that it doubles the mean Average Precision of UI element detection compared to the Syn dataset.

Details

IUI '21: 26th International Conference on Intelligent User Interfaces, College Station, TX, USA, April 13-17, 2021, Companion.

Authors

Presented at

26th International Conference on Intelligent User Interfaces, 2021 , College Station, Texas , US.

Published in

26th International Conference on Intelligent User Interfaces , by Tracy Hammond, Katrien Verbert, Dennis Parra , p. 67-69 ; ACM .

Document Actions