Part of the Institute of Computer Graphics @ Johannes Kepler University Linz, Austria

Comparative Evaluations of Visualization Onboarding Methods

CompareEvaluations Teaser

Abstract

Comprehending and exploring large and complex data is becoming increasingly important for users in a wide range of application domains. Still, non-experts in visual data analysis often have problems with correctly reading and interpreting information from visualizations that are new to them. To support novices in learning how to use new digital technologies, the concept of onboarding has been successfully applied in other fields and first approaches also exist in the visualization domain. However, empirical evidence on the effectiveness of such approaches is scarce. Therefore, we conducted 3 studies: 1) Firstly, we explored the effect of vis onboarding, using an interactive step-by-step guide, on user performance for four increasingly complex visualization techniques. We performed a between-subject experiment with 596 participants in total. The results showed that there are no significant differences between the answer correctness of the questions with and without onboarding. Furthermore, participants commented that for highly familiar visualization types no onboarding is needed. 2) Second, we performed another study with MTurk workers to assess if there is a difference in user performances on different onboarding types: step-by-step, scrollytelling tutorial, and video tutorial. The study revealed that the video tutorial was ranked as the most positive on average, based on sentiment analysis, followed by the scrollytelling tutorial and the interactive step-by-step guide. 3) For our third study with students, we gathered data on users' experience in using an in-situ scrollytelling for the VA tool. The results showed that they preferred scrollytelling over the tutorial integrated into the landing page. In summary, the in-situ scrollytelling approach works well for visualization onboarding and a video tutorial can help to introduce interaction techniques.


Citation

Christina Stoiber, Conny Walchshofer, Margit Pohl, Benjamin Potzmann, Florian Grassinger, Holger Stitz, Marc Streit, Wolfgang Aigner
Comparative Evaluations of Visualization Onboarding Methods
Visual Informatics, 6(4): 34--50, doi:10.1016/j.visinf.2022.07.001, 2022.

BibTeX

@article{2022_visinf-compare-eval,
    title = {Comparative Evaluations of Visualization Onboarding Methods},
    author = {Christina Stoiber and Conny Walchshofer and Margit Pohl and Benjamin Potzmann and Florian Grassinger and Holger Stitz and Marc Streit and Wolfgang Aigner},
    journal = {Visual Informatics},
    publisher = {Elsevier},
    doi = {10.1016/j.visinf.2022.07.001},
    url = {https://doi.org/10.1016/j.visinf.2022.07.001},
    volume = {6},
    number = {4},
    pages = {34--50},
    year = {2022}
}

Acknowledgements

The authors wish to thank Victor Adriel de Jesus Oliveira for his valuable feedback. This work was funded by the Austrian Ministry for Transport, Innovation and Technology (BMVIT) under the ICT of the Future program via the SEVA project (no. 874018), as well as the FFG, Contract No. 881844: “Pro2Future is funded within the Austrian COMET Program Competence Centers for Excellent Technologies under the auspices of the Austrian Federal Ministry for Climate Action, Environment, Energy, Mobility, Innovation and Technology, the Austrian Federal Ministry for Digital and Economic Affairs and of the Provinces of Upper Austria and Styria. COMET is managed by the Austrian Research Promotion Agency FFG.”