Screen2vec: Semantic embedding of gui screens and gui components

TJJ Li, L Popowski, T Mitchell, BA Myers - Proceedings of the 2021 CHI …, 2021 - dl.acm.org
Representing the semantics of GUI screens and components is crucial to data-driven
computational methods for modeling user-GUI interactions and mining GUI designs. Existing …

Multi-modal repairs of conversational breakdowns in task-oriented dialogs

TJJ Li, J Chen, H Xia, TM Mitchell… - Proceedings of the 33rd …, 2020 - dl.acm.org
A major problem in task-oriented conversational agents is the lack of support for the repair of
conversational breakdowns. Prior studies have shown that current repair strategies for these …

Pumice: A multi-modal agent that learns concepts and conditionals from natural language and demonstrations

TJJ Li, M Radensky, J Jia, K Singarajah… - Proceedings of the …, 2019 - dl.acm.org
Natural language programming is a promising approach to enable end users to instruct new
tasks for intelligent agents. However, our formative study found that end users would often …

Supporting end-user debugging of trigger-action rules for IoT applications

M Manca, F Paternò, C Santoro, L Corcella - International Journal of …, 2019 - Elsevier
End users need tools to enable them to control and personalise Internet of Things (IoT)
applications, which may involve hundreds of interconnected objects. Trigger-action …

End-user development for personalizing applications, things, and robots

F Paternò, C Santoro - International Journal of Human-Computer Studies, 2019 - Elsevier
The pervasiveness of ICT technologies has led to a growing need to empower people to
obtain applications that meet their specific requirements. End-User Development (EUD) is a …

From awareness to action: Exploring end-user empowerment interventions for dark patterns in ux

Y Lu, C Zhang, Y Yang, Y Yao, TJJ Li - … of the ACM on Human-Computer …, 2024 - dl.acm.org
The study of UX dark patterns, ie, UI designs that seek to manipulate user behaviors, often
for the benefit of online services, has drawn significant attention in the CHI and CSCW …

Novelette, a usable visual storytelling digital learning environment

A Addone, R De Donato, G Palmieri… - IEEE …, 2021 - ieeexplore.ieee.org
Educational digital storytelling is widely recognised as a powerful approach in developing
literary skills, experiencing problem-solving and computational thinking, information and …

Interactive task learning from GUI-grounded natural language instructions and demonstrations

TJJ Li, T Mitchell, B Myers - … of the 58th Annual Meeting of the …, 2020 - aclanthology.org
We show SUGILITE, an intelligent task automation agent that can learn new tasks and
relevant associated concepts interactively from the user's natural language instructions and …

To reuse or not to reuse? A framework and system for evaluating summarized knowledge

MX Liu, A Kittur, BA Myers - Proceedings of the ACM on Human …, 2021 - dl.acm.org
As the amount of information online continues to grow, a correspondingly important
opportunity is for individuals to reuse knowledge which has been summarized by others …

New perspectives for cyber security in software development: when end-user development meets artificial intelligence

V Santa Barletta, F Cassano, A Pagano… - … on Innovation and …, 2022 - ieeexplore.ieee.org
Current research faces a" war" between Human-Computer Interaction and Artificial
Intelligence. Despite the shared goal to support the user in his/her choices, researchers …