Large language model-brained gui agents: A survey

C Zhang, S He, J Qian, B Li, L Li, S Qin, Y Kang… - arXiv preprint arXiv …, 2024 - arxiv.org
GUIs have long been central to human-computer interaction, providing an intuitive and
visually-driven way to access and interact with digital systems. The advent of LLMs …

Flowy: Supporting UX Design Decisions Through AI-Driven Pattern Annotation in Multi-Screen User Flows

Y Lu, Z Tong, Q Zhao, Y Oh, B Wang, TJJ Li - arXiv preprint arXiv …, 2024 - arxiv.org
Many recent AI-powered UX design tools focus on generating individual static UI screens
from natural language. However, they overlook the crucial aspect of interactions and user …

Persist Or Abandon? Exploring Chinese Users' Continuance Intentions Toward AI Painting Tools

J Xu, X Ren, Y Huang, R Wu, Y Pan - IEEE Access, 2024 - ieeexplore.ieee.org
With the rapid development of Artificial Intelligence (AI) technology in the art field, AI painting
tools (AIPT) have attracted a high interest in the creation of painting art. However, the factors …

Limitations of the LLM-as-a-Judge Approach for Evaluating LLM Outputs in Expert Knowledge Tasks

A Szymanski, N Ziems, HA Eicher-Miller, TJJ Li… - arXiv preprint arXiv …, 2024 - arxiv.org
The potential of using Large Language Models (LLMs) themselves to evaluate LLM outputs
offers a promising method for assessing model performance across various contexts …

Generative AI in Multimodal User Interfaces: Trends, Challenges, and Cross-Platform Adaptability

J Bieniek, M Rahouti, DC Verma - arXiv preprint arXiv:2411.10234, 2024 - arxiv.org
As the boundaries of human computer interaction expand, Generative AI emerges as a key
driver in reshaping user interfaces, introducing new possibilities for personalized …

AI-Assisted Causal Pathway Diagram for Human-Centered Design

R Zhong, D Shin, R Meza, P Klasnja… - Proceedings of the CHI …, 2024 - dl.acm.org
This paper explores the integration of causal pathway diagrams (CPD) into human-centered
design (HCD), investigating how these diagrams can enhance the early stages of the design …

Misty: UI Prototyping Through Interactive Conceptual Blending

Y Lu, A Leung, A Swearngin, J Nichols… - arXiv preprint arXiv …, 2024 - arxiv.org
UI prototyping often involves iterating and blending elements from examples such as
screenshots and sketches, but current tools offer limited support for incorporating these …

Optimizing Accessibility in UX Design through the Integration of Artificial Intelligence

L Lundqvist - 2024 - diva-portal.org
This study aims to investigate whether artificial intelligence (AI) can be utilized in the user
experience design process to enhance the accessibility of websites for users with diverse …

AdvisorAI: Transforming Graphic Advertising with Generative AI-Enhanced Design Strategies

P Wu, X Li, Z He, L Fan - International Conference on Human-Computer …, 2024 - Springer
With the advancement of generative artificial intelligence, design is moving towards a new
era of intelligence. Generative AI models have demonstrated remarkable capabilities in …

Situated AI Usage: A Methodological Approach to Generative AI User Research

C Albrecht-Crane, D Comi, S Flanagan… - Proceedings of the 42nd …, 2024 - dl.acm.org
Students face ever-shifting, conflicting information about AI usage within and beyond the
university. On the one hand, AI literacy is framed as a necessary skill for students to learn in …