Flowy: Supporting UX Design Decisions Through AI-Driven Pattern Annotation in Multi-Screen User Flows

Y Lu, Z Tong, Q Zhao, Y Oh, B Wang, TJJ Li - arXiv preprint arXiv …, 2024 - arxiv.org
Many recent AI-powered UX design tools focus on generating individual static UI screens
from natural language. However, they overlook the crucial aspect of interactions and user …

Graphic Design with Large Multimodal Model

Y Cheng, Z Zhang, M Yang, H Nie, C Li, X Wu… - arXiv preprint arXiv …, 2024 - arxiv.org
In the field of graphic design, automating the integration of design elements into a cohesive
multi-layered artwork not only boosts productivity but also paves the way for the …

DynEx: Dynamic Code Synthesis with Structured Design Exploration for Accelerated Exploratory Programming

J Ma, K Sreedhar, V Liu, S Wang, PA Perez… - arXiv preprint arXiv …, 2024 - arxiv.org
Recent advancements in large language models have significantly expedited the process of
generating front-end code. This allows users to rapidly prototype user interfaces and ideate …

Misty: UI Prototyping Through Interactive Conceptual Blending

Y Lu, A Leung, A Swearngin, J Nichols… - arXiv preprint arXiv …, 2024 - arxiv.org
UI prototyping often involves iterating and blending elements from examples such as
screenshots and sketches, but current tools offer limited support for incorporating these …

Leveraging Multimodal LLM for Inspirational User Interface Search

S Park, Y Song, S Lee, J Kim, J Seo - arXiv preprint arXiv:2501.17799, 2025 - arxiv.org
Inspirational search, the process of exploring designs to inform and inspire new creative
work, is pivotal in mobile user interface (UI) design. However, exploring the vast space of UI …

Predicting the usability of mobile applications using AI tools: the rise of large user interface models, opportunities, and challenges

A Namoun, A Alrehaili, ZU Nisa, H Almoamari… - arXiv preprint arXiv …, 2024 - arxiv.org
Predicting the usability of mobile applications using AI tools: the rise of large user interface
models, opportunities, and chal Page 1 Available online at www.sciencedirect.com …

AssistantGPT: Enhancing User Interaction with LLM Integration

KB Neszlényi, A Milos, A Kiss - 2024 IEEE 22nd Jubilee …, 2024 - ieeexplore.ieee.org
As user interface technologies evolve, integrating voice and text commands becomes crucial
for enhancing interaction with operating systems. AssistantGPT leverages OpenAI's GPT API …