Keywords: Process Mining

Keywords: Artificial Intelligence, Event Log Completion, Event Log Creation, Event Log Quality Improvement, Multimodal data
Astract: Process mining uses data from event logs to understand which activities were undertaken, their timing, and the involved entities, providing a data trail for process analysis and improvement. However, a significant challenge involves ensuring that these logs accurately reflect the actual processes. Some processes leave few digital traces, and their event logs often lack details about manual and physical work that does not involve computers or simple sensors. We introduce the Business-knowledge Integration Cycles (BICycle) method and mm_proc_miner tool to convert raw and unstructured data from various modalities, such as video, audio, and sensor data, into a structured and unified event log, while keeping human-in-the-loop. Our method analyzes the semantic distance between visible, audible, and textual evidence within a self-hosted joint embedding space. Our approach is designed to consider (1) preserving the privacy of evidence data, (2) achieving real-time performance and scalability, and (3) preventing AI hallucinations. We also publish a dataset consisting of over 2K processes with 16K steps to facilitate domain inference-related tasks. For the evaluation, we created a novel test dataset in the domain of DNA home kit testing, for which we can guarantee that it was not encountered during the training of the employed AI foundational models. We show positive insights in both event log enrichment with multimodal evidence and human-in-the-loop contribution.

Keywords: multi-modal AI, Conceptual Modeling, Comprehension, Large Language Models (LLMs)

Keywords: Process Models, Transformer models, Multimodal Evidence, Process Representation
Astract: Stakeholders can struggle to understand and engage with process models due to a mismatch between the technical language used and their own domain-specific jargon and personal communication styles. The paper explores the application of transformer-based architectures to enhance the representation of process models and additional multimodal process data by tailoring them to the language of stakeholders. We present an approach that personalizes process model representations through two types of paraphrasers: one that aligns with domain-specific jargon and another that adapts to individual stakeholder styles. We developed a golden dataset from process model-stakeholder interaction simulation and a silver dataset using large language models to train and validate our approach. Initial findings suggest that these methods could enhance stakeholder engagement and contribute to better teaching of process mining and procedural thinking.

Keywords: Process Mining, Multi-Modal Segmentation, Conceptual Modeling