Keywords: Artificial Intelligence, Event Log Completion, Event Log Creation, Event Log Quality Improvement, Multimodal data
Astract: Process mining uses data from event logs to understand which activities were undertaken, their timing, and the involved entities, providing a data trail for process analysis and improvement. However, a significant challenge involves ensuring that these logs accurately reflect the actual processes. Some processes leave few digital traces, and their event logs often lack details about manual and physical work that does not involve computers or simple sensors. We introduce the Business-knowledge Integration Cycles (BICycle) method and mm_proc_miner tool to convert raw and unstructured data from various modalities, such as video, audio, and sensor data, into a structured and unified event log, while keeping human-in-the-loop. Our method analyzes the semantic distance between visible, audible, and textual evidence within a self-hosted joint embedding space. Our approach is designed to consider (1) preserving the privacy of evidence data, (2) achieving real-time performance and scalability, and (3) preventing AI hallucinations. We also publish a dataset consisting of over 2K processes with 16K steps to facilitate domain inference-related tasks. For the evaluation, we created a novel test dataset in the domain of DNA home kit testing, for which we can guarantee that it was not encountered during the training of the employed AI foundational models. We show positive insights in both event log enrichment with multimodal evidence and human-in-the-loop contribution.

Keywords: multi-modal AI, Conceptual Modeling, Comprehension, Large Language Models (LLMs)

Keywords: Process Models, Transformer models, Multimodal Evidence, Process Representation
Astract: Stakeholders can struggle to understand and engage with process models due to a mismatch between the technical language used and their own domain-specific jargon and personal communication styles. The paper explores the application of transformer-based architectures to enhance the representation of process models and additional multimodal process data by tailoring them to the language of stakeholders. We present an approach that personalizes process model representations through two types of paraphrasers: one that aligns with domain-specific jargon and another that adapts to individual stakeholder styles. We developed a golden dataset from process model-stakeholder interaction simulation and a silver dataset using large language models to train and validate our approach. Initial findings suggest that these methods could enhance stakeholder engagement and contribute to better teaching of process mining and procedural thinking.

Keywords: Process Mining, Multi-Modal Segmentation, Conceptual Modeling

Keywords: Conceptual Modeling, Analytical models, Systematics, Machine Learning algorithms, Bibliographies, Semantics, Machine Learning
Astract: Conceptual models are essential in Software and Information Systems Engineering to meet many purposes since they explicitly represent the subject domains. Machine Learning (ML) approaches have recently been used in conceptual modeling to realize, among others, intelligent modeling assistance, model transformation, and metamodel classification. These works en-code models in various ways, making the encoded models suitable for applying ML algorithms. The encodings capture the models' structure and/or semantics, making this information available to the ML model during training. Therefore, the choice of the encoding for any ML-driven task is crucial for the ML model to learn the relevant contextual information. In this paper, we report findings from a systematic literature review which yields insights into the current research in machine learning for conceptual modeling (ML4CM). The review focuses on the various encodings used in existing ML4CM solutions and provides insights into i) which are the information sources, ii) how is the conceptual model's structure and/or semantics encoded, iii) why is the model encoded, i.e., for which conceptual modeling task and, iv) which ML algorithms are applied. The results aim to structure the state of the art in encoding conceptual models for ML.