Researchers test accuracy of 10 LLMs for process modeling

Researchers have assessed the accuracy of 10 state-of-the-art artificial intelligence (AI) large language models (LLMs) in generating process models from standard and deliberately atypical process evidence. The study introduced and investigated the phenomenon of knowledge-driven hallucination in LLMs, where a model’s pretrained knowledge overrides explicit source evidence, leading to factually incorrect but plausible-looking outputs.

It found that LLMs exhibit a tendency to prioritize their generalized internal schemas over contradictory evidence provided in the prompt. This was evident as models frequently reverted to generating a standard process flow even when the input described a reversed or structurally shuffled version.

Even in cases where the models did not fully hallucinate, their performance in correctly modeling atypical processes was significantly degraded compared to standard ones, highlighting the disruptive effect of the conflict between evidence and internal knowledge.

Join the PEX Network community

Don't miss any news, updates or insider tips from PEX Network by getting them delivered to your inbox. Sign up to our newsletter and join our community of experts. 

Learn More

Implications extend beyond process modeling

The implications of the study extend far beyond process modeling and raise critical concerns about the reliability of LLMs in any evidence-based domain, according to the researchers 

The danger of knowledge-driven hallucination lies in its deceptive nature, they added. Generated artifacts are often coherent, logical and well-formed, masking the fact that they do not accurately represent the source data.

“This ‘plausibility trap’ poses a significant risk in fields such as legal analysis, financial reporting and scientific research, where strict adherence to source evidence is essential,” the paper read.

The work underscores that simple prompt engineering, such as instructing the model to be faithful to the input, can mitigate but not eliminate this deep-seated behavior.

Future work should focus on two key areas, the researchers noted. First, there is a clear need to develop more robust mitigation techniques beyond prompting that allow for better control over the influence of pre-trained knowledge.

Second, the experimental methodology could be adapted to investigate knowledge-driven hallucination in other structured generation tasks, such as code generation from legacy specifications or data schema creation from business requirements.

“Ultimately, our study serves as a critical reminder that as we delegate more complex analytical tasks to AI, we must also develop rigorous methods to validate its outputs and ensure that its powerful inferential capabilities do not come at the cost of factual integrity.”

Register for All Access: AI in Business Transformation 2025!

Process modeling in business transformation

The PEX Report 2025/26, based on the results of a survey of more than 200 respondents, found that 38 percent of organizations use process modeling/mapping tools to support business transformation. Meanwhile, 26 percent plan to increase investment in process modeling/mapping tools in the next 12 months.

At All Access: BPM Business Process Management, process modeling topped the list of solutions deployed by participants.

These survey results underscore a clear truth: process is at the heart of business transformation, said Ryan Bazler, senior director, product marketing at Nintex. “The fact that workflow automation and process modeling/mapping are at the top of the list shows that organizations recognize they can’t transform effectively without first understanding and streamlining how work gets done.”

Interestingly, the data also shows a maturity curve, Bazler added. “Higher adoption of modeling, workflow automation and reengineering suggest businesses are focusing on building strong process foundations before layering on more advanced capabilities like RPA [robotic process automation], process mining or digital twins.” This progression makes sense; without well-defined, optimized processes, adding advanced automation or intelligence is like building on shaky ground.

All Access: Future of BPM 2026

You asked, and we listened. Business process management (BPM) remains the cornerstone technology for driving organizational transformation, according to the survey results featured in the latest PEX Report. As we look toward 2026 and beyond, generative AI, agentic AI, and intelligent process orchestration are redefining how processes are designed, executed, and optimized. BPM is your key to adapting swiftly and effectively in this new era.

PEX Network is bringing together industry leaders, technology innovators, and thought leaders to answer your biggest questions and explore the advancements reshaping business today. And you're invited. Register for free to save your spot now! 

Register Now