Learning expert workflows
A staged method for teaching a model a billing workflow from recorded expert work. Supervised fine-tuning establishes task knowledge; reinforcement learning introduces process-based feedback for refinement.
Read the paperRESEARCH / DIGITAL HUMAN
From screen recordings to training data
The preparation pipeline that turns recorded work into usable training examples, connecting actions to goals and process knowledge while checking that evaluation cases and answers stay separate.
Read on this pageAt a glance
Expert demonstrations need preparation before they become useful training data. This paper describes how screenshots and interface events from real work are organised into cases, connected to a written process guide, and converted into examples for several related learning tasks.
The central challenge is keeping the answer out of the model's question. A preparation system may need the full case history to create a good label, while the trained model must act with only the information available at that moment. The paper describes checks that remove this extra context and keep cases separate across training and evaluation. It offers an account of how to teach actions, goals, screen interpretation and process knowledge from the same body of recorded work.
What the paper covers
Case boundaries and process knowledge give raw screen recordings a useful structure.
One collection of demonstrations supports action prediction, goal prediction and workflow questions.
Checks remove surplus context and identify remaining label defects before training.
Research context
This paper covers data preparation rather than downstream model performance. It identifies copyable goal labels and residual question-answer defects, and does not independently verify every generated label against business reality. Those limits must be considered when designing a clean evaluation.
THE ORIGINAL WHITEPAPER
Download PDF Read it here.
The complete paper, with its original figures, tables and references.
32 pages · Read without leaving this pageFROM RESEARCH TO YOUR WORK
Discuss this research Bring your questions.
Discuss the approach, evidence and relevance to your team with the people building it.