Hands-On Versus Demonstration-Based Identity and Access Management Training for Cloud Engineers: A Randomised Controlled Trial

Abstract

Identity and access management errors remain a common cause of cloud security incidents, yet training often relies on instructor demonstrations rather than repeated configuration practice. We conducted a parallel-group randomised controlled trial with 118 early-career cloud engineers assigned to hands-on laboratory training or demonstration-based instruction covering role design, least privilege, policy evaluation, and access troubleshooting. Knowledge, configuration accuracy, task completion time, and transfer to novel scenarios were assessed immediately and four weeks later. Both groups improved from baseline, but laboratory participants achieved higher post-training configuration accuracy (86% versus 72%) and completed novel access-control tasks more successfully at follow-up. Knowledge-test differences were smaller than performance differences, suggesting that conceptual understanding did not fully reflect operational competence. Hands-on learners initially required more training time but made fewer privilege-escalation and policy-scope errors during transfer tasks. The findings support practical configuration exercises when the learning objective is reliable IAM implementation. Demonstration remains useful for introducing concepts, but assessment should include performance on unseen configurations rather than relying exclusively on recall-based tests.