Training Transfer from Sandbox Environments to Production Storage Operations: A Prospective Cohort Study

Authors

  • Khalid Alvi Department of Information Systems and Human Factors, University of Zurich Author
  • Mina Demir Department of Information Systems and Human Factors, KU Leuven Author

Abstract

Training environments simplify risk and access constraints, raising questions about whether skills demonstrated in a sandbox transfer to production storage operations. We prospectively followed 76 engineers for six months after completing a practical storage-automation programme. Sandbox performance was recorded at course completion, while production transfer was assessed from supervised change records, configuration-review findings, incident reports, and manager-rated independence. Strong sandbox performers generally completed production tasks more accurately, but the relationship weakened for changes involving legacy dependencies and organisational approval processes. Participants who completed at least three supervised production changes within the first month reached independent-operation criteria a median of four weeks earlier than those with delayed exposure. The most persistent transfer errors involved environment discovery, exception handling, and rollback preparation rather than command syntax. Confidence immediately after training was not a reliable predictor of safe independent work. Practical training provided a useful foundation, but transfer depended on structured exposure to production complexity. Organisations should pair sandbox instruction with graduated supervision, real-system observation, and feedback on contextual decision-making.

References

1. Nazir M. Automating enterprise storage provisioning with PowerShell and Python: effects on deployment time and configuration errors. Journal of Multidisciplinary Research and Integrated Sciences. 2021;1(1). Available from: https://jmris.com/index.php/jmris/article/view/2021P1V6

2. Rose S, Borchert O, Mitchell S, Connelly S. Zero trust architecture. Gaithersburg (MD): National Institute of Standards and Technology. 2020;NIST SP 800-207. doi:10.6028/NIST.SP.800-207.

3. Swanson M, Bowen P, Phillips AW, Gallup D, Lynes D. Contingency planning guide for federal information systems. Gaithersburg (MD): National Institute of Standards and Technology. 2010;NIST SP 800-34 Rev. 1. doi:10.6028/NIST.SP.800-34r1.

4. Grance T, Nolan T, Burke K, Dudley R, White G, Good T. Guide to test, training, and exercise programs for IT plans and capabilities. Gaithersburg (MD): National Institute of Standards and Technology. 2006;NIST SP 800-84. doi:10.6028/NIST.SP.800-84.

5. Petersen R, Santos D, Wetzel K, Smith MC, Witte G. Workforce framework for cybersecurity (NICE Framework). Gaithersburg (MD): National Institute of Standards and Technology. 2020;NIST SP 800-181 Rev. 1. doi:10.6028/NIST.SP.800-181r1.

6. Sweller J. Cognitive load during problem solving: effects on learning. Cogn Sci. 1988;12(2):257-285. doi:10.1207/s15516709cog1202_4.

7. Salas E, Tannenbaum SI, Kraiger K, Smith-Jentsch KA. The science of training and development in organizations: what matters in practice. Psychol Sci Public Interest. 2012;13(2):74-101. doi:10.1177/1529100612436661.

8. Sitzmann T. A meta-analytic examination of the instructional effectiveness of computer-based simulation games. Pers Psychol. 2011;64(2):489-528. doi:10.1111/j.1744-6570.2011.01190.x.

9. Hart SG, Staveland LE. Development of NASA-TLX (Task Load Index): results of empirical and theoretical research. Adv Psychol. 1988;52:139-183. doi:10.1016/S0166-4115(08)62386-9.

10. Hattie J, Timperley H. The power of feedback. Rev Educ Res. 2007;77(1):81-112. doi:10.3102/003465430298487.

11. Freeman S, Eddy SL, McDonough M, Smith MK, Okoroafor N, Jordt H, et al. Active learning increases student performance in science, engineering, and mathematics. Proc Natl Acad Sci U S A. 2014;111(23):8410-8415. doi:10.1073/pnas.1319030111.

12. Salas E, DiazGranados D, Klein C, Burke CS, Stagl KC, Goodwin GF, et al. Does team training improve team performance? A meta-analysis. Hum Factors. 2008;50(6):903-933. doi:10.1518/001872008X375009.

13. Baldwin TT, Ford JK. Transfer of training: a review and directions for future research. Pers Psychol. 1988;41(1):63-105. doi:10.1111/j.1744-6570.1988.tb00632.x.

14. Arthur W Jr, Bennett W Jr, Edens PS, Bell ST. Effectiveness of training in organizations: a meta-analysis of design and evaluation features. J Appl Psychol. 2003;88(2):234-245. doi:10.1037/0021-9010.88.2.234.

15. Lakens D. Sample size justification. Collabra Psychol. 2022;8(1):33267. doi:10.1525/collabra.33267.

16. Nosek BA, Ebersole CR, DeHaven AC, Mellor DT. The preregistration revolution. Proc Natl Acad Sci U S A. 2018;115(11):2600-2606. doi:10.1073/pnas.1708274114.

17. Lakens D. Calculating and reporting effect sizes to facilitate cumulative science: a practical primer for t-tests and ANOVAs. Front Psychol. 2013;4:863. doi:10.3389/fpsyg.2013.00863.

18. Koo TK, Li MY. A guideline of selecting and reporting intraclass correlation coefficients for reliability research. J Chiropr Med. 2016;15(2):155-163. doi:10.1016/j.jcm.2016.02.012.

Issue5

Published

2024-06-01

Issue

Vol. 5 No. 01 (2024): Generative Intelligence, Cyber-Physical Systems and Advanced Information Technologies

Section

Articles

License

Authors retain copyright. Articles published under the Creative Commons Attribution 4.0 International licence (CC BY 4.0) may be shared and adapted for any purpose, including commercially, provided appropriate credit is given, a link to the licence is supplied, and changes are indicated. No additional legal or technological restrictions may be imposed. Third-party material is included only where its credit line permits. Licence: https://creativecommons.org/licenses/by/4.0/. Earlier publications remain subject to their stated licence and author agreements unless the rights holder authorizes a change.