Explainable Infrastructure Investment Recommendations for Small and Medium-Sized Enterprises: A Human-in-the-Loop Evaluation
Abstract
Infrastructure investment tools can rank upgrade options using cost and reliability models, but opaque recommendations may be difficult for smaller organisations to trust or appropriately challenge. We evaluated an explainable decision-support system with 72 IT and finance professionals from small and medium-sized enterprises. Participants reviewed infrastructure investment cases using either model recommendations alone or recommendations accompanied by cost drivers, risk contributions, sensitivity ranges, and alternative scenarios. Explanations improved identification of cases where the model relied on uncertain assumptions and increased appropriate rejection of recommendations after deliberately perturbed inputs. Decision time increased modestly, but agreement with independently reviewed choices improved from 68% to 81%. Participants valued scenario comparisons more than feature-importance graphics, particularly when proposed investments involved resilience rather than direct revenue. Excessively detailed explanations reduced usability for non-technical reviewers. Human-in-the-loop infrastructure planning benefited most from concise, decision-relevant explanations that exposed assumptions and uncertainty. Explainability should therefore support challenge and revision of investment recommendations rather than merely justify a model-generated ranking.
References
1. Azam MA, et al. Advancing explainable artificial intelligence and predictive business analytics to strengthen digital transformation, operational resilience, and productivity among U.S. small and medium-sized enterprises. European Journal of Management, Economics and Business. 2024;1(1):70-94. Available from: https://hal.science/hal-05722649/
2. Bhujel K, Haque GMM, Ansari I, Azam MA, Jahid A. Business analytics maturity and competitive advantage: evidence from data-driven enterprises. The American Journal of Management and Economics Innovations. 2026;8(3):1-23. doi:10.37547/tajmei/Volume08Issue03-01.
3. Nazir M. Performance comparison of AWS EBS, EFS and S3 for enterprise cloud-storage workloads. International Journal of Business & Computational Sciences. 2022;2(1). Available from: https://ijbcs.org/index.php/IJBCS/article/view/2022-01-05
4. Haque GMM, Ansari I, Bhujel K, Jahid A, Azam MA. Digital transformation strategies and IT governance: aligning business value with technology investments. The American Journal of Management and Economics Innovations. 2026;8(3):24-48. doi:10.37547/tajmei/Volume08Issue03-02.
5. Bharadwaj A, El Sawy OA, Pavlou PA, Venkatraman N. Digital business strategy: toward a next generation of insights. MIS Q. 2013;37(2):471-482. doi:10.25300/MISQ/2013/37:2.3.
6. Melville N, Kraemer K, Gurbaxani V. Review: information technology and organizational performance: an integrative model of IT business value. MIS Q. 2004;28(2):283-322. doi:10.2307/25148636.
7. Vial G. Understanding digital transformation: a review and a research agenda. J Strateg Inf Syst. 2019;28(2):118-144. doi:10.1016/j.jsis.2019.01.003.
8. Podsakoff PM, MacKenzie SB, Lee JY, Podsakoff NP. Common method biases in behavioral research: a critical review of the literature and recommended remedies. J Appl Psychol. 2003;88(5):879-903. doi:10.1037/0021-9010.88.5.879.
9. Lundberg SM, Lee SI. A unified approach to interpreting model predictions. Adv Neural Inf Process Syst. 2017;30. Available from: https://arxiv.org/abs/1705.07874
10. Ribeiro MT, Singh S, Guestrin C. "Why should I trust you?": explaining the predictions of any classifier. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 2016. p. 1135-1144. doi:10.1145/2939672.2939778.
11. Guo C, Pleiss G, Sun Y, Weinberger KQ. On calibration of modern neural networks. Proc Mach Learn Res. 2017;70:1321-1330. Available from: https://proceedings.mlr.press/v70/guo17a.html
12. Jacovi A, Goldberg Y. Towards faithfully interpretable NLP systems: how should we define and evaluate faithfulness?. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 2020. p. 4198-4205. doi:10.18653/v1/2020.acl-main.386.
13. Sweller J. Cognitive load during problem solving: effects on learning. Cogn Sci. 1988;12(2):257-285. doi:10.1207/s15516709cog1202_4.
14. Koo TK, Li MY. A guideline of selecting and reporting intraclass correlation coefficients for reliability research. J Chiropr Med. 2016;15(2):155-163. doi:10.1016/j.jcm.2016.02.012.
15. Lakens D. Sample size justification. Collabra Psychol. 2022;8(1):33267. doi:10.1525/collabra.33267.
16. Nosek BA, Ebersole CR, DeHaven AC, Mellor DT. The preregistration revolution. Proc Natl Acad Sci U S A. 2018;115(11):2600-2606. doi:10.1073/pnas.1708274114.
17. Wilkinson MD, Dumontier M, Aalbersberg IJ, Appleton G, Axton M, Baak A, et al. The FAIR Guiding Principles for scientific data management and stewardship. Sci Data. 2016;3:160018. doi:10.1038/sdata.2016.18.
18. Lakens D. Calculating and reporting effect sizes to facilitate cumulative science: a practical primer for t-tests and ANOVAs. Front Psychol. 2013;4:863. doi:10.3389/fpsyg.2013.00863.
Published
Issue
Section
License
Authors retain copyright. Articles published under the Creative Commons Attribution 4.0 International licence (CC BY 4.0) may be shared and adapted for any purpose, including commercially, provided appropriate credit is given, a link to the licence is supplied, and changes are indicated. No additional legal or technological restrictions may be imposed. Third-party material is included only where its credit line permits. Licence: https://creativecommons.org/licenses/by/4.0/. Earlier publications remain subject to their stated licence and author agreements unless the rights holder authorizes a change.
