Explainable Artificial Intelligence Framework for Trustworthy Decision Support Systems

Ali Ahmad

doi:10.55927/fjst.v5i8.143

Authors

Ali Ahmad Universitas Darunnajah

DOI:

https://doi.org/10.55927/fjst.v5i8.143

Keywords:

Explainable Artificial Intelligence, Decision Support System, User Trust, Transparency, Thematic analysis.

Abstract

The application of artificial intelligence in decision support systems faces transparency challenges that can reduce user confidence in the results of recommendations. This research aims to develop an Explainable Artificial Intelligence framework that supports a transparent, understandable, and accountable decision support system. The research used an exploratory qualitative approach with semi-structured interviews with 20 informants consisting of AI developers, information systems experts, and professional users. The data was analyzed using thematic analysis to identify the need for explanation, confidence-building factors, and the characteristics of information that were considered relevant. The results show that clarity of decision reasons, consistency of explanations, traceability of processes, and suitability of context are the main elements of forming trust. This research produces an XAI conceptual framework that can be the basis for the development of a more transparent, reliable, and user-oriented decision support system

References

Adeoye-Olatunde, O. A., & Olenik, N. L. (2021). Research and scholarly methods: Semi-structured interviews. JACCP: Journal of the American College of Clinical Pharmacy, 4(10), 1358–1367. https://doi.org/10.1002/jac5.1441

Adler, R. H. (2022). Trustworthiness in qualitative research. Journal of Human Lactation, 38(4), 598–602. https://doi.org/10.1177/08903344221116620

Ahmad, M., & Wilkins, S. (2025). Purposive sampling in qualitative research: A framework for the entire journey. Quality & Quantity, 59, 1461–1479. https://doi.org/10.1007/s11135-024-02022-5

Amelia, A., & Sartika, F. (2025). Factors affecting customer trust in chatbot usage: Evidence from Indonesia. Journal of Indonesian Economy and Business, 40(3), 327–342. https://doi.org/10.22146/jieb.v40i3.11047

Andriati, S. L., Rizki, I. K., & Malian, A. N. B. M. (2024). Justice on trial: How artificial intelligence is reshaping judicial decision-making. Journal of Indonesian Legal Studies, 9(2), 909–942. https://doi.org/10.15294/jils.v9i2.13683

Antoniadi, A. M., Du, Y., Guendouz, Y., Wei, L., Mazo, C., Becker, B. A., & Mooney, C. (2021). Current challenges and future opportunities for XAI in machine learning-based clinical decision support systems: A systematic review. Applied Sciences, 11(11), Article 5088. https://doi.org/10.3390/app11115088

Aprianto, K., & Anasanti, M. D. (2025). Classifying heart disease through fusion of multi-source datasets: Integration of feature selection and explainable machine learning techniques. IJCCS (Indonesian Journal of Computing and Cybernetics Systems), 19(3), 247–258. https://doi.org/10.22146/ijccs.92395

Bayer, S., Gimpel, H., & Markgraf, M. (2022). The role of domain expertise in trusting and following explainable AI decision support systems. Journal of Decision Systems, 32(1), 110–138. https://doi.org/10.1080/12460125.2021.1958505

Bienefeld, N., Boss, J. M., Lüthy, R., Brodbeck, D., Azzati, J., Blaser, M., Willms, J., & Keller, E. (2023). Solving the explainable AI conundrum by bridging clinicians’ needs and developers’ goals. npj Digital Medicine, 6, Article 94. https://doi.org/10.1038/s41746-023-00837-4

Busroh, F. F., Khairo, F., Niravita, A., & Nunna, B. P. (2025). Digital transformation with the impact of AI in government decision making. Journal of Law and Legal Reform, 6(4), 1839–1874. https://doi.org/10.15294/jllr.v6i4.30339

Byrne, D. (2022). A worked example of Braun and Clarke’s approach to reflexive thematic analysis. Quality & Quantity, 56(3), 1391–1412. https://doi.org/10.1007/s11135-021-01182-y

Eke, C. I., & Shuib, L. (2025). The role of explainability and transparency in fostering trust in AI healthcare systems: A systematic literature review, open issues and potential solutions. Neural Computing and Applications, 37, 1999–2034. https://doi.org/10.1007/s00521-024-10868-x

Ferrario, A. (2024). Justifying our credences in the trustworthiness of AI systems: A reliabilistic approach. Science and Engineering Ethics, 30, Article 55. https://doi.org/10.1007/s11948-024-00522-z

Habiba, U.-e., Habib, M. K., Bogner, J., Fritzsch, J., & Wagner, S. (2025). How do ML practitioners perceive explainability? An interview study of practices and challenges. Empirical Software Engineering, 30, Article 18. https://doi.org/10.1007/s10664-024-10565-2

Hendryani, A., Nurdinawati, V., & Komarudin, A. (2025). Explainable artificial intelligence model for pneumonia detection: A hybrid CNN-ViT and Grad-CAM. Jurnal Nasional Teknik Elektro dan Teknologi Informasi, 14(4), 235–244. https://doi.org/10.22146/jnteti.v14i4.19822

Hennink, M., & Kaiser, B. N. (2022). Sample sizes for saturation in qualitative research: A systematic review of empirical tests. Social Science & Medicine, 292, Article 114523. https://doi.org/10.1016/j.socscimed.2021.114523

Jung, J., Lee, H., Jung, H., & Kim, H. (2023). Essential properties and explanation effectiveness of explainable artificial intelligence in healthcare: A systematic review. Heliyon, 9(5), Article e16110. https://doi.org/10.1016/j.heliyon.2023.e16110

Kim, J., Maathuis, H., & Sent, D. (2024). Human-centered evaluation of explainable AI applications: A systematic review. Frontiers in Artificial Intelligence, 7, Article 1456486. https://doi.org/10.3389/frai.2024.1456486

Kiseleva, A., Kotzinos, D., & De Hert, P. (2022). Transparency of AI in healthcare as a multilayered system of accountabilities: Between legal requirements and technical limitations. Frontiers in Artificial Intelligence, 5, Article 879603. https://doi.org/10.3389/frai.2022.879603

Kostopoulos, G., Davrazos, G., & Kotsiantis, S. (2024). Explainable artificial intelligence-based decision support systems: A recent review. Electronics, 13(14), Article 2842. https://doi.org/10.3390/electronics13142842

Markus, A. F., Kors, J. A., & Rijnbeek, P. R. (2021). The role of explainability in creating trustworthy artificial intelligence for health care: A comprehensive survey of the terminology, design choices, and evaluation strategies. Journal of Biomedical Informatics, 113, Article 103655. https://doi.org/10.1016/j.jbi.2020.103655

Mohseni, S., Zarei, N., & Ragan, E. D. (2021). A multidisciplinary survey and framework for design and evaluation of explainable AI systems. ACM Transactions on Interactive Intelligent Systems, 11(3–4), Article 24, 1–45. https://doi.org/10.1145/3387166

Naiseh, M., Al-Thani, D., Jiang, N., & Ali, R. (2021). Explainable recommendation: When design meets trust calibration. World Wide Web, 24(5), 1857–1884. https://doi.org/10.1007/s11280-021-00916-0

Naiseh, M., Al-Thani, D., Jiang, N., & Ali, R. (2023). How the different explanation classes impact trust calibration: The case of clinical decision support systems. International Journal of Human–Computer Studies, 169, Article 102941. https://doi.org/10.1016/j.ijhcs.2022.102941

Nauta, M., Trienes, J., Pathak, S., Nguyen, E., Peters, M., Schmitt, Y., Schlötterer, J., van Keulen, M., & Seifert, C. (2023). From anecdotal evidence to quantitative evaluation methods: A systematic review on evaluating explainable AI. ACM Computing Surveys, 55(13s), Article 295. https://doi.org/10.1145/3583558

Rismala, R., Maulidevi, N. U., & Surendro, K. (2025). Enhancing the comprehensiveness of criteria-level explanation in multi-criteria recommender system. Journal of Information Systems Engineering and Business Intelligence, 11(2), 160–172. https://doi.org/10.20473/jisebi.11.2.160-172

Scharowski, N., Perrig, S. A. C., Svab, M., Opwis, K., & Brühlmann, F. (2023). Exploring the effects of human-centered AI explanations on trust and reliance. Frontiers in Computer Science, 5, Article 1151150. https://doi.org/10.3389/fcomp.2023.1151150

Shin, D. (2021). The effects of explainability and causability on perception, trust, and acceptance: Implications for explainable AI. International Journal of Human–Computer Studies, 146, Article 102551. https://doi.org/10.1016/j.ijhcs.2020.102551

Susanto, A. P., Lyell, D., Widyantoro, B., Juzar, D. A., Santoso, A., Berkovsky, S., & Magrabi, F. (2025). AI-assisted cardiovascular risk assessment by general practitioners in resource-constrained Indonesian settings using a conceptual prototype: Randomized controlled study. Journal of Medical Internet Research, 27, Article e73131. https://doi.org/10.2196/73131

Vilone, G., & Longo, L. (2021). Notions of explainability and evaluation approaches for explainable artificial intelligence. Information Fusion, 76, 89–106. https://doi.org/10.1016/j.inffus.2021.05.009

Wang, X., & Yin, M. (2022). Effects of explanations in AI-assisted decision making: Principles and comparisons. ACM Transactions on Interactive Intelligent Systems, 12(4), Article 27. https://doi.org/10.1145/3519266

Wibowo, E. M., Nte, N. D., & Kusuma, B. H. (2025). AI in administrative dispute resolution: Exploring opportunities and risks. Indonesian State Law Review, 7(2), 137–146. https://doi.org/10.15294/islrev.v7i2.18877