10th International Congress on Information and Communication Technology in concurrent with ICT Excellence Awards (ICICT 2025) will be held at London, United Kingdom | February 18 - 21 2025.
Authors - Youssef Baklouti, Tarik Echcherqaoui, Ines Abdeljaoued-Tej Abstract - This release introduces an innovative interactive chatbot designed to engage with sensitive enterprise data. Leveraging Azure Machine Learning Promptflow and Retrieval-Augmented Generation (RAG) architecture, the chatbot facilitates secure data retrieval and generation within the enterprise environment. To assess the model’s performance, we utilized over 10 query examples, providing ground-truth and context data. Evaluation strategies included system-based metrics like the F1-Score, which yielded an average score of 0.59, and AI-evaluating- AI metrics such as Coherence, Groundedness, Similarity, Fluency, and Relevance, scoring 4.50, 4.20, 4.50, 4.10, and 4.40 respectively. While AI-evaluating-AI strategies showed decent scores, the relatively low F1- Score indicates potential for improvement through fine-tuning or selecting a more suitable vector database. Overall, this interactive solution not only enhances internal operations but also demonstrates AI’s potential in automating and streamlining complex processes.
