A project by Prof. Spyridon Giagtzoglou on Experizon.
This project investigates how generative AI systems can be evaluated using different performance metrics. It focuses on measuring output quality, factual accuracy, bias, coverage, and computational efficiency. The goal is to identify which metrics are most suitable for assessing generative AI in real-world applications. Context Generative AI models, such as GANs and diffusion models, are widely used to generate realistic images, text, audio, and synthetic data. However, evaluating these models is challenging because generated outputs are often subjective and cannot always be judged using standard accuracy-based metrics. This project focuses on understanding and investigating the metrics used to evaluate generative AI systems, including image quality, diversity, realism, stability, bias, and computational efficiency. Students will explore how different metrics help assess whether a generative model produces useful, reliable, and high-quality outputs. Objectives Students will learn how to evaluate Generative AI models, such as GANs and diffusion models, using appropriate quantitative and qualitative metrics. They will gain a better understanding of how model quality, diversity, stability, bias, and computational efficiency can be measured and compared. By the end of the project, students will be able to select suitable evaluation metrics, interpret model performance, and critically assess the strengths and limitations of different generative models in real-world applications. Requirements Students should have a basic understanding of Generative AI, especially models such as GANs and diffusion models. They should also be familiar with core mathematical foundations, including statistics, calculus, linear algebra, and probability. A background in machine learning and deep learning is recommended, including knowledge of neural networks, loss functions, optimization, training/evaluation workflows, and model performance metrics. Basic Python programming experience is also useful for running experiments and analyzing results.
Students need to deliver a short presentation that explains how generative AI systems can be evaluated using different metrics. They should also create a github and there upload the code of the experiments and the analysis that have been done. Finally a report of the findings will be the last deliverable for this project.