Loading...
1 results
Search Results
Now showing 1 - 1 of 1
- Interpretable Face Verification Using Visual ExplanationsPublication . Claro, Bernardo Manuel Marques; Neves, João Carlos Raposo; Proença, Hugo Pedro Martins CarriçoA significant number of authentication systems rely on Face Verification (FV) due to its high accuracy, scalability and ease of integration in real-world applications. These systems often achieve performance levels that surpass human verification capabilities. However, they typically function as black boxes, offering binary outputs without insight into the decisionmaking process. This lack of interpretability raises serious concerns regarding trust, fairness and transparency, particularly in sensitive contexts where decisions must be explainable. Additionally, current explanation methods for FV still exhibit limitations in terms of precision, clarity and reliability of the visual explanations they generate. This dissertation presents a novel model-agnostic framework for explaining FV decisions using realistic perturbations guided by semantically segmented face regions. The proposed approach combines semantic face masks with a state-of-the-art face inpainting model to reconstruct masked regions in a visually coherent manner. For each face pair, the system systematically masks individual semantic face regions, inpaints the occluded areas, and compares the similarity scores before and after modification. This process quantifies the contribution of each region to the final verification decision and produces a similarity map that highlights the most influential facial areas. Two complementary perturbation strategies are introduced: Single Inpaint (S0), which assesses the individual impact of each semantic region, and Greedy Inpaint (S1), which incrementally evaluates combinations of regions to capture joint contributions. Extensive qualitative analysis shows that the proposed method produces more interpretable, precise, and visually coherent explanation maps than existing state-of-the-art techniques, which typically rely on random or unrealistic occlusions. This is further validated through a quantitative ablation study using Deletion and Insertion metrics, which confirms that the integration of semantic guidance with inpainting significantly improves the accuracy, reliability, and faithfulness of the resulting explanations.