Understanding how Recurrent Neural Networks (RNNs) make decisions is crucial, particularly in time-sensitive domains like healthcare and finance. My research focuses on enhancing the explainability of RNNs, specifically Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU) models, using deterministic finite-state automata (DFA) as a post hoc explanation method. By mapping the complex internal states of RNNs onto an interpretable state-transition diagram, my goal is to bridge the gap between high-performing models and their interpretability. Additionally, I am working on the quantitative evaluation of DFA as a post hoc XAI method to ensure its reliability.