Post hoc explanation for RNNs using state transition representations
AI and advanced machine learning techniques have had a significant impact on several facets of our life in recent years, taking over human positions in a variety of complex tasks. In domains as diverse as healthcare, banking, justice, and defence, their applications have had great success. Deep neural networks (DNNs), such as convolutional neural networks (CNNs) and recurrent neural networks (RNNs), have gotten a lot of attention in recent years among these machine learning techniques. Since then, they have demonstrated unparalleled performance in tasks such as speech recognition, image recognition, natural language processing, and recommendation systems, outperforming humans in learning. However, it is observed that these best performing models are way too complex, abstract and opaque due to their complex deep architecture and non-linearity. Henceforth, they lack explainability and reliability. As they do not justify their decisions and predictions, it is difficult for humans to trust them. Unsurprisingly until recently, state-of-the-art CNNs, RRNs and other deep learning models, in general have been commonly regarded as “black boxes” or “black-box models”. Building trust in the deep learning model by validating its predictions and ensuring that it works as predicted and dependably on unseen or unfamiliar real-world data is unquestionably vital. In critical domains such as healthcare applications and autonomous vehicles, a single incorrect decision can have a catastrophic effect on society. Understanding, analyzing, visualizing, and explaining the rationale behind the model’s judgments and predictions is critical for ensuring the model’s reliability and understanding the model’s potential limitations and faultsA recurrent neural network (RNN) is a type of artificial neural network that uses sequential or time-series data. Recurrent neural networks, like feedforward and convolutional neural networks (CNNs), use training data to learn. They are distinguished by their “memory,” which allows them to use information from previous inputs to influence the current input and output. One of the challenging tasks with RNNs is to comprehend and evaluate their behavior. This is because it is difficult to understand what exactly they learn and also, they tend to work poorly on sequences requiring long-term memorization, despite having this capacity in principle. Many scholars have previously investigated a variety of strategies to address the aforementioned difficulties in recent years. The fundamental goal of this study is to investigate alternative methods for extracting interpretable state representations such as graphs, finite state machines, deterministic finite state machines from trained recurrent neural networks. The findings of this study will be useful across various domains in the industry and research community in order to provide better explanations to society for the deep learning applications they build and also comply with the GDPR rules.