Handwritten keyword spotting using deep neural networks and certainty prediction

Abstract

Word spotting using deep Convolutional Neural Networks (CNN) has recently obtained significant results in handwritten documents retrieval application. In this paper, we propose a novel word spotting method based on Monte-Carlo dropout CNN to compute the certainty of extracted features that can be used in both query-by-example (QBE) and query-by-string (QBS) word spotting scenarios. In the QBE and during the training, an adaptable certainty threshold is assigned for the words of each class. Cosine distance between the predicted certainty of the query image and the retrieval set is compared with the certainty threshold of each class in the matching step. For the QBS, the query class is compared to the class of the retrieval set obtained by the certainty prediction. We evaluated our proposed method on four public handwritten databases. Experimental results showed that the accuracy achieved in both QBE and QBS scenarios outperforms the state-of-the-art methods.