Hybrid Convolutional Neural Network with Domain adaptation for Sketch based Image Retrieval

Abstract

Background and Objectives: Freehand sketching is an easy-to-use but effective instrument for computer-human connection. Sketches are highly abstract so the domain gap that exists between the intended sketch and actual image. In addition to appearance information, it is thought that shape information is also very efficient in sketch recognition and retrieval. Methods: In the realm of machine vision, comprehending Freehand Sketches has grown more crucial due to the widespread use of touchscreen devices. In addition to appearance information, it is thought that shape information is also very efficient in sketch recognition and retrieval. The majority of sketch recognition and retrieval methods employ tactics based on appearance information. A hybrid network architecture comprising two networks—S-Net (Sketch Network) and A-Net (Appearance Network)—is shown in this article under the heading of hybrid convolution. These subnetworks, in turn, describe appearance and shape information. Conversely, a module known as the CCA Conventional Correlation Analysis technique module is utilized to match the range and enhance the sketch retrieval performance to decrease the range gap distance. Finally, sketch retrieval using the hybrid Convolutional Neural Network and CCA domain adaptation module is tested using many datasets, including Sketchy, Tu-Berlin, and Flickr-15k. The final experimental results demonstrated that compared to more sophisticated methods, the hybrid CNN and CCA module produced high accuracy and results. Results: The proposed method has been evaluated in the two fields of image classification and Sketch-Base Image Retrieval (SBIR). Related to results the proposed hybrid convolution, works better than other basic networks. it achieves a classification score of 84.44% for the TU-Berlin dataset and 82.76% for Sketchy. Also in SBIR The proposed method is in a good place among the methods based on deep learning and it performs better than non-deep methods by a long distance. Conclusion: This research presented the hybrid convolutional framework, which is based on deep learning, for pattern recognition. Compared to the best available methods, hybrid network convolution has increased recognition and retrieval accuracy by around 5%. It is an efficient and thorough method which demonstrated valid results in sketch-based image classification and retrieval on TU-Berlin, Flickr 15k wide, and sketchy datasets.