Skip to content

Optimized Reverse Image Search using Deep Convolutional Neural Nets as well as traditional ML models

Notifications You must be signed in to change notification settings

SanatSharma/RevSearch

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

49 Commits
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

RevSearch

Optimized Reverse Image Search for all! This project aims to create a scalable and robust system for reverse Image search. There are 2 types of models implemented: Neural Models, Machine Learning Model. This code is part of the following paper:https://apps.cs.utexas.edu/apps/tech-reports/171206 (User Interfaces for Optimized Reverse Image Search)

For the Neural Model on a custom dataset of San Francisco images, approx. 69% of our test searches returned at least one relevant image in the top 3 images returned. Average latency was 450ms.

Initial dataset: CIFAR 10

To run the following packages are needed:

torch
torchvision
numpy
sklearn
python3x

Running script

cd models
python3 search.py

This project is highly customizable and you can use it to do reverse image search on your own dataset. Some customizable parameters are:

--model, type=str, default='ML', help="Model to run (CONV / ML)"
--train_type, type=str, default="CIFAR10", help="Data type - Cifar10 or CUSTOM"
--train_path, type=str, default='data/Reference/', help='Path to the training set'
--sift_path', type=str, default='model/sift.npy', help='Path to the sift file. Only useful if using ML model'
--num_clusters, type=int, default=100, help='Number of kmeans clusters for traditional ML model'

Models

  1. Neural model - The neural model utilizes transfer learning and uses a pre-trained VGG-16 model, trained on the ImageNet dataset for feature-extraction of images. PCA (principal component analysis) is used to reduce the dimensionality of this vector to construct an image dataset of Image id -> feature vector.

  2. ML Model - The traditional machine learning breaks down images into visual words and constructs an inverted file of words -> images for extremely fast retrieval. This model is heavily influenced by (http://www.cs.utexas.edu/~grauman/courses/fall2009/papers/bag_of_visual_words.pdf).

About

Optimized Reverse Image Search using Deep Convolutional Neural Nets as well as traditional ML models

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages