Localization is the process of determining the pose of the robot in an environment from sensory information and some sort of map or representation of the environment. Localization has applications in mobile robotics, autonomous vehicles and driver assistance systems. This paper presents a vision based localization method suitable for use in large scale urban environments especially for Indian roads. The method builds upon existing Bag-Of-Words techniques ~\cite{vgoogle, vocabTree} to address specific challenges that arise in urban environments such as occlusions by vehicles and similarity in visual appearance between different locations. The method can be used as a local localization scheme if a priori information of pose is available or can be used as a global localization method.

In outdoor environments, GPS is widely used for localization as it provides a straightforward way to obtain fairly accurate position and velocity estimates. In urban environments buildings can block the visibility of satellites and reflect signals which can cause GPS localization to fail. Crowded urban spaces with many buildings and indoor environments where GPS is inaccurate or unusable tend to be rich in visual data. Thus vision based localization complements GPS well.

Vision based localization can be performed using image retrieval. An image taken from the robot's current pose is compared against a database of images taken from various poses in the environment to find the best matching image, as images coming from nearby poses in the environment tend to have similar content it can be inferred that the robot is near the pose at which the retrieved image was captured. One approach is to directly match local features (such as SIFT features ~\cite{Lowe04}) between the current view and each of the database images as done in ~\cite{ZhangKosecka06}. However, for large image databases this method is too slow and requires too much storage to be feasible.

Bag-Of-Words image retrieval techniques where images are modelled as collections of visual words can scale effectively to much larger image databases. In ~\cite{topo07} visual words are matched and verified geometrically for localization in an indoor environment. In ~\cite{Brown07City} BoW based image retrieval is used to perform localization from a side mounted camera in an urban environment. Instead of using a previously determined, fixed vocabulary, the visual vocabulary used is built by choosing words which are more informative for the localization process. This results in an increase in performance for a given vocabulary size but requires all the images to be available before the vocabulary is built. If images are added incrementally, the chosen vocabulary will be suboptimal. FABMAP ~\cite{fabmap} learns a generative model of the set of visual words  describing appearance of a scene. This model can be used to compute the similarity between the current view and database images and also determine the probability of the images having come from the same location which allows it to handle environments where spatially seperated locations are highly similar in appearance.


The following paragraph tries to emphasise on the fact that vision based localization for Indian roads is different from the previous work done in ~\cite{ZhangKosecka06}. There may not be many landmarks in Indian roads as in ~\cite{ZhangKosecka06}. Figures \ref{} and \ref{} depict this difference. Moreover traffic in India is heterogeneous, i.e, it is dominated by motorcycles, autorickshaws, cycles and buses and sometimes pedestrians too as seen in figure \ref{}. These dynamic objects bring in a scattered occlusion which have to be taken care of. The presence of many dynamic objects in the same scene makes it tricky to recognize places. Thus localization in Indian roads in different on two acounts. One, lack of enough landmarks and two, presence of too many dynamic objects.

 %This pattern is not going to change in the near future and hence it motivated us to come up with a localization method for Indian uraban scenario. ~\cite{ZhangKosecka06} used a side mounted camera assuming which enabled it to recognise lot of landmarks on the way. But we use a forward mounted camera and as seen in the images there are not many landmarks on the sides of the roads. The comparison between the dataset is shown in Figure ~\ref{}. Moreover, the heterogeneous traffic brings in a lot of dynamic objects in between which have to be taken care of. The features on these dynamic objects have to be filtered to help in better localization. 

%We used a heuristic method to filter those. and it gave better localization result.

%Since we do not assume  the presence of landmarks in our data set, the question of choosing relevant features for building the dictionary doesn't arise as in ~\cite{ZhangKoseck06}. Needed something better than Simple sift feature matching. So, the motivation to assign weights to features arises, giving less weights to features that are seen in many places and more weights to features seen in few places. 


While navigating through an urban environment many dynamic objects such as other vehicles and pedestrians will be visible as shown in figures ~\ref{fig:frame_traffic} and ~\ref{fig:result_success}. These dynamic objects can occlude the scene and corrupt localization results. Our method uses geometric inferencing to detect moving objects and filter out the features they generate. The resulting filtered sets of visual words are used for localization. Urban areas also contain many structures and objects that appear repeatedly at different positions in the environment. As a result, widely seperated locations may have similar appearances (figure ~\ref{fig:frame_traffic}). Features in an image belonging to these common structures will be less informative for the purpose of determining robot pose. The method presented addresses this issue of non discriminative features by assigning weights to features on the basis of mutual information between robot pose and feature observation. 

Even though the current state of the art feature detectors generate good matches between image pairs, a large fraction of the features they detect are unstable to even small changes in viewpoint and are non repeatable. These unstable features will act like noise in the bag of words describing an image. They are unlikely to generate valid matches and could produce invalid matches with unrelated images in the image database. Our method addresses this problem by using closely sampled images instead of well seperated key images but only adding features that are stable to the database.   

The method presented was tested by attaching a camera to the top of a car recording a video sequence while driving along a path roughly 5km in length down city roads. A localization database was built from the frames in this video and the algorithm was tested for both global localization and local localization by matching frames from two other runs along the same path to the database built from the training run. GPS readings were used to verify the correctness of the localization results. 