Abstract
The recent advancement in remote sensing technologies has resulted in the availability of different imaging modes and higher resolution satellite images. Accessibility of these remote sensing or satellite images, automatic ship detection and tracking has become an important research topic in the field of maritime surveillance. In this paper, a novel method for ship detection using satellite images is proposed. First the preprocessing is carried out to remove the noise from the images using Ship Detection and Tracking (SDT) filter. Then, the land masking (sea-land area separation) and cloud masking is carried out based on the gradient feature extraction using SDT edge detection, along with SDT segmentation. Finally, the ships are identified using the Machine Learning (ML) classifiers like Support Vector Machine (SVM), Random Forest Classifier (RFC), Linear Discriminant Analysis (LDA), Logistic Regression (LR), KNN, and Gaussian Naïve Bayes-based classifier based on the features extracted from Histogram of Oriented Gradients (HOG). The proposed work is cross validated using the Google earth data. Performance of our proposed method is evaluated using the recall and the precision values. Further, for tracking ships, an improved multiple hypothesis tracking (MHT) algorithm is proposed and tested using the Kaggle dataset.
Keywords
Introduction
Remote sensing technology provides important capabilities to the intelligence community through the gathering of huge information from geographic areas of interest [7]. The data that’s collected by these remote sensors needs additional analysis to detect uncommon activity or the presence of targets of interest in vicinity. As technology continues to boost, it’s become more and more challenging for analysts to visually examine the big quantity of accumulated device knowledge [6]. This complexity may be self-addressed through the machine learning algorithms optimized for police investigation and also for classifying objects of specific interest to the defense in satellite representational process.
Synthetic Aperture Radar (SAR) imagery makes use of radio waves to capture the image of the Earth’s surface. In comparison with the optical imagery, the wavelengths used by the SAR are not affected by the external conditions such as time or other meteorological conditions, thus enabling imagery to be obtained day or night, with cloudy or clear skies [5].
Remote sensed ship detection is a key technology which is active in maritime security, including border protection, ship rescue, traffic surveillance, protection against illegal fisheries, discharging of hazardous wastes or oils, sea pollution monitoring etcetera [1,29]. Currently coastal radars and patrol aircraft are being widely used for ship detection and monitoring whereas the usage of satellite for ship detection is under being tested. Remote sensing provides significant advantages for ship tracking as it can collect pictures effectively covering the large area of the ocean [13,18,25].
The challenges that are identified are as follows:
∙ Detection and continuous monitoring of ships using satellite images is a complex task due to dynamic weather conditions, long distance from earth, light effects and other atmospheric noises.
∙ Land and cloud masking is necessary as it may hinder the ship detection process.
∙ Though Automatic Identification System (AIS) is used for tracking ships, it is possible that those transponders can be turned off and ships can’t be tracked.
The objective of the proposed work:
To design and train an algorithm which automatically detects and locates the out-of-port ships in given scene (border surveillance) using ML. To derive and test a method to infer the ship location by the Rational Polynomial Coefficient (RPC) model by matching with AIS data from Ground Control Points (GCP) and to track using the multiple hypothesis tracking (MHT). To derive an algorithm which adaptively computes window size for median
filtering, and the threshold value for Otsu method for feature enhancement.
Our contributions in this paper can be summarized as follows:
∙ Ship Detection and Tracking (SDT) filtering algorithm is developed for removing the noise from the images.
∙ The SDT segmentation algorithm improves the sea-land segmentation by choosing the threshold value dynamically for each image.
∙ Experiments performed on the satellite dataset shows that the proposed model Novel Ship Detection and Tracking (NSDT) shows better tracking accuracy while using with MPP techniques.
The rest of the paper is arranged as follows: Section 2 narrates a literature survey focusing on the existing research works. In Section 3, the proposed work is illustrated with necessary system architecture and mathematical modeling. The proposed NSDT model is discussed in details in this section as well. The implementation of the proposed algorithm and the performance evaluation and result analysis is interpreted in Section 4. Finally, the study is concluded in Section 5.
Related work
In the literature, many works are published related to ocean tracking applications. Song et al. worked on a hierarchical method for detecting ships in Inland River in China [13,25]. Shape analysis of connected components was used to detect the isolated ships and a model namely Deformable Part Models (DPM) and Histogram of Oriented Gradient (HOG) were used to detect the clustered ships using Back Propagation Neural Network (BPNN). Huang et al. proposed a neural network architecture, Squeeze Excitation Skip-connection Path networks (SESPNets) and Non-Maximum Suppression (NMS) to boost ship detection capability by adding a bottom-up path to the pyramid network functionality [24,25]. Vasan et al. [26] proposed a model which first selects the prominent candidate regions across entire detection scene using a mechanism of bottom-up visual attention. The two complementary types of top-down indicators were used to discriminate against the preferred ship candidates. Corbane et al. developed an effective ship detection algorithm that used high spatial resolution optical imagery [14,26]. The concept of automatic detection was based on techniques including wavelet analysis and Radon transform. To overcome the drawback of detecting ships from Space-borne Optical Images (SOI) since it has only less number of SAR sensors, relatively long revisit cycle and lower resolution. Zhu et al. [26] proposed a Novel Hierarchical complete and operational SDSOI (Ship Detection from Space-borne Optical Images) which were considered a sequential coarse-to-fine elimination process of false alarms based on shape and texture features.
Yao et al. [30] proposed a three step ship tracking algorithm using PSNR, RPCs and AIS in GF-4 satellite sequential imagery which was considered to have great scope for coastal surveillance. Surveillance with satellite synthetic aperture radar (SAR) was proposed by Brusch, S. et al., which have the capability to provide ship detection at high resolution over wide swaths and under all climatic conditions [8,21]. Blackman [3] highlighted important areas of research and development of Multiple Hypothesis Tracking (MHT). The author also proved that MHT can be combined with multiple filter models to improve the performance. Liu et al. proposed a complete ship detection and tracking based on the low-resolution geostationary optical satellite for coastal monitoring [11,12,31].
The literature study highlights that there is requirement of preprocessing the satellite images to improve the performance Upcoming sections provide a possible solution to better ship detection and tracking using Novel Ship Detection and Tracking (NSDT) algorithm with greater accuracy.
In this paper, a novel method for ship detection using satellite images is proposed. First the preprocessing is carried out to remove the noise from the images using Ship Detection and Tracking (SDT) filter. Then, the land masking (sea-land area separation) and cloud masking is carried out based on the gradient feature extraction using SDT edge detection, along with SDT segmentation. Finally, the ships are identified using the Machine Learning (ML) classifiers like Support Vector Machine (SVM), Random Forest Classifier (RFC), Linear Discriminant Analysis (LDA), Logistic Regression (LR), KNN, and Gaussian Naïve Bayes (GNB)-based classifier based on the features extracted from HOG.
Proposed work
System architecture of proposed NSDT
The architecture diagram for our proposed work NSDT is shown in Fig. 1(a) and contains the (i) Image acquisition module, (ii) Image preprocessing module, (iii) Feature extraction module, (iv) Learning module and (v) Performance evaluation module.
In the acquisition module, the satellite images of size of
The detailed explanation of the SDT pre-processing techniques models which are used in this algorithm are shown in Fig. 1(b). The SDT filtering is used for removal of speckle noises from the images with window size
Ship detection algorithm
In the proposed NSDT Algorithm 1, various novel pre-processing techniques based on median filter, prewitt operator and otsu threshold are used to improve the performance. The algorithm is given in Algorithm 1 and is pictographically represented in Fig. 1(a) and Fig. 1(b).

NSDT algorithm

(a) System architecture of the proposed NSDT. (b) Proposed SDT pre-processing techniques.
The SDT filtering is used to remove the noise from the images based on adaptive median filtering in which the size of the mask mainly depends on the image. The mask size initially,
The median filtering output is
The PSNR value is computed using the formula
Where MSE is Mean Square Error, MAXI is maximum possible pixel value of the image.
Higher the value of PSNR, better the quality of the image retained. So, the mask size having higher PSNR value will be considered as the best mask size for the image.

SDT filtering algorithm
The objective of this Algorithm 3 is to detect the edges of the ship by calculating the difference between the corresponding image pixel intensities. The correct threshold value is found by applying various thresholds values to the images.

SDT edge detection algorithm
The SDT segmentation is used to further detect the edges more clearly in the images. The algorithm is explained in Algorithm 4. First the background and object pixels are identified. Then the weights of the background and the object is calculated using the equations (6) and (7) respectively.
The mean gray-level value for the background and object pixel is calculated using (8) and (9) respectively.
The eqn (10) and eqn (11) calculates the variance value for the background and object pixel is given below
The within class variance and between class variance are obtained using the equation (12) and (13) from which a threshold value
This threshold value

SDT segmentation algorithm
Rational polynomial coefficients (RPCs)
The Rational Polynomial Coefficients (RPCs) model is the standard sensor model used mainly in the field of remote sensing and aerial photogrammetry. The image co-ordinates are represented as ratios of the cubic polynomials in the ground co-ordinates in the form of
After ship detection, the image coordinates of the ships are been obtained and RPC model are used to estimate the ground coordinates of the ships.
MHT algorithm
The MHT Algorithm is the best data association algorithm which is widely used in the modern tracking system. The log likelihood ratio (LLR) in MHT is used for analyzing track formation hypotheses and the track score
The candidate tracks are evaluated using the M-out-of-N logic method where a tentative track is confirmed if there are at least M detections in the last N scans, otherwise the track is discarded. A confirmed track is terminated if no detections in the past L consecutive frames have been validated or the speed of a target is unfeasible.
Performance evaluation
Simulation environment
The Kaggle, Planet satellite imagery dataset of San Francisco Bay and San Pedro Bay areas of California is used for testing and training using python 3, in Jupiter platform. The dataset consists of 4000 RGB images with
Dataset details
Dataset details
Parameters for machine learning models
The section mainly deals with the analysis of our proposed work. First the performance of the pre-processing techniques like median filter, prewitt operator, otsuthresholding are discussed. It is then followed by the analysis of the evaluation parameters like accuracy, precision, recall, f1-score, roc curve and specificity for different models which are calculated with the help of confusion matrix.
From Fig. 2, it is inferred that median filter with size 9 (

SDT filtering (a) original image (b) filtered image with size
Various threshold values were applied to the prewitt operator to obtain the better threshold value. From Fig. 3, it is clear that the optimum threshold value for edge detection is 150 for the given dataset. This is because the lines, edges and curves of the ships got detected accurately using this threshold value.

SDT edge detection at different threshold.
The Fig. 4 shows the Otsu thresholding for two different images where the ship boundaries are retained and the unnecessary things like sea, land, rocks are excluded in this stage. The threshold value is selected separately for each and every image in the given dataset based on the intensity of the image.

SDT segmentation thresholding.
The features are extracted from the images that are obtained after all the pre-processing techniques using the HOG. Figure 5 shows the HOG representation of the images where the cell size varies depending upon the image.

Feature extraction using HOG.
From the Fig. 6(a), it is clear that MPP-SVM model outperforms other machine learning models with an accuracy of 95%. The next better model is MPP-LDA by an accuracy of 89% which is 6% less than MPP-SVM model and the least is MPP-GNB with an accuracy of 72% which is 23% less than MPP-SVM model. The accuracy of MPP-SVM model is better than NPP-SVM model approximately by 6.3%.The F1-score of MPP-SVM is 95% which is 6.44% higher than the NPP-SVM model whose value is 89%. This is shown in Fig. 6(b). It is also evident from the figure that MPP ML models has better F1 score than the NPP ML models which suggest that which implies that the ships are detected correctly. The MPP-SVM model has high recall (0.96) which is 10% greater than the NPP-SVM model (0.86). In the MPP machine learning models, MPP-SVM models suppresses all other models recall value by 14 to 35%. The higher recall value suggest that some images are detected as ships even though they are non- ships. NPP-GNB model has the lowest recall value of 0.52 which is approximately 16% lesser than the MPP-GNB whose recall value is 0.61. These are shown in Fig. 6(c).The Fig. 6(d) suggest that the MPP-LDA (precision = 0.96) model is better than the MPP-SVM (precision = 0.94) model in terms of precision by 2% even though the latter is found to have high accuracy. The NPP-LDA model has precision value of 0.92 which is nearly 4% less than the MPP-LDA model. The difference in precision value between these models can be explained by the large number of false positives predicted by the MPP-SVM model. The higher precision value suggests a smaller number of non-ships are detected as ships. From the Fig. 6(e), it is clear that MPP-LDA has high specificity value of 0.96 which is 3% greater than NPP-LDA whose specificity value is 0.93. High value of specificity implies that high value of true negatives and low values of false positives. Here specificity means that images that don’t contain ships are predicted correctly as the images with no ships and vice versa. In Fig. 6(f), the ROC curve for different machine learning models are shown. It shows that SVM classifies the ship class more accurately than any other machine learning model. It is then followed by RFC and LR models with the least being GNB model.

Comparison of different models based on (a) accuracy (b) F1-score (c) recall (d) precision (e) specificity (f) ROC curve.
The paper proposed a novel method for ship detection using the SAR satellite imagery. The Planet imagery satellite covering the areas of San Francisco Bay and San Pedro Bay areas of California was used as data-set to proposed ship detection method. The SDT filter removed the speckle noise from the images by deceptively changing the mask size for filtering thereby increasing the PSNR values of the images. Increased PSNR values mean that the image has less error and therefore better quality. The SDT edge detection used the threshold value of the image adaptively for edge detection. Similarly, the SDT segmentation changes its threshold value for each image in the dataset to improve the segmentation results. Features extraction was performed using the HOG where the cell size was chosen based on the image values. The SDT per-processing algorithms proved to improve the detection results of the machine learning models like SVM, kNN, RFC, GNB, LDA, LR. The experimental results of the proposed ship detection algorithm (MPP-models) showed better results in terms of accuracy, precision, recall, f1-score, roc curve and specificity when compared with NPP-models. Delay in acquiring remote sensing image is one of the challenges and it can be minimized for improving efficiency.
Conflict of interest
The authors have no conflict of interest to report.
