Improving multi-class Boosting-based object detection

Abstract

In recent years we have witnessed significant progress in the performance of object detection in images. This advance stems from the use of rich discriminative features produced by deep models and the adoption of new training techniques. Although these techniques have been extensively used in the mainstream deep learning-based models, it is still an open issue to analyze their impact in alternative, and computationally more efficient, ensemble-based approaches. In this paper we evaluate the impact of the adoption of data augmentation, bounding box refinement and multi-scale processing in the context of multi-class Boosting-based object detection. In our experiments we show that use of these training advancements significantly improves the object detection performance.

Keywords

Object detection multi-class Boosting data augmentation bounding box adjust

1. Introduction

Visual object detection is the first step in many relevant Computer Vision problems such as facial facial landmarks detection [1], body pose estimation [2], self driving cars [3], vehicle type analysis [4], text reading [5], etc. The object detection pipeline can be described in terms of three basic components: 1) image features: Haar-like-features, Histograms of Oriented Gradient (HoGs) features, Integral Channel Features (ICF), Locally Decorrelated Channel Features (LDCF), Convolutional Neural Networks (CNNs), etc.; 2) classification algorithm: AdaBoost, RealBoost, Support Vector Machines (SVMs), Neural Nets, etc.; 3) detection strategy: sliding window, candidate window proposals and bounding box regression, direct regression from image features (Single Shot Detectors), Hough voting, etc.

Since the publication of Viola and Jones [6] seminal work, the performance of object detection algorithms has improved by changing several parts of the detection pipeline. Visual features have evolved from being hand-crafted [7] to automatically selected from a pool with Boosting [8], Random Forest [9] or learned with a CNN [10]. The brute force sliding window approach [6, 7, 8] has evolved into fast region proposal algorithms [5, 10] and more recently Single Shot Detectors [11, 12]. The Boosting approach has received much attention because it is computationally very efficient and achieves very good performance in various object detection problems such as pedestrians [13, 14], multi-view faces [15] or multi-view cars [16]. The key to its success is the use of the feature selection capabilities of Boosting together with different pooling strategies [13] and rich image descriptions such as the ICF [8]. Further, object detection has evolved from detecting a single object category to being able to detect multiple categories and different views at the same time [17]. The usual framework for Boosting-based object detection uses binary classification (e.g. AdaBoost). In this regard, essentially multi-class detection problems, such as face [15] or car [16] detection, have been usually solved with binary classifiers: either with a monolithic detector (i.e. Object-vs-Background) or with a detector per object view or positive class (i.e. using $K$ detectors).

Powered by the use of deep neural nets [18, 19], modern object detection algorithms have achieved remarkable precision [10, 11, 12]. This is due to the use of discriminative features produced by deep models and the adoption of new training strategies. However, these approaches require advanced computational resources, such as Graphical Processing Units (GPUs) to achieve real-time performance. This prevents their use in devices with limitations in computing power or energy supply, such as aerial vehicles, micro-robots or mobile phones. Hence, the necessity of developing very efficient object detection algorithms, such as, for example, those based on Boosting approaches.

One of such methodological advances in object detection is the refinement of the bounding box of the detected objects. Classifiers are usually trained with fixed bounding box size and aspect ratio (AR). The bounding box refinement step is crucial to achieve top performance when dealing with objects showing different aspect ratios depending on their pose or configuration. Modern CNN-based detectors such as Faster RCNN [10], YOLO [11] and SSD [12] perform bounding box regression. Also, some of the best results in the KITTI car detection benchmark [20, 21] iteratively adjust the detection bounding box. In Section 4 we introduce a bounding box refinement scheme. The most similar approach to ours was introduced by Juránek et al. [22]. They use a Real AdaBoost binary classifier for car detection. In their work the estimation of the 3D orientation of the car is performed using a similar scheme to our bounding box aspect ratio estimation. However, in their solution the bounding box aspect ratio is fixed and depends on the car orientation. This is a limitation, since cars with the same orientation may have different aspect ratios.

Another training improvement is the augmentation of training data by producing realistic image transformations that do not change their labels. In the SSD detector [12] the newly generated data accounts for an 8.8% increase of the performance in the VOC 2007 test dataset. It is also a common strategy in the top performing car detectors in the KITTI dataset [21]. An alternative way of using additional data is to pre-train the model in a related but different dataset. All top performing algorithms in KITTI cars detection use pre-trained models in the ImageNet classification problem [23, 20, 21]. In the Boosting detection literature it is typical to use the geometric augmentation with image translation, rotation [8], image horizontal flip [24] and aspect ratio changes [25].

Recently, theoretically sound results in the context of multi-class Boosting provide new tools to address the unbalanced and asymmetric classification problems arising in object detection [24]. In this paper we endow this algorithm with modern training strategies, such as data augmentation, bounding box regression and multi-scale processing and evaluate the increase in performance achieved with these improvements.

2. Multi-class boosting algorithm

A Boosting classification algorithm is a supervised learning scheme that builds a binary prediction model by combining a collection of simpler base models or weak learners [26]. It receives as input a set of $N$ training data instances $\left\{(\textbf{x}_{i},l_{i})\right\}_{i=1}^{N}$ , where $\textbf{x}_{i}\in\mathbb{R}^{m}$ represents an object to be classified with class label $l_{i}$ . In our multi-class Boosting scheme [27] each label $l_{i}\in L=\{1,2,\ldots,K\}$ has a corresponding margin vector $\textbf{y}_{l}\in Y$ , where $Y=\left\{\textbf{y}_{1},\ldots,\textbf{y}_{K}\right\}$ . $\textbf{y}_{l}$ has a value 1 in the $l$ -th coordinate and $\frac{-1}{K-1}$ elsewhere. So, if $l=1$ , the margin vector representing class 1 is $\textbf{y}_{1}=\left(1,\frac{-1}{K-1},\ldots,\frac{-1}{K-1}\right)^{\top}$ . Hence, it is immediate to see the equivalence between classifiers $G$ defined over $L$ and classifiers g defined over $Y$ , $G(\textbf{x})=\linebreak l\in L\Leftrightarrow\textbf{g}(\textbf{x})=\textbf{y% }_{l}\in Y$ .

2.1 BAdaCost: Cost-sensitive multi-class Boosting classification

Cost-sensitive classification endows the traditional Boosting scheme with the capability to modify pair-wise class boundaries. In this way, we can reduce the number of errors between positive classes (e.g. different target orientations) and improve recall when object classes have different aspect ratios. To this end we use BAdaCost [27] (Boosting Adapted for Cost matrix), a recently introduced multi-class cost-sensitive Boosting classifier. In this section we briefly introduce it.

Costs are encoded in a $K\times K$ -matrix C, where each entry $C(i,j)$ represents the cost of miss-classifying an instance with real label $i$ as $j$ . Here it is assumed that $C(i,i)=0,\forall i\in L,$ i.e. the cost of correct classifications is zero.

Let $\textbf{C}^{*}$ be a $K\times K$ -matrix defined in the following way

$\displaystyle C^{*}(i,j)=\left\{\begin{array}[]{ll}C(i,j)&\text{if}\>\>i\neq j% \\ -\sum_{h=1}^{K}C(j,h)&\text{if}\>\>i=j\\ \end{array}\right.,$

(1) $\displaystyle\forall i,j\in L.$

In a cost-sensitive classification problem each value $C^{}(j,j)$ represents a “reward” associated to a correct classification. The $j$ -th row in $\textbf{C}^{}$ , denoted as $\textbf{C}(j,-)$ , is a margin vector that encodes the costs associated to the $j$ -th label. The multi-class cost-sensitive margin associated to instance $(\textbf{x},l)$ is given by $z_{C}:=\textbf{C}^{}\linebreak(l,-)\cdot\textbf{g}(\textbf{x})$ . It is easy to verify that if $\textbf{g}(\textbf{x})=\textbf{y}_{i}\in Y$ , for a certain $i\in L$ , then $\textbf{C}^{}(l,-)\cdot\textbf{g}(\textbf{x})=\frac{K}{K-1}\textbf{C}^{}(l,i)$ . Hence, using this generalized margin, BAdaCost defines a Cost-sensitive Multi-Class Exponential Loss Function* (CMELF):

$\displaystyle\mathcal{L}_{C}\left(l,\textbf{g}(\textbf{x})\right):=\exp\left(z% _{C}\right)=\exp\left(\textbf{C}^{}(l,-)\cdot\textbf{g}(\textbf{x})\right)=% \exp\left(\frac{K}{K-1}\textbf{C}^{}(l,G(\textbf{x}))\right).$ (2)

The margin, $z_{C}$ , yields negative values when the classification is correct under the cost-sensitive point of view, and positive values for costly (wrong) outcomes. The CMELF is a generalization of the Multi-class Exponential Loss introduced in [28].

BAdaCost resorts to the CMELF Eq. (2) for evaluating classifications encoded with margin vectors. The expected loss is minimized using a stage-wise additive gradient descent approach. The strong classifier that arises has the following structure:

$\displaystyle H(\textbf{x})=\arg\min_{k}\left(\textbf{C}^{}(k,-)\cdot\sum^{M}% _{m=1}\beta_{m}\textbf{g}_{m}(\textbf{x})\right)=\arg\min_{k}\left(\textbf{C}^% {}(k,-)\cdot\textbf{f}(\textbf{x})\right),$ (3)

where $\textbf{f}(\textbf{x})$ is a linear combination of $M$ cost-sensitive weak learners, $\{\textbf{g}_{m}(\textbf{x})\}_{m=1}^{M}$ , that the algorithm learns incrementally. In this case $\textbf{f}(\textbf{x})$ is a vector with the estimated class margin vector from the feature vector x.
2.2 Object detection score for BAdaCost

When building an object detector it is necessary to have a confidence measure or detection score. In BAdaCost the predicted costs incurred when classifying sample x in one of the K classes are given by the vector:

$\displaystyle\textbf{c}=\textbf{C}^{*}\cdot\textbf{f}(\textbf{x})=(c_{1},% \ldots,c_{K})^{\top}.$ (4)

From now on, in multi-class detection problems, we assume that the background (negative) class has label $l=1$ and the object classes (e.g. different views of a car) have label $l>1$ . Therefore, we can compute the score of x as

$\displaystyle s(\textbf{x})=(c_{1}-\min(c_{2},\ldots,c_{K})).$ (5)

This score has desirable properties for detection problems: 1) $s(\textbf{x})>0$ when the winner class (i.e. the class with lowest cost) has label $l>1$ ; 2) $s(\textbf{x})<0$ when the winner class is $l=1$ . With this score we can improve the classifier efficiency by using a cascade calibration algorithm, for example [29], and stop the evaluation of weak learners whenever the score falls below a calibrated threshold.

3. Boosting data augmentation (DA)

One of the main problems for training an object detector is the limited amount of training data, that could cause the classifiers to overfit. The solution adopted in the literature is to generate new synthetic data from the training set. This is known as data augmentation (DA).

Figure 1.

Sample KITTI photometric data augmentation. In the first column we display the original training image, followed by 8 generated images.

We model the object to be detected as multiple positive classes depending on the orientation, see Fig. 6. To augment and balance our dataset we increase the number of images in the small classes. To this end we sequentially apply a combination of basic photometric changes to the image RGB values (see Fig. 1). We start by generating a random number $r$ in the range [0, 1]. If $r>0.5$ we make the following transformations:

Brightness change,

Contrast change,

Saturation change,

Hue Change.

Otherwise:

Brightness change,

Saturation change,

Hue Change,

Contrast change.

These photometric changes are implemented as:

Brightness change. Generate a random number $r_{b}$ in the range [0, 1]. If $r_{b}<p_{b}$ a random value in the range $[-\Delta b,\Delta b]$ is added to all pixel RGB values.

Contrast change. Generate a random number $r_{c}$ in the range [0, 1]. If $r_{c}<p_{c}$ all pixel values are multiplied by a random value in the range $[c_{\textit{low}},c_{\textit{high}}]$ .

Saturation change. Generate a random number $r_{s}$ in the range [0, 1]. If $r_{s}<p_{s}$ the image is transformed to the HSV color space. All pixel saturation values in the image are multiplied by a random value in the range $[s_{\textit{low}},s_{\textit{high}}]$ and the resultant HSV values are transformed back to RGB.

Hue change. Generate a random number $r_{h}$ in the range [0, 1]. If $r_{h}<p_{h}$ the image is transformed to the HSV color space, a random value in the range $[-\Delta h,\Delta h]$ is added to all hue values in the image and the new HSV values are transformed back to RGB.

As we show in Section 6.1 the generation of synthetic data is crucial to achieve a 5.5% improvement in detection results (see Table 2, Moderate). In our detection experiments some views have fewer examples. The use of photometric augmentation allows us to balance the dataset. Further, it allows us to increase the classifier capacity with more and deeper decision trees. This is consistent with similar results in the problem of pedestrian detection [25].

4. Bounding box aspect ratio estimation (BB)

As we show in our experiments, refining the bounding box estimation increases the performance of our detector. To this end we modify the multi-class Boosting classifier introduced in Section 2.

The classifier learns $m$ cost-sensitive decision trees with Gini impurity measure. The modifications of the tree to make it cost-sensitive are two-fold:

On each tree node $S$ , the probability of class $l$ , $p(l)$ , is multiplied by the percentage of costs associated to class $l$ (i.e. the sum of a cost matrix row), $c(l)$ :

$\displaystyle p^{\prime}(l)=\underbrace{\frac{\sum_{i\in S}w_{i}\cdot I(l_{i}=% l)}{\sum_{i\in S}w_{i}}}_{p(l)}\underbrace{\frac{K\cdot\sum_{k=1}^{K}C(l,k)}{% \sum_{i=1}^{K}\sum_{j=1}^{K}C(i,j)}}_{c(l)},$ (6)

where $I()$ is the indicator function, and $w_{i}$ is the weight assigned by the BAdaCost algorithm to the $i$ -th datum in node $S$ . Then, $p^{\prime}(l)$ is used in the Gini impurity.

Then we classify with the minimum cost rule on each leaf node $S$ :

$\displaystyle h_{S}=\arg\min_{l}(p(1),\ldots,p(K)){}\cdot\textbf{C}(-,l)$ (7)

where $p(l)=\frac{\sum_{i\in S}w_{i}\cdot I(l_{i}=l)}{\sum_{i\in S}w_{i}}$ and $\textbf{C}(-,l)$ is the $l$ -th column of the cost matrix.

During the training phase of the baseline BAdaCost algorithm, for every decision tree and leaf node $S$ , we store the minimum cost label, $h_{S}$ . During the detection phase, the trees are traversed with the feature vector x corresponding to a candidate window (see Fig. 2). As we have seen in Section 2, we compute vector $\textbf{f}(\textbf{x})$ as a linear combination of tree labels $h$ , coded as margin vectors $\textbf{g}(\textbf{x})=\textbf{y}_{h}\in Y$ . Then we use vector $\textbf{f}(\textbf{x})$ in Eq. (3) to obtain the minimum cost view class estimation. To correct the bounding box we have two options:

Figure 2.

Bounding box aspect ratio estimation. We learn an ensemble of multi-class cost-sensitive trees. We estimate the AR distribution using all the trees from $t_{p}$ onwards. The estimated AR is that of the minimal cost class.

The baseline BAdaCost detector computes the mean AR of each class from the ground truth training data bounding boxes in a vector $\textbf{a}[h]$ . The final estimated class, $h$ , is that corresponding to the minimum value in Eq. (3) using the estimated $\textbf{f}(x)=\sum_{m}\beta_{m}\textbf{g}_{m}(\textbf{x})$ . The aspect ratio of each detection is that associated to $h$ .

Our new procedure to estimate the aspect ratio follows a similar approach to the computation of $\textbf{f}(\textbf{x})$ in BAdaCost. The $m$ -th WL tree outputs two vectors of size $K$ (i.e. number of classes): a) the $\textbf{g}_{m}(\textbf{x})$ margin vector that codifies the estimated class and b) the $\textbf{a}_{m}(\textbf{x})$ vector of per class mean aspect ratios of the training data that fall in the corresponding leaf node. Then, both sets of vectors are added using the boosting weight of each tree, generating $\textbf{f}(x)=\sum_{m}\beta_{m}\textbf{g}_{m}(\textbf{x})$ vector (for classes) and $\textbf{a}(\textbf{x})=\sum_{m}\beta_{m}\textbf{a}_{m}(\textbf{x})$ vector (for aspect ratios). The final estimated class, $h$ , is that corresponding to the minimum value in Eq. (3) using the estimated $\textbf{f}(\textbf{x})$ . The aspect ratio is that corresponding to the estimated class, $\textbf{a}(\textbf{x})[h]$ (see Fig. 2).

We correct each window keeping the height constant and adapting the AR.

Note that in option B we may exclude the first $t_{p}-1$ trees from the model. The rationale is that in the first trees the classifier is not accurate enough. In the experimental section we will see that this is in fact true, but the difference in performance is marginal. The overall approach to simultaneous detection and AR estimation is shown in Fig. 2.

Figure 3.

Multi-scale detection. We apply classifiers trained with different base scale (i.e. blue, green and red rectangles) to detect objects with various dimensions.

Figure 4.

Car parts and location of the features selected by the detector.

Figure 5.

Location of features selected by different car detectors. Top, middle and bottom images show respectively the LDCF features of the detector trained with 48 $\times$ 84, 56 $\times$ 98 and 64 $\times$ 112 pixel images.

5. Multi-scale detection

The classifiers used in object detection are typically learned with a fixed scale. Both Boosting-based [8, 30, 15, 13, 25, 24] and CNN-based [10, 11] detectors are all trained with a fixed base image size. However, objects of the same class at different sizes usually need different features to be detected. CNN-based detectors overcome this problem using feature maps extracted from layers with different resolutions [12, 31]. Boosting-based detectors use a fixed size classifier in an sliding window on a pyramid of feature maps extracted at decreasing image resolutions. This enables the detection of objects at different sizes. The detection rate also improves using a classifier trained at several base sizes [32, 16].

In this paper we use a multi-scale (MS) detection approach training different BAdaCost detectors at different base sizes. For the KITTI car dataset, our multi-scale approach trains three classifiers with 1.75 aspect ratio: 48 $\times$ 84, 56 $\times$ 98 and 64 $\times$ 112 pixels. For the KITTI cyclists we also train three classifiers but in this case with aspect ratio 1.5: 48 $\times$ 72, 56 $\times$ 84 and 64 $\times$ 96 pixels. Note here that the fixed aspect ratio is only used to detect the presence of the target. Then the detected window is adjusted with the procedure described in Section 4. Finally, we slide these detectors along an efficiently estimated feature pyramid [8, 33], see Fig. 3.

As we show in Fig. 5, features selected at different scales show better definition of interesting areas as the resolution increases. In the car problem, these features are mainly around the rear lights and the edges that define the car shape at different orientations (see Fig. 4). In Section 6 we show that we achieve a significant improvement in recall using detectors trained at different scales.

6. Experiments

To make our experiments we have added BAdaCost’s cost-sensitive decision trees and multi-class detection to Piotr Dollar’s Matlab Toolbox.1 Our implementation is publicly available at https://github.com/jmbuena/ toolbox.badacost.public.

In the experiments we use detection problems where the target object changes its Bounding Box AR depending on the view angle. Car and cyclist detection in the KITTI dataset [34] are good examples of this kind of problems. The database presents three subsets: easy, moderate and hard (easy $\subset$ moderate $\subset$ hard). We carry out the evaluation in each level separately. In total there are 7481 images for training and 7518 for testing. Since the testing images have no ground truth, we split the training set into training and validation subsets: the first 6733 images (90%) for training (KITTI-train90) and the last 748 images (10%) for validation (KITTI-train10). In all the experiments we use cost-sensitive decision tree weak learners that select features from LDCF channels [33]. We augment our data by flipping images horizontally. To compare detectors in the KITTI dataset we use the Average Precision (AP), computed using 11 (AP ${}_{11}$ ) and 40 (AP ${}_{40}$ ) points in the precision-recall curve [35]. The traditional performance measure in KITTI was AP ${}_{11}$ . However, since October 2019 it has changed to AP ${}_{40}$ . Thus, we report both. We select the classifier parameters by grid search in the validation set, KITTI-train10.

For data augmentation we use the following parameters (see Section 3): $p_{b}=$ 0.5, $\Delta b=$ 32 for brightness; $p_{c}=$ 0.5, $c_{\textit{low}}=$ 0.5, $c_{\textit{high}}=$ 1.5 for contrast; $p_{s}=$ 0.5, $s_{\textit{low}}=$ 0.5, $s_{\textit{high}}=$ 1.5 for saturation; and $p_{h}=$ 0.5, $\Delta h=$ 18 for hue changes.

6.1 Car detection experiments

Figure 6.

The car classes we use in our experiments. Note the relation between classes, i.e. car view orientation, and bounding box aspect ratios.

Figure 7.

The cost matrix, C, used in the KITTI car dataset experiments.

Here we improve the results in our previous work [36] by increasing the number of view classes ( $K=$ 20 to 25), the depth of the weak learners trees ( $D=$ 8 to 10) and the regularization (shrinkage from 0.1 to 0.05, fraction of features searched in the training of each weak learner from 1/16 to 1/32). Increasing tree depths improves classifier capacity, hence enabling the use of more orientation classes and achieving better predictions, but at the expense of stronger regularization. Therefore, in the experiments we divide the car training images into $K=$ 25 view classes (see Fig. 6) and use KITTI-train90 for training and KITTI-train10 for validation. In all the experiments we train a 48 $\times$ 84 pixels car model, $AR=$ 1.75. We start the pyramid one octave above the actual image size.

During training, we perform 4 rounds of hard negatives mining with the KITTI training image subset (KITTI-train90). The best number of cost-sensitive trees is $T=$ 1024 (4 rounds with 32, 128, 256 and $T$ weak learners, respectively), tree depth is $D=$ 10, the number of negatives per round is $N=$ 7500 and the total amount of negatives used in the final round is $NA=$ 30000.

The cost matrix is set to weigh up gross errors between view classes. This is important because estimating the wrong class will produce a Bounding Box with the wrong AR (e.g. frontal car, $AR=$ 1.0, left side car, $AR\gg$ 1.0). We show our cost matrix in Fig. 7. The non-car class has label 1. Positive classes have the labels shown in Fig. 6 increased in one to account for the non-car class.

Table 1

AP ${}_{40}$ for different IoU values on the KITTI train90/train10 car detection experiment

	Algorithm/IoU	0.5	0.6	0.7	0.8	0.9
Easy	Fixed-Class-Mean	99.5%	99.2%	95.6%	52.2%	1.2%
	Estimated-AR	99.5%	99.2%	96.3%	60.8%	3.2%
Moderate	Fixed-Class-Mean	96.4 %	95.8%	88.9%	51.1%	1.9%
	Estimated-AR	96.4%	95.8%	89.3%	57.0%	3.9%
Hard	Fixed-Class-Mean	81.1%	78.3%	71.6%	39.9%	1.3%
	Estimated-AR	81.1%	78.4%	72.0%	44.8%	2.9%

Figure 8.

Aspect ratio AP results depending on the first tree used in the estimation.

Figure 9.

Precision-recall curves and (AP ${}_{11}$ , AP ${}_{40}$ ) values for different AR estimation strategies.

First, we train only one detector with the mean aspect ratio of each view class stored in the tree leaves. We set the detection threshold to an intersection over union (IoU) value 0.7, as established in the KITTI benchmark. Then, we use different AR estimation strategies. First we test the AR estimation algorithm introduced Section 4 with different values of $t_{p}$ (first tree to use in the estimation). In Fig. 8 we show that we get marginal improvements when selecting the best $t_{p}$ .

Second, we compare different AR estimation strategies (we use here $t_{p}=$ 1): use the output of the fixed size sliding window detector trained at 48 $\times$ 84 pixels (Fixed-Equal), modify the fixed aspect ratio window with the estimated class view (Fixed-Class-Mean) and, finally, our proposal (Estimated-AR). In Fig. 9 we confirm that a fixed aspect ratio detector, Fixed-Equal, produces the worst results. Results in the Moderate KITTI subset (the one used for ranking) improve significantly with the Fixed-Class-Mean strategy. Finally, we may improve even further the AP ${}_{40}$ by using the proposed Estimated-AR strategy. We improve AP ${}_{40}$ by 0.7%, 0.4% and 0.4%, respectively, in the Easy, Moderate and Hard settings. Since our bounding box estimation procedure is computationally cheap, it is a significant improvement. In fact, it only takes slightly more memory to store the $K$ possible aspect ratios, $\textbf{a}_{m}$ , on each tree node. On the other hand, the added computation is negligible and consists on computing the aspect ratios weighted vector, $\textbf{a}(\textbf{x})=\sum_{m=t_{p}}^{M}\beta_{m}\textbf{a}_{m}(\textbf{x})$ .

To further analyze the performance of our procedure (Estimated-AR) with respect to the baseline (Fixed-Class-Mean), we have performed an additional experiment varying the IoU threshold (see Table 1). The good behavior of our approach is more evident when we look for higher overlapping in the detection. With a minimum required IoU $=$ 0.8, Estimated-AR, in Moderate subset, improves by 11.54% (from 51.1% to 57%) and with IoU $=$ 0.9 the improvement is 105.26% (from 1.9% to 3.9%).

Table 2

Car detection ablation study in KITTI’s validation set (KITTI-train10)

Training/testing	Algorithm	Easy		Moderate		Hard
		AP ${}_{11}$	AP ${}_{40}$	AP ${}_{11}$	AP ${}_{40}$	AP ${}_{11}$	AP ${}_{40}$
Train90/train10	BAdacost, $K=$ 20 [24]	84.8%	88.3%	82.1%	83.2%	66.9%	66.2%
	BAdacost $+$ I, $K=$ 20	87.3%	91.9%	86.4%	86.1%	69.1%	68.8%
	BAdacost $+$ I	87.7%	92.5%	85.9%	85.9%	69.3%	68.9%
	BAdacost $+$ BB	88.3%	93.1%	86.3%	86.2%	69.2%	69.0%
	BAdacost $+$ DA	88.1%	93.0%	87.1%	88.7%	69.9%	71.5%
	BAdacost $+$ DA $+$ BB	95.4%	96.3%	88.0%	89.3%	70.5%	72.0%
	BAdacost $+$ DA $+$ BB $+$ MS	93.6%	96.1%	88.1%	89.0%	70.6%	73.5%

Key: stronger regularization (I), Bounding Box estimation (BB), Data Augmentation (DA), and multi-scale detection (MS), average precision with $n$ curve points (AP ${}_{n}$ ).

Figure 10.

Precision-recall curves and (AP ${}_{11}$ , AP ${}_{40}$ ) values for the car detection ablation experiment.

In an ablation analysis shown in Fig. 10 and Table 2, we evaluate different detector configurations and use synthetic data to enhance even further the performance of our detector. Data augmentation allows us to avoid overfitting and increase the Boosting classifier capacity with deeper decision trees using a finer quantization in car orientation. We use $K=$ 25 views whereas in our previous approaches [24, 36] we used $K=$ 20. We depart from the baseline results in [24], “BAdaCost, K $=$ 20”, obtained with $T=$ 1024 trees of depth $D=$ 8, $K=$ 20 views and $NA=$ 30000 hard negatives (shrinkage 0.1 and $\textit{fracFtrs}=$ 1/16 fraction of features explored on each tree training). Increasing the tree depth to $D=$ 10 we have to regularize more (shrinkage 0.05 and $\textit{fracFtrs}=$ 1/32, less features explored on each weak learner), but we improve AP ${}_{40}$ by 2.9% (see “BAdaCost $+$ I, K $=$ 20”, Moderate results). If we now increase the number of views to $K=$ 25, in some cases the AP ${}_{40}$ decreases. This is because there are new classes with few data (see “BAdaCost $+$ I”, Moderate results). By including the bounding box adjustment procedure described in Section 4, we get slightly better results (see “BAdaCost $+$ I $+$ BB”). On the other hand, when we augment the data with synthetic samples we obtain a substantial improvement (see “BAdaCost $+$ DA”), greater than that achieved with the bounding box adjustment, aided by the increased capacity of the classifier and the increased number of views. Combining the bounding box adjustment (BB) and the data augmentation (DA) we get the best single classifier detector (see “BAdaCost $+$ DA $+$ BB”) which in the Moderate examples is reporting 6.1% AP ${}_{40}$ improvement with respect to base line (83.2 to 89.3).

To further improve the AP of the detector we have to take into account that a car at different resolutions may not need the same features. Therefore, we train three detectors at different base sizes: 48 $\times$ 84, 56 $\times$ 98 and 64 $\times$ 112 pixels. We apply these classifiers to the image pyramid obtaining slightly better results in the hard examples (see “BAdaCost $+$ DA $+$ BB $+$ MS”, Hard). Finally, in the testing subset of KITTI2 (see Fig. 11 and Table 3) the multiscale approach (“BadaCost $+$ DA $+$ BB $+$ MS”, Moderate) is significantly better than the single scale approach (“BAdaCost $+$ DA $+$ BB”, Moderate) increasing in 2.8% the AP ${}_{40}$ . The main reason is that it gets slightly better precision and better recall.

Table 3

Car detection comparison experiment in KITTI-testing evaluation server

Training/testing	Algorithm	Easy		Moderate		Hard
		AP ${}_{11}$	AP ${}_{40}$	AP ${}_{11}$	AP ${}_{40}$	AP ${}_{11}$	AP ${}_{40}$
Train90/testing	BAdacost, $K=$ 20 [24]	77.9%	81.4%	67.1%	66.6%	51.2%	52.2%
	BAdacost $+$ DA $+$ BB	84.9%	85.5%	69.5%	70.9%	59.1%	56.2%
	BAdacost $+$ DA $+$ BB $+$ MS	83.2%	85.2%	74.3%	73.7%	59.0%	57.8%
Training/testing (Boosting)	Subcat [16]	81.4%	84.1%	75.5%	76.4%	59.7%	60.6%
Unknown/testing (Boosting)	Regionlets [17]	86.5%	88.7%	76.6%	77.0%	59.8%	60.5%
	spLBP [39]	80.2%	81.7%	77.4%	78.7%	60.6%	61.7%
ImageNet $+$ training/testing (CNNs)	TuSimple [23]	90.8%	95.1%	90.3%	94.5%	82.9%	86.5%
	DeepManta [20]	97.2%	98.9%	90.0%	93.5%	80.6%	83.2%
	RRC [21]	90.6%	95.7%	90.2%	93.4%	87.4%	87.4%

Best BAdaCost results are shown in bold and overall best results in italic.

Figure 11.

Car detection comparison in KITTI-testing using the evaluation server. We report (AP ${}_{11}$ , AP ${}_{40}$ ) for each method.

Since the KITTI dataset is not very large, the best CNN based detectors use a VGG network [37] pretrained with ImageNet [38] and finetuned with KITTI. Thus, they are not fully comparable to the Boosting methods trained only with KITTI. Moreover, there are various of methods using LIDAR or the stereo image pair data. However, we are only interested in algorithms comparable with ours. In Fig. 11 and Table 3 we show the results of the top three detectors that use a single RGB image: TuSimple [23], DeepManta [20] and RRC [21]. The reason for their success is the use of convolutional features adapted for the problem at hand. However, the need for a powerful GPU could be determinant in some engineering applications with limited devices. In these cases, the Boosting algorithms presented in this paper represent a computationally efficient alternative.

We have also compared the execution time of RRC [21], one of leading deep learning-based car detectors, with our Boosting-based detector in a CPU. We have compiled the implementation of RRC3 with OpenBLAS support4, that adds thread-based parallelism to matrix operations. On average RRC takes 82 seconds to process an image, with 100% CPU usage (Intel Xeon E5620 2.4 GHz). Such long processing time is caused by the large size of KITTI images (1242 $\times$ 375 pixels) and the fact that RRC doubles their resolution to detect small cars. On the other hand, our DA $+$ BB $+$ MS car detector takes on average 34 seconds per frame when doubling the input image size (i.e. one octave up the input size) and 6 to 12 seconds when using the standard size. The simpler BAdaCost detector, DA $+$ BB, takes 17 seconds per image with double resolution and 4 seconds using the plain input images. Our implementation also uses OpenMP parallelism. However, our algorithm only uses 50% of the CPU. Confirming the superior efficiency of Boosting-based detection approaches.

Finally, we compare our approach with other Boosting algorithms in KITTI testing (see Table 3): SubCat [16], Regionlets [17] and spLBP [39]. We use LDCF features [33], like SubCat, whereas Regionlets and spLBP benefit from a more discriminative set of features. However, we cannot fairly compare the Regionlets result with ours, since the KITTI experiment is not described in the original paper. Similarly, the spLBP paper does not describe the images used for training their test result. With all this in mind, with respect to Regionlets, our method has better precision up to a recall of 0.7 (see Fig. 11, Moderate). Compared with SubCat, our approach is better in the Easy subset and with recall values below 0.7.

Figure 12.

Features selected by BAdaCost in the cyclists detection problem (top left) and sample training images (rest). Note that most of the features are around the rider and the wheels.

6.2 Cyclist detection experiments

For completeness, in this section we evaluate the proposed technique in the cyclists detection KITTI dataset. As for cars, shrinkage is 0.05 and 1/32 the fraction of features to train each weak learner. There are only 1027 cyclists examples in the KITTI-train90 subset. Given the few available training images, the number of view classes is set to $K=$ 5. This election provides training data for all views, i.e. 101, 298, 155, 161 and 312 training examples respectively for class 1, 2, 3, 4 and 5. After image flipping we get 2054 training samples (i.e. 181, 652, 318, 189, 714 respectively on each positive class.) We train a 48 $\times$ 72 pixels model, $AR=$ 1.5. We start the pyramid one octave above the actual image size. We use the parameters obtained with the best AP: the best number of cost-sensitive trees is $T=$ 2048 (4 rounds with 32, 128, 256 and $T$ weak learners, respectively), tree depth is $D=$ 5, the number of negatives per round is $N=$ 2500 and the total amount of negatives is $NA=$ 10000. Finally, the cost matrix is defined to weigh up gross errors between view classes.

Table 4
Cyclist detection ablation study

Training/testing	Algorithm	Easy		Moderate		Hard
		AP ${}_{11}$	AP ${}_{40}$	AP ${}_{11}$	AP ${}_{40}$	AP ${}_{11}$	AP ${}_{40}$
Train90/train10	BAdacost $+$ I	87.2%	87.8%	79.5%	80.4%	71.6%	73.0%
	BAdacost $+$ BB	62.2%	62.7%	53.8%	56.0%	53.1%	51.1%
	BAdacost $+$ DA	83.4%	85.9%	79.6%	82.7%	75.2%	75.5%
	BAdacost $+$ DA $+$ BB	87.3%	89.4%	79.7%	83.1%	75.0%	75.7%
	BAdacost $+$ DA $+$ BB $+$ MS	91.7%	94.3%	85.8%	87.4%	78.5%	79.3%

Key: Stronger regularization (I), Bounding Box estimation (BB), Data Augmentation (DA), and multi-scale detection (MS), average precision with $n$ curve points (AP ${}_{n}$ ).

Table 5

Cyclist detection experiment comparison in KITTI-testing evaluation server

Training/testing	Algorithm	Easy		Moderate		Hard
		AP ${}_{11}$	AP ${}_{40}$	AP ${}_{11}$	AP ${}_{40}$	AP ${}_{11}$	AP ${}_{40}$
Train90/testing	BAdacost $+$ DA $+$ BB	30.3%	26.9%	24.3%	20.0%	21.0%	16.8%
	BAdacost $+$ DA $+$ BB $+$ MS	36.9%	33.9%	29.3%	25.5%	25.4%	21.1%
Unkown/testing (Boosting)	Regionlets [17]	70.1%	71.1%	58.7%	58.5%	51.8%	50.8%
ImageNet $+$ training/testing (CNNs)	RRC [21]	85.0%	86.8%	76.5%	76.8%	65.5%	66.6%
	TuSimple [23]	81.4%	83.7%	74.3%	75.2%	64.9%	65.2%
	SubCNN [40]	77.8%	79.4%	70.8%	71.7%	62.7%	62.7%

Best BAdaCost results are shown in bold and overall best results in italic.

Figure 13.

Precision-recall curves and (AP ${}_{11}$ , AP ${}_{40}$ ) for the cyclist detection ablation study.

Again, we train one detector with the mean aspect ratio of each view class stored on the tree leaves. We use KITTI-train90 for training and KITTI-train10 for testing. We set the detection threshold to an IoU of 0.5, as established in the KITTI benchmark for cyclists, needing less precision in the bounding box than in the cars case. Thus, it is expected for the bounding box adjustment procedure to have less impact in this detection problem.

We show in Fig. 12 sample training data and the selected features. In Fig. 13 and Table 4 we display the result of this final set of experiments. Our detections deteriorate if we include the bounding box adjustment procedure, “BAdaCost $+$ I $+$ BB”. The reason is that there is a lack of data to estimate the per class aspect ratio for each node in our decision trees. However, there is data to estimate the overall class mean aspect ratio. Hence the improvement with “BAdacost $+$ I”. When we augment the data with synthetic samples, “BAdaCost $+$ DA”, we get better AP ${}_{40}$ than “BAdaCost $+$ BB” and “BAdaCost $+$ I” in the Moderate and Hard cases. However, the new data worsen the estimation in the Easy group. By combining the bounding box adjustment (BB) and the data augmentation (DA) we get the best single classifier detector, “BAdaCost $+$ DA $+$ BB”, which in the Moderate examples is reporting 2.7% AP ${}_{40}$ improvement with respect to base line, from 80.4 to 83.1.

Finally, to improve AP ${}_{40}$ , we train three detectors at different base sizes: 48 $\times$ 72, 56 $\times$ 84 and 64 $\times$ 96. We use these classifiers with an image pyramid obtaining and improvement greater than 6% with respect to the base line in all cases, “BAdaCost $+$ DA $+$ BB $+$ MS”.

As in the car case, the best CNN based detectors with cyclists that use only RGB images [21, 40, 23] use a pretrained VGG network on ImageNet and finetune it with the KITTI data. It makes these results no fully comparable to the Boosting ones trained only on the KITTI dataset. In the cyclists, because of the lack of training data, almost all methods take advantage of the LIDAR measures and/or the stereo pairs available. We use only RGB images data. In Table 5 we show the results of the three best published cyclists detectors on KITTI testing using only RGB information: RRC [21], TuSimple [23] and SubCNN [40].

In Table 5 we cite [17] the top Boosting result in the KITTI evaluation server. However, we cannot fairly compare this result with ours, since the KITTI experiment is not described in the paper.

In summary, our results in this section support the use of new techniques borrowed from the deep learning literature for training Boosting based detectors.

7. Conclusions

Detection algorithms have evolved over time by changing various components of their pipeline. Some of these improvements, however, have been exploited only in the context of modern deep neural nets. In this paper we have improved the performance of Boosting-based detectors by using data augmentation, refining the detection bounding box, and using multi-scale processing. This is a relevant result in the construction of detectors, given the computational efficiency of this family of algorithms.

In the experiments we show that Boosting-based approaches significantly improve their performance with respect to the multi-class baseline Boosting scheme [24] when using the new training techniques. We achieve an improvement of 7.1% and 5% respectively in the AP ${}_{40}$ of the car and cyclist KITTI benchmarks. On the other hand, the our Boosting-based detectors are between 2X and 3.5X faster than their deep learning-based counterpart, at the expense of a significant decrease in precision.

In spite of the remarkable accuracy achieved by deep learning-based detectors, we believe there is still room for research in approaches based on more efficient traditional machine learning techniques and their application in different problems [41]. In the future we will investigate better box regression algorithms, the use of efficient and more discriminative image features and detection strategies better than the brute force sliding window.

Footnotes

https://github.com/pdollar/toolbox.

See BdCost $+$ DA $+$ BB and BdCost $+$ DA $+$ BB $+$ MS at the KITTI results page: http://www.cvlibs.net/datasets/kitti/eval_object.php.

https://github.com/xiaohaoChen/rrc_detection.

http://www.openblas.net.

Acknowledgments

This work was funded by the Spanish Ministerio de Economía y Competitividad, project TIN2016-75982-C2-2-R. José M. Buenaposada was also partially funded by the Comunidad de Madrid project RoboCity2030-DIH-CM (S2018/NMT-4331).

References

Kazemi

Sullivan

. One Millisecond Face Alignment with an Ensemble of Regression Trees. In: CVPR; 2014.

Ferrari

Marin-Jimenez

Zisserman

. Progressive Search Space Reduction for Human Pose Estimation. In: Proc. IEEE Conf. on Computer Vision and Pattern Recognition; 2008.

Chen

Kundu

Zhang

Fidler

Urtasun

. Monocular 3D Object Detection for Autonomous Driving. In: Proc. IEEE Conf. on Computer Vision and Pattern Recognition; 2016. pp. 2147–2156.

Molina-Cabello

Luque-Baena

López-Rubio

Thurnhofer-Hemsi

. Vehicle type detection by ensembles of convolutional neural networks operating on super-resolved images. Integrated Computer-Aided Engineering. 2018; 25(4): 321–333.

Jaderberg

Vedaldi

Zisserman

. Deep Features for Text Spotting. In: Proc. European Conf. Computer Vision; 2014.

Viola

Jones

. Fast and Robust Classification using Asymmetric AdaBoost and a Detector Cascade. In: Conf. Neural Information Processing Systems; 2001. pp. 1311–1318.

Dalal

Triggs

. Histograms of oriented gradients for human detection. In: Proc. IEEE Conf. on Computer Vision and Pattern Recognition. Vol. 1; 2005. pp. 886–893.

Dollar

Appel

Belongie

Perona

. Fast feature pyramids for object detection. IEEE Trans Pattern Analysis and Machine Intelligence. 2014; 36(8): 1532–1545.

Gall

Yao

Razavi

Gool

Lempitsky

. Hough forests for object detection, tracking, and action recognition. IEEE Trans Pattern Analysis and Machine Intelligence. 2011 Nov; 33(11): 2188–2202.

10.

Ren

Girshick

Sun

. Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. In: Conf. Neural Information Processing Systems; 2015.

11.

Redmon

Divvala

Girshick

Farhadi

. You Only Look Once: Unified, Real-Time Object Detection. In: Proc. IEEE Conf. on Computer Vision and Pattern Recognition; 2016.

12.

Liu

Anguelov

Erhan

Szegedy

Reed

, et al. SSD: Single Shot MultiBox Detector. In: Proc. European Conf. Computer Vision; 2016. pp. 21–37.

13.

Zhang

Benenson

Schiele

. Filtered channel features for pedestrian detection. In: Proc. IEEE Conf. on Computer Vision and Pattern Recognition; 2015.

14.

Benenson

Mathias

Timofte

Van Gool

. Pedestrian detection at 100 frames per second. In: Proc. IEEE Conf. on Computer Vision and Pattern Recognition; 2012.

15.

Mathias

Benenson

Pedersoli

Van Gool

. Face detection without bells and whistles. In: Proc. European Conf. Computer Vision; 2014.

16.

Ohn-Bar

Trivedi

. Learning to detect vehicles by clustering appearance patterns. IEEE Trans Intelligent Transportation Systems. 2015 Oct; 16(5): 2511–2521.

17.

Wang

Yang

Zhu

Lin

. Regionlets for generic object detection. IEEE Trans Pattern Analysis and Machine Intelligence. 2015; 37(10): 2071–2084.

18.

Ahmadlou

Adeli

. Enhanced probabilistic neural network with local decision circles: a robust classifier. Integr Comput-Aided Eng. 2010 Aug; 17(3): 197–210.

19.

Rafiei

Adeli

. A new neural dynamic classification algorithm. IEEE Transactions on Neural Networks and Learning Systems. 2017 Dec; 28(12): 3074–3083.

20.

Chabot

Chaouch

Rabarisoa

Teulière

Chateau

. Deep MANTA: A Coarse-to-fine Many-Task Network for joint 2D and 3D vehicle analysis from monocular image. CoRR. 2017; abs/1703.07570.

21.

Ren

Chen

Liu

Sun

Pang

Yan

, et al. Accurate Single Stage Detector Using Recurrent Rolling Convolution. In: Proc. IEEE Conf. on Computer Vision and Pattern Recognition; 2017.

22.

Juranek

Herout

Dubska

Zemcik

. Real-Time Pose Estimation Piggybacked on Object Detection. In: Proc. Int’l Conf. Computer Vision; 2015.

23.

Yang

Choi

Lin

. Exploit All the Layers: Fast and Accurate CNN Object Detector With Scale Dependent Pooling and Cascaded Rejection Classifiers. In: Proc. IEEE Conf. on Computer Vision and Pattern Recognition; 2016.

24.

Fernández-Baldera

Buenaposada

Baumela

. BAdaCost: multi-class boosting with costs. Pattern Recognition. 2018; 79: 467–479.

25.

Ohn-Bar

Trivedi

. To Boost or Not to Boost? On the Limits of Boosted Trees for Object Detection. In: Proc. Int’l Conf. Pattern Recognition; 2016.

26.

Friedman

Hastie

Tibshirani

. Additive logistic regression: a statistical view of boosting. The Annals of Statistics. 2000; 28(2): 337–407.

27.

Fernández-Baldera

Buenaposada

Baumela

. Multi-class Boosting for Imbalanced Data. In: Proc. of Iberian Conf. Pattern Recognition and Image Analysis; 2015. pp. 57–64.

28.

Zhu

Zou

Rosset

Hastie

. Multi-class AdaBoost. Statistics and its Interface. 2009; 2: 349–360.

29.

Zhang

Viola

. Multiple-Instance Pruning For Learning Efficient Cascade Detectors. In: Conf. Neural Information Processing Systems; 2007. pp. 1681–1688.

30.

Yang

Yan

Lei

. Aggregate channel features for multi-view face detection. In: IEEE Int’l J. Conference on Biometrics; 2014. pp. 1–8.

31.

Ramanan

. Finding Tiny Faces. In: Proc. IEEE Conf. on Computer Vision and Pattern Recognition; 2017.

32.

Dollar

Belongie

Perona

. The Fastest Pedestrian Detector in the West. In: Proc. British Machine Vision Conf.; 2010. pp. 68.1–11.

33.

Nam

Dollar

. Local Decorrelation For Improved Pedestrian Detection. In: Conf. Neural Information Processing Systems; 2014.

34.

Geiger

Lenz

Urtasun

. Are we ready for Autonomous Driving? The KITTI Vision Benchmark Suite. In: Proc. IEEE Conf. on Computer Vision and Pattern Recognition; 2012.

35.

Simonelli

Bulò

SRR

Porzi

López-Antequera

Kontschieder

. Disentangling Monocular 3D Object Detection; 2019.

36.

Buenaposada

Baumela

. Boosting Object Detection in Cyberphysical Systems. In: Understanding the Brain Function and Emotions, LNCS volume 11486. Cham: Springer International Publishing; 2019. pp. 309–318.

37.

Simonyan

Zisserman

. Very Deep Convolutional Networks for Large-Scale Image Recognition. In: International Conference on Learning Representations; 2015.

38.

Russakovsky

Deng

Krause

Satheesh

, et al. Image net large scale visual recognition challenge. International Journal of Computer Vision (IJCV). 2015; 115(3): 211–252.

39.

Paisitkriangkrai

Shen

van den Hengel

Porikli

. Fast detection of multiple objects in traffic scenes with a common detection framework. IEEE Transactions on Intelligent Transportation Systems. 2016 April; 17(4): 1002–1014.

40.

Xiang

Choi

Lin

Savarese

. Subcategory-Aware Convolutional Neural Networks for Object Proposals and Detection. In: 2017 IEEE Winter Conference on Applications of Computer Vision (WACV); 2017. pp. 924–933.

41.

Roegiers

Allebosch

Veelaert

Philips

. Human action recognition using hierarchic body related occupancy maps. Integrated Computer-Aided Engineering. 2019; 26(3): 223–241.

42.

Serpanos

. The cyber-physical systems revolution. Computer. 2018 March; 51(3): 70–73.

43.

Wolf

. Cyber-physical systems. Computer. 2009 March; 42(3): 88–89.

44.

Diller

Metin Sitti

. Micro-scale mobile robotics. Foundations and Trends in Robotics. 2013; 2(3): 143–259.

45.

Liu

Ouyang

Wang

Fieguth

Chen

Liu

, et al. Deep Learning for Generic Object Detection: A Survey. CoRR. 2018; abs/1809.02165. Available from: http://arxiv.org/abs/1809.02165.

46.

Guo

Viktor

. Learning from imbalanced data sets with boosting and data generation: the DataBoost-IM approach. SIGKDD Explorations Newsletter. 2004 Jun; 6(1): 30–39.

47.

Kumar

Banerjee

Vemuri

Pfister

. Trainable convolution filters and their application to face recognition. IEEE Trans Pattern Analysis and Machine Intelligence. 2012; 34(7): 1423–1436.

48.

Han

Otto

Liu

Jain

. Demographic estimation from face images: human vs. machine performance. IEEE Trans Pattern Analysis and Machine Intelligence. 2015; 37(6): 1148–1161.

49.

Wang

Hua

Sukthankar

Xue

Niu

Zheng

. Video Object Discovery and Co-Segmentation with Extremely Weak Supervision. IEEE Trans Pattern Analysis and Machine Intelligence. 2017.

50.

Shotton

Blake

Cipolla

. Multiscale categorical object recognition using contour fragments. IEEE Trans Pattern Analysis and Machine Intelligence. 2008 July; 30(7): 1270–1281.

51.

Mahamud

Hebert

Shi

. Object recognition using boosted discriminants. In: Proc. IEEE Conf. on Computer Vision and Pattern Recognition. Vol. 1; 2001. pp. I–551–I–558.

52.

Dietterich

Bakiri

. Solving Multiclass Learning Problems via Error-Correcting Output Codes. J of Artificial Intelligence Research. 1995; 263–286.

53.

Freund

Schapire

. Experiments with a New Boosting Algorithm. In: Proc. Int’l Conf. on Machine Learning; 1996. pp. 148–156.

54.

Freund

Schapire

. A decision theoretic generalization of on-line learning and an application to boosting. J of Computer and System Sciences. 1997; 55: 199–139.

55.

Schapire

. Using Output Codes to Boost Multiclass Learning Problems. In: Proc. Int’l Conf. on Machine Learning; 1997. pp. 313–321.

56.

O’Brien

Gupta

Gray

. Cost-sensitive Multi-class classification from probability estimates. In: Proc. Int’l Conf. on Machine Learning; 2008. pp. 712–719.

57.

Ting

Zheng

. Boosting cost-sensitive trees. In: Proc. International Conference on Discovery Science; 1998. pp. 244–255.

58.

Domingos

. MetaCost: A General Method for Making Classifiers Cost-Sensitive. In: Proc. Int’l Conf. on Knowledge Discovery and Data Mining; 1999. pp. 155–164.

59.

Fan

Stolfo

Zhang

Chan

. AdaCost: Misclassification Cost-sensitive Boosting. In: Proc. Int’l Conf. on Machine Learning; 1999. pp. 97–105.

60.

Guruswami

Sahai

. Multiclass learning, boosting and error correcting codes. In: Proc. Annual Conference on Learning Theory; 1999. pp. 145–155.

61.

Schapire

Singer

. Improved boosting algorithms using confidence-rated predictions. Machine Learning. 1999; 37: 297–336.

62.

Allwein

Schapire

Singer

. Reducing multiclass to binary: a unifying approach for margin classifiers. J of Machine Learning Research. 2000; 1: 113–141.

63.

Ting

. A Comparative Study of Cost-Sensitive Boosting Algorithms. In: Proc. Int’l Conf. on Machine Learning; 2000. pp. 983–990.

64.

Elkan

. The Foundations of Cost-Sensitive Learning. In: Proc. Int’l Joint Conf. on Artificial Intelligence; 2001. pp. 973–978.

65.

Abe

Zadrozny

Langford

. An iterative method for multi-class cost-sensitive learning. In: Proc. Int’l Conf. on Knowledge Discovery and Data Mining; 2004. pp. 3–11.

66.

Lee

Lin

Wahba

. Multicategory Support Vector Machines: theory and application to the classification of microarray data and satellite radiance data. J American Statistical Association. 2004; 99: 67–81.

67.

Torralba

Murphy

Freeman

. Sharing Features: Efficient Boosting Procedures for Multiclass Object Detection. In: Proc. IEEE Conf. on Computer Vision and Pattern Recognition; 2004. pp. 762–769.

68.

Viola

Jones

. Robust real-time face detection. Int’l JComputer Vision. 2004; 57(2): 137–154.

69.

Bourdev

Brandt

. Robust object detection via soft cascade. In: Proc. IEEE Conf. on Computer Vision and Pattern Recognition. Vol. 2; 2005. pp. 236–243.

70.

Sun

Wong

AKC

Wang

. Parameter Inference of Cost-Sensitive Boosting Algorithms. In: Proc. Int’l Conf. on Machine Learning and Data Mining; 2005. pp. 21–30.

71.

Demsar

. Statistical comparisons of classifiers over multiple data sets. J of Machine Learning Research. 2006; 7: 1–30.

72.

Sun

Kamel

Wang

. Boosting for Learning Multiple Classes with Imbalanced Class Distribution. In: Proc. Int’l Conference on Data Mining; 2006. pp. 592–602.

73.

Masnadi-Shirazi

Vasconcelos

. Asymmetric Boosting. In: Proc. Int’l Conf. on Machine Learning; 2007. pp. 609–619.

74.

Sun

Kamel

Wong

AKC

Wang

. Cost-sensitive boosting for classification of imbalanced data. Pattern Recognition. 2007; 40(12): 3358–3378.

75.

Lozano

Abe

. Multi-class Cost-sensitive Boosting with p-norm Loss Functions. In: Proc. Int’l Conf. on Knowledge Discovery and Data Mining; 2008. pp. 506–514.

76.

Zou

Zhu

Hastie

. New multicategory boosting algorithms based on multicategory fisher-consistent losses. Annals of Applied Statistics. 2008; 2: 1290–1306.

77.

Garcia

. Learning from imbalanced data. IEEE Trans on Konwledge and Data Engineering. 2009; 21(9): 1263–1284.

78.

Xia

Yang

Zhou

Cai

Zeng

. A closed-form reduction of multi-class cost-sensitive learning to weighted multi-class learning. Pattern Recognition. 2009; 42(7): 1572–1581.

79.

Zhou

Liu

. On multi-class cost-sensitive learning. Computational Intelligence. 2010; 26(3): 232–257.

80.

Everingham

Van Gool

Williams

CKI

Winn

Zisserman

. The pascal visual object classes (VOC) challenge. Int’l JComputer Vision. 2010 Jun; 88(2): 303–338.

81.

Jain

Learned-Miller

. FDDB: A Benchmark for Face Detection in Unconstrained Settings. University of Massachusetts, Amherst; 2010. UM-CS-2010-009.

82.

östinger

Wohlhart

Roth

Bischof

. Annotated Facial Landmarks in the Wild: A large-scale, real-world database for facial landmark localization. In: Proc. Int’l Conf. Computer Vision Workshops; 2011. pp. 2144–2151.

83.

Masnadi-Shirazi

Vasconcelos

. Cost-sensitive boosting. IEEE Trans Pattern Analysis and Machine Intelligence. 2011; 33(2): 294–309.

84.

Saberian

Vasconcelos

. Multiclass Boosting: Theory and Algorithms. In: Conf. Neural Information Processing Systems; 2011.

85.

Landesa-Vázquez

Alba-Castro

. Shedding light on the asymmetric learning capability of AdaBoost. Pattern Recognition Letters. 2012; 33(3): 247–255.

86.

Liu

Zhou

. Towards Cost-Sensitive Learning for Real-World Applications. In: Pacific-Asia Conf. Knowledge Discovery and Data Mining Workshops. Vol. 7104; 2012. pp. 494–505.

87.

Zhu

Ramanan

. Face detection, pose estimation, and landmark localization in the wild. In: Proc. IEEE Conf. on Computer Vision and Pattern Recognition; 2012. pp. 2879–2886.

88.

Benenson

Mathias

Tuytelaars

Van Gool

. Seeking the strongest rigid detector. In: Proc. IEEE Conf. on Computer Vision and Pattern Recognition; 2013.

89.

Mathias

Benenson

Timofte

Van Gool

. Handling Occlusions with Franken-classifiers. In: Proc. Int’l Conf. Computer Vision; 2013.

90.

Wang

. Boosting the generalized margin in cost-sensitive multiclass classification. J Computational and Graphical Statistics. 2013; 22(1): 178–192.

91.

Fernández-Baldera

Baumela

. Multi-class boosting with asymmetric weak-learners. Pattern Recognition. 2014; 47(5): 2080–2090.

92.

Benenson

Omran

Hosang

Schiele

. Ten years of pedestrian detection, what have we learned? In: Proc. European Conf. Computer Vision, CVRSUAD Workshop; 2014.

93.

Parambath

SAP

Usunier

Grandvalet

. Optimizing F-measures by Cost-sensitive Classification. In: Conf. Neural Information Processing Systems; 2014. pp. 2123–2131.

94.

Zhu

. Learning and-or model to represent context and occlusion for car detection and viewpoint estimation. IEEE Trans Pattern Analysis and Machine Intelligence. 2016 Sept; 38(9): 1829–1843.

95.

Yang

Yan

Lei

. Convolutional Channel Features. In: Proc. Int’l Conf. Computer Vision; 2015. pp. 82–90.

96.

Nikolaou

Edakunni

Kull

Flach

Brown

. Cost-sensitive boosting algorithms: do we really need them? Machine Learning. 2016; 104(2–3): 359–384.

97.

Cai

Fan

Feris

Vasconcelos

. In: A Unified Multi-scale Deep Convolutional Neural Network for Fast Object Detection. Springer; 2016. pp. 354–370.

98.

Oeljeklaus

Hoffmann

Bertram

. A Fast Multi-Task CNN for Spatial Understanding of Traffic Scenes. In: 2018 21st International Conference on Intelligent Transportation Systems (ITSC); 2018. pp. 2825–2830.

Improving multi-class Boosting-based object detection

Abstract

Keywords

1. Introduction

2. Multi-class boosting algorithm

2.1 BAdaCost: Cost-sensitive multi-class Boosting classification

6. Experiments

6.1 Car detection experiments

Table 4 Cyclist detection ablation study

Footnotes

Acknowledgments

References

Table 4
Cyclist detection ablation study