Person re-identification based on multi-scale global feature and weight-driven part feature

Abstract

Person re-identification (ReID) is a crucial task in identifying pedestrians of interest across multiple surveillance camera views. ReID methods in recent years have shown that using global features or part features of the pedestrian is extremely effective, but many models do not have further design models to make more reasonable use of global and part features. A new model is proposed to use global features more rationally and extract more fine-grained part features. Specifically, our model captures global features by using a multi-scale attention global feature extraction module, and we design a new context-based adaptive part feature extraction module to consider continuity between different body parts of pedestrians. In addition, we have added additional enhancement modules to the model to enhance its performance. Experiments show that our model achieves competitive results on the Market1501, Dukemtmc-ReID, and MSMT17 datasets. The ablation experiments demonstrate the effectiveness of each module of our model. The code of our model is available at: https://github.com/davidtqw/Person-Re-Identification.

Keywords

Person re-identification global feature part feature multi-scaler weight-driven non-local mudule

1. Introduction

The purpose of the Person re-identification(ReID) task is to determine whether the pedestrians captured by different cameras belong to the same pedestrian and the ReID is generally used to address cross-camera tracking and surveillance security issues. Compared with face recognition tasks, the ReID task has many practical application problems, i.e., the appearance of pedestrians is easily influenced by lighting conditions, posture, viewpoint, and resolution. Some examples are shown in Fig. 1.

Fig. 1.

Some of the challenges faced by person re-identification in the Market1501 dataset. (a). Same pedestrian images under different lighting conditions. (b). Same pedestrian images under different posture. (c). Same pedestrian images under different viewpoints. (d). Same pedestrian images under different resolution cameras.

Traditional methods focused on low-level features such as color [7], texture, and some combination of low-level features [8]. In recent years, convolutional neural networks (CNN) have achieved great success in many computer vision tasks. Compared with traditional methods, deep learning technology performs better in the ReID task [49]. Most of the current ReID methods based on deep learning are using global features because the global features are effective when the external factors are less variable. However, global features cannot effectively describe some fine-grained information about pedestrians. Therefore, to further improve the performance of the ReID model, some additional descriptions of pedestrians are required. After the emergence of part-based methods, some improved part-based methods [24,41,45] use enhanced part features as pedestrian descriptors, which leads to a significant improvement in the efficiency of person re-identification.

In recent years, part-based methods can be classified into three categories: (1). The first category is to divide different body parts according to the pedestrians’ posture [17,20,29]. (2). The second category is the direct division method, also known as ‘hard division’, in which horizontal stripes are cut on the image [4,22,44]. (3). The third category is the flexible division of pedestrians, also called the ‘soft division’ [25,37], and this division method is more accurate and reasonable for ReID tasks.

In this paper, we propose a new global feature extraction method and a new part division method to obtain fine-grained features of pedestrians. Compared with existing methods of global and part features, our method has several contributions as follows. (1). For global features, our method uses multi-scale convolution kernels and a new lightweight spatial attention module to extract more refined global features. (2). For part features, our method employs a part feature fusion strategy that drives the model to focus on continuous part features of pedestrians. (3). To further enhance the feature mining capability of the model, we add the Non-local module after stage1 and stage2 of the backbone network respectively. From the results, the Non-local module can effectively improve the model performance.

2. Related work

Hand-Crafted-Based Person Re-Identification. Before deep learning methods were widely used, hand-crafted methods had been proven to be effective. For example, in the literature [2], the author extracted three mutually complementary color features of pedestrians, weighted color histogram, the most stable color area, and high repetitive structure color block, then the three features were combined as a pedestrian descriptor. The author in [7] combined color features and texture features, and to improve the efficiency of ReID. Although these low-level visual features are helpful and simple, their performance is relatively poor.

In addition, the method of dividing the person image into several part features is effective. For example, the authors in [5,47] divided the image into the upper and lower body, the authors in [52] divided the pedestrian into several horizontal stripes, and in the literature [17], the authors used sliding windows to construct local descriptors. In the literature [17], the authors proposed a GOG descriptor for ReID, which uses a hierarchical Gaussian operator to describe the color and texture information of different segmented parts. Although these methods perform relatively well, the hand-crafted methods cannot adapt to changes in image viewpoint and are not comparable to deep learning methods in performance.

Global and Local Feature-Based Person Re-Identification. CNN had achieved great success in the ReID task in recent years. In the literature [16,21,31], the authors identified different pedestrians by using global features, but these methods were less robust to pose changes and viewpoint changes. In the literature [6,32,39,50], the authors extracted additional local features to complement the global features, and this method of combining global and local features overcame the viewpoint, pose and changes achieving good results. It is worth noting that the authors in [46] proposed an efficient mechanism to dynamically align the different parts of pedestrians, and it greatly reduces the identification error caused by the misalignment of the pedestrian pose. All the above methods are based on ‘hard division’, but different parts of the pedestrian’s body are continuity, and these ‘hard division’ methods ignore this continuity. To solve such problems, the authors in [41,43] proposed the ‘soft division’ strategy to make the division of pedestrians more accurate. The attention mechanism aims to focus on the most salient features in the pedestrian image and suppress unimportant ones. In the literature [27], the authors constructed an attention module by overlapping several convolutional layers. The authors in [30] used two fully connected (FC) layers to compute the channel attention map of an image. The authors in [12] proposed a relation-aware global attention (RGA) model that extracts useful information from global features, and this method effectively considers the relationship between each feature node and the global features.

To make more rational use of global features and part features, we propose a multi-scale global feature module and a part feature fusion module, and the multi-scale global feature module extracts fine-grained global features by using several different scales convolution kernels and a new lightweight spatial attention module, and the weight-driven part feature fusion module uses an adaptive part feature fusion strategy to drive the model to consider the correlation between each part and its neighboring parts.

3. Method

Our model uses ResNet50 [10] as the backbone network, but we add some extra modules in the model to improve the feature extraction ability of the model. Specifically, we introduce three feature mining modules. (1). Non-local modules [34]. The traditional Resnet50 network contains five basic modules as feature extraction units (stage0-stage4), and we add the non-local module after stage1 and stage2 respectively to enhance the model’s salient feature extraction ability. (2). Multi-scale global feature module. We introduce a multi-scale global feature module to extract more comprehensive and refine global features by using multiple convolution kernels of different scales and the new lightweight spatial attention module. (3). Weight-driven part feature fusion module. We fuse each part feature of a pedestrian with neighboring part information by using a weight-driven adaptive method. Compared with mainstream part-based methods, our method not only considers some information contained in a single part but also designs an adaptive weight-driven method to learn the correlation between pedestrian part features and adjacent part features. The architecture of our model is shown in Fig. 2.

Fig. 2.

Overview of our training/testing framework.

3.1. Non-local module

Convolutional neural networks have some shortcomings when applied to the ReID task. For example, (1). Obtaining global features with large receptive fields generally requires large-scale convolutional kernels, but large convolutional kernels introduce a large number of parameters, which leads to inefficient model training. (2). As the model deepens, the overall model gradient needs to be considered when designing other enhancement modules. (3). The traditional convolution-based backbone network is slightly inadequate in mining salient features of pedestrian images, and additional feature extraction modules are needed to help the whole network to improve the mining ability of salient features. Non-local module, which can capture a larger range of image receptive fields, was first proposed in the literature [34].

The computational flow of the Non-local module is shown in Fig. 3, from Fig. 3 we can observe that the staged features are input into three identical convolutional layers to obtain three feature maps of the same size, $f_{q}$ , $f_{k}$ and $f_{v}$ , then the final output $y_{i}$ is calculated as in Eq. (1). $\begin{matrix} (1) & y_{i} = \frac{P (F {f_{q}, f_{k}}) G (f_{v})}{C (x)} + x \end{matrix}$

Where x is the input feature map, the $F {f_{q}, f_{k}}$ is a function to calculate the correlation between $f_{q}$ and $f_{k}$ , $P ()$ is the softmax function and the $G (f_{v})$ is a dimensional transformation operation that changes the dimension of the feature map $f_{v}$ from $N \times H \times W$ to $N \times H W$ . $C (x)$ is a combination of standardized operations, which contains convolution and reshape operations, and it transforms the dimension of the feature map to be the same as the input feature map x.

Fig. 3.

Non-local module.

3.2. Multi-scale global feature module

In this section, we introduce the multi-scale global feature module. We use different combinations of convolution and pooling layers to extract global features at different scales. After several sets of experiments, we finally use branch 1 ( $1 \times 1$ Conv layer), branch 2 (the combination of $1 \times 1$ and $3 \times 3$ Conv layer), branch 3 (the combination of $1 \times 1$ and $5 \times 5$ Conv layers), and branch 4 (the combination of $3 \times 3$ average-pool and $1 \times 1$ Conv layers), to reduce the channel dimension of the feature maps. Thus the four refined global feature maps $f_{glo 1}$ , $f_{glo 2}$ , $f_{glo 3}$ , $f_{glo 4}$ have the same dimension as shown in Fig. 4.

Fig. 4.

Multi-scale global feature module.

To further enhance the feature extraction capability of the multi-scale global feature module, we designed a tiny spatial attention (TSA) module and added it to the multi-scale global feature module, as shown in Fig. 5. We perform spatial construction of the feature map with four suboperations (10 parameters), which include a global cross-channel averaging pooling operation (0 parameters), a $3 \times 3$ Conv layer with a step size of 2 (9 parameters), a bilinear interpolation operation to resize the feature map (0 parameters), and a scaling Conv layer (1 parameter), as shown in Fig. 5. Specifically, the channel averaging pooling operation [38] is an efficient operation that takes into account the information of all channels because all channels share the same spatial attention map. It is calculated according to Eq. (2). After obtaining the mean matrix $A (i, j)$ of all channels, we downsample the feature map by using a convolutional layer with kernel size = $3 \times 3$ and step size = 2. The purpose of this operation is to further extract the fine-grained features $B (i, j)$ . Finally, we obtain the spatial attention map ω by using a $1 \times 1$ convolution and sigmoid function and then multiply it with the original feature map to obtain the fine-grained global features. Then we concatenate the different scale features. The complete process is shown in Fig. 4. $\begin{matrix} (2) & A (i, j) = \frac{1}{M} \sum_{n = 0}^{M} f_{n} (i, j) \end{matrix}$

Fig. 5.

Tiny spatial-attention module.

Inspired by the ResNet residual module [10], we know that the use of residual structure can effectively solve the model gradient problem, so we sum up the multi-scale global feature map and stage4 feature map to get the final features $F_{final}$ as in Eq. (3). $\begin{matrix} (3) & F_{final} = {cat}_{\dim = channel} (σ {f_{{glo}_{1}}}, σ {f_{{glo}_{2}}}, σ {f_{{glo}_{3}}}, σ {f_{{glo}_{4}}}) + f_{glo}, \end{matrix}$ where $f_{glo}$ represents the stage4 features, $f_{{glo}_{1}}$ , $f_{{glo}_{2}}$ , $f_{{glo}_{3}}$ , $f_{{glo}_{4}}$ represents the features maps generated by branch1, branch2, branch3, branch4, ${cat}_{\dim = channel} ()$ represents the concatenation operation on the channel dimension. $σ {}$ represents the tiny spatial-attention operation.

3.3. Weight-driven part feature fusion module

In this section, we introduce the weight-driven part feature fusion module, which is independent of the multi-scale global feature module. The stage4 feature $= N \times 2048 \times 18 \times 9$ (N represents the batch size, C represents the number of channels, H and W represent the length and width of the feature map) is extracted by the backbone network, then we divide the feature map into six horizontal stripes of the same size, and that represent different body parts of the pedestrian. Intuitively, the correlation between different parts of the body is very significant, and these correlations include the continuity of pedestrian clothing color, textures, salient logos, etc. However, the ‘hard division’ method does not take into account the correlation between different parts of the body. To drive the model to consider these correlations, we construct the refine part feature by adaptive weighting. The specific method is shown in Fig. 6.

Fig. 6.

Weight-driven part feature fusion module. We divide the pedestrian image into six parts. Correspondingly, we calculate six ‘remainder parts’, which are the sum of one or two parts adjacent to the part. Then we use two FC layers and the sigmoid function to obtain two sets of weights.

Firstly, we define the sum of the parts adjacent to the current part (p1–p6 in Fig. 6) as the ‘remainder part’ (t1–t6 in Fig. 6). In general, it is difficult to determine the importance between part features and ‘remainder part’ features rapidly by setting the weights manually, in order to drive the model to explore the importance of each part and remainder part adaptively, we use two FC layers and a sigmoid function to obtain adaptive weights for the part and the ‘remainder part’. Specifically, we use $w_{p i}$ and $w_{t i}$ ( $i = 1, . ., 6$ ) to represent the adaptive weight, as in Eq. (4), (5) and $t_{i}$ ( $i = 1, . ., 6$ ) represents the features adjacent to $p_{i}$ (when $p_{i}$ is a part feature, and ‘remainder part’ $t_{i}$ is one or two features adjacent to it). Finally, the refined pedestrian part descriptors are obtained by multiplying the weights matrix $w_{p i} / w_{t i}$ with the part features $p_{i} / t_{i}$ , as shown in Eq. (6). $\begin{array}{l} (4) & w_{p i} = Sigmoid (FC (p_{i})), (i = 1, . ., 6) \\ (5) & w_{t i} = Sigmoid (FC (t_{i})), (i = 1, . ., 6) \end{array}$

Where FC and sigmoid represent two full connection layers and sigmoid functions respectively. $p_{i}$ is a part feature, and $t_{i}$ is one or two features adjacent to it $p_{i}$ . $\begin{matrix} (6) & f_{i} = w_{p i} \times p_{i} + w_{t i} \times t_{i}, (i = 1, . ., 6) \end{matrix}$

Where we use adaptive weights $w_{p i}$ and $w_{t i}$ to multiply $p_{i}$ and $t_{i}$ respectively to obtain the refined pedestrian part descriptors $f_{i}$ .

3.4. Secondary modules

After the images are input to the backbone network, we get the features of stage 3 and stage 4. The feature of stage 3 are shallow features and is not enough to be used as the final classification feature. The stage 4 features are the final features obtained from the Resnet50 network, which contains precise image information. We perform average pooling and maximum pooling operations on the features of the two stages and then perform summation operations on the average pooled features and the maximum pooled features, finally input them to the FC layer to obtain the prediction results for calculating the cross-entropy loss. In addition, we calculate the triplet loss for the average pooled features and the maximum pooled features respectively, as shown in Fig. 2.

3.5. Loss computation

Label smoothing cross-entropy loss can weaken the supervision and prevent the overfitting of the model to some extent, and this strategy is also widely used in classification tasks. The label-smoothed cross-entropy loss is shown in Eq. (7). $\begin{matrix} (7) & L_{id} = \sum_{i = 1}^{C} - q_{i} log (p_{i}), q_{i} = \{\begin{matrix} 1 - \frac{C - 1}{C} α, & i = y \\ α / C, & i \neq y \end{matrix} \end{matrix}$ where C is the number of person IDs in the training set, y represents the person ID label, $p_{i}$ is the ID prediction logits of class i, α is a small constant to encourage the model to be less confident on the training set. In this paper, we set $α = 0.1$ .

In addition, to measure the most similar pedestrians from the dataset, we introduce the triplet loss to drive the model to find useful features, and the triplet loss is used to improve the final ranking performance. We calculate the triplet loss according to Eq. (8). $\begin{matrix} (8) & L_{tr} = \frac{1}{N} {\sum_{i}^{M} {[{| f (x_{i}^{a}) - f (x_{i}^{P}) |}^{2} - {| f (x_{i}^{a}) - f (x_{i}^{N}) |}^{2} + margin]}_{+}} \end{matrix}$

Where $x_{i}^{a}$ is a feature map of the training sample, $x_{i}^{P}$ is the positive sample feature map of the same label as training sample, and $x_{i}^{N}$ is the negative sample feature map of a different label from the training sample. $margin$ is the margin parameter, which aims to increase the distance between negative samples and reduce the distance between positive samples. In this paper, we set the $margin$ to 0.3.

In this paper, our loss computation is divided into three main parts. (1). For stage3 and stage4 features, we compute the triplet loss for the average pooled feature and the maximum pooled feature and compute the cross-entropy loss for the sum of the average pooled feature and the maximum pooled feature. (2). For the multi-scale global feature module, we only compute its cross-entropy loss. (3). For the weight-driven part feature fusion module, we calculate the cross-entropy loss for each $f_{i}$ respectively.

Finally, we jointly train an end-to-end multi-stage network loss with intermediate supervision and we train and optimize the model according to the loss of Eq. (9). $\begin{matrix} (9) & L_{sum} = L_{stage_id} / 2 + L_{stage_tr} / 4 + L_{part_id} / 6 \end{matrix}$ where $L_{stage_id}$ represents the sum of the cross-entropy loss of stage3 and stage4. $L_{part_id}$ represents the sum of the cross-entropy loss of all $f_{i}$ . $L_{stage_tr}$ represents the triple loss of the average pooling feature and maximum pooling feature of stage3 and stage4. The weights in front of $L_{stage_id}$ , $L_{stage_tr}$ and $L_{part_id}$ represent the number of times they are computed (e.g., we calculate the cross-entropy loss of the two sets of stage features, so multiply by one-half, and we compute cross-entropy loss for six groups of parts so divide by six to get the average).

3.6. Model test

In the model testing phase, we input the query image to the backbone network to obtain the stage 3 features and stage 4 features. Then we sum the global maximum-pooled features and global average-pooled features of stage3 to obtain $f_{3}$ , which is used as the descriptor of stage 3 feature. We perform the same operation on the stage 4 feature to obtain the descriptor $f_{4}$ . Finally, we concatenate $f_{3}$ and $f_{4}$ to obtain $f_{test}$ , and we use it to match the image features of the gallery. The testing framework is shown in Fig. 2.

4. Experiments

4.1. Datasets

In this paper, we use three datasets to evaluate our model, which are Market-1501 [48], DukeMTMC-ReID [53], and MSMT17 [36]. We summarize the detailed data of the three datasets in Table 1.

Market-1501. The Market-1501 dataset is a medium-sized dataset proposed in recent years, and many existing ReID methods use the Market-1501 dataset as a benchmark. The dataset is divided into a training set with 12,936 images of 751 persons and a testing set with 3,368 query images and 19,732 gallery images of 750 persons.

Table 1
Statistics of the datasets used for training and testing

Dataset Train IDs Train Images Test IDs Query Images Cameras Total Images

Market-1501 751 12,936 750 3368 6 32,217

DukeMTMC-reID 702 16,522 702 2228 8 36,441

MSMT17 1041 32,621 3060 11,659 15 126,441

Dataset	Train IDs	Train Images	Test IDs	Query Images	Cameras	Total Images
Market-1501	751	12,936	750	3368	6	32,217
DukeMTMC-reID	702	16,522	702	2228	8	36,441
MSMT17	1041	32,621	3060	11,659	15	126,441

DukeMTMC-ReID. Compare with market1501, DukeMTMC-ReID is a larger dataset, and there is a high similarity between different pedestrians. The dataset consists of 36,411 images of 1,812 persons from 8 high-resolution cameras, of which 16,522 images of 702 persons are randomly selected from the dataset as the training set, and the remaining 702 persons are divided into the testing set which contains 2,228 query images and 17,661 gallery images.

MSMT17. The MSMT17 dataset is closer to the real scene and it proposed in 2018. It is a most challenging dataset in recent years because some existing methods do not perform well on this dataset. The training set of the MSMT17 dataset contains 32621 images from 1041 pedestrians, and the testing set contains 93820 images from 3060 pedestrians.

Note: The ReID task is tested with a query image, the trained model is used to extract features, and then we find the same pedestrians in the gallery image by feature matching (e.g. Euclidean distance). Thus, in the public dataset of ReID, there is a “query image folder” containing the pedestrian images to be queried and a “gallery image folder” containing the queried pedestrian images.

4.2. Implementation details

Experimental Platform. We use python 3.8 to write the code and train the model on an RTX20606G graphics card.

Preparation Phase. We pre-train Resnet50 on ImageNet [3] as the backbone network. However, we remove the FC layer in the original structure and change the down sampling of the last layer to $1 \times 1$ to obtain a larger feature map. Finally, we adjust the image size of each database to $288 \times 144$ .

Learning Rate. The warm-up strategy [19], has played an important role in many deep learning tasks. In the first five rounds of training, we use the gradually increasing learning rate to initialize the network parameters and optimize the model to better search space. This is a more effective strategy commonly used in deep learning tasks. To accelerate the convergence of the network, we update the learning rate, which attenuates the learning rate by 0.1 times for every 40 rounds, as in Eq. (10). $\begin{matrix} (10) & lr (t) = \{\begin{matrix} 5 \times 10^{- 2} \times \frac{t}{5} & if t ⩽ 5 \\ 5 \times 10^{- 3} & if 5 > t ⩾ 40 \\ 5 \times 10^{- 4} & if 40 > t ⩾ 80 \\ 5 \times 10^{- 5} & if 80 > t ⩾ 100 \end{matrix} \end{matrix}$

Where $lr (t)$ represents the learning rate, t represent the number of training rounds. In the paper, we use a simple linear strategy [19] to gradually increase the learning rate from 0 to an initial learning rate of 0.05 for the first 5 epochs, and the learning rate is down to one-tenth of the original rate every forty rounds.

Optimizer. To make the network fully converge, we set the number of training rounds to 100 on three datasets. To accelerate the convergence of the network, we use the stochastic gradient descent (SGD) optimizer to update the model parameter.

Data Enhancement. Each image has a probability of being cropped and flipped horizontally in each iteration, and the probability parameter is set to 0.5. In addition, to simulate the real scene in each iteration, we randomly adjust the brightness, contrast, and saturation of each image.

Loss Function Parameters. We set the margin of the triplet loss to 0.3 and set the dropout of each FC layer to 0.5 for avoiding overfitting.

4.3. Performance evaluation

4.3.1. Evaluation metrics

We use the cumulative matching characteristic (CMC) and mean average precision (mAP) metrics on all datasets. The CMC curve records the actual match within the topk-Ranks, while the mAP evaluates the overall performance of the method considering the precision.

Rank-k. Rank-k is the percentage of correct labels in the first k images in the search results. Rank-1 represents the probability that the first image is correct. Rank-5 represents the percentage of correct images in the first five images. Rank-10 represents the percentage of correct images in the first 10 images.

mAP. The average precision (mAP) is the multi-image matching mechanism related to the order of the query images. In short, mAP is an evaluation index calculated by recall rate and accuracy rate.

4.3.2. Ablation studies

The effect of each module on the model Performance. To verify the effect of each module on the performance, we have added each module or two modules to the model respectively and observed the variation of Rank-1 and mAP. From Table 2, it can be observed that compared to the baseline model, our model (all modules added) improved the Rank-1 and mAP by +4.7% and +7.7%, respectively. This also demonstrates that each module of the model is effective.

Table 2
Effect of each module on the model performance. NL represents the non-local module, MSG represents the multi-scale global characterization module, and WDP represents the weight-driven part feature fusion module

Rank-1 mAP

Baseline (our Resnet50) 90.5 79.2

+ NL 92.9 82.6

+ MSG 93.5 83.1

+ WDP 93.9 83.4

+ MSG + WDP 94.5 84.7

+ NL + MSG 94.0 84.9

+ NL + WDP 94.5 85.8

+ NL + MSG + WDP 95.2 86.9

	Rank-1	mAP
Baseline (our Resnet50)	90.5	79.2
+ NL	92.9	82.6
+ MSG	93.5	83.1
+ WDP	93.9	83.4
+ MSG + WDP	94.5	84.7
+ NL + MSG	94.0	84.9
+ NL + WDP	94.5	85.8
+ NL + MSG + WDP	95.2	86.9

Multi-scale global feature module. In order to verify the effect of each branch of C-P2 on the model performance, we use a combination of several branches for our experiments. The details of the comparison are shown in Table 3. From Table 3, it can be observed that compared to No MSG, our model (all branches) improved the Rank-1 and mAP by +0.7% and +2.1%, respectively. It is worth noting that the addition of each branch improves the performance of model, which can be further improved by combining two or three branches. While Table 4 demonstrates the enhanced capability of TSA, it can be observed from Table 4 that using TSA module improves Rank-1/mAP by 0.4%/1.6% compared to not using TSA module.

Table 3

Effect of each branch of C-P2 on the model performance. TSA represents tiny spatial attention module

C-P2	Rank-1	mAP
No MSG (+TSA)	94.5	84.8
+ branch1	94.7	84.9
+ branch2	94.7	85.0
+ branch3	94.6	84.9
+ branch4	94.7	85.1
+ branch1 + branch2	94.9	85.6
+ branch1 + branch2 + branch3	95.0	86.0
+ branch1 + branch2 + branch3 + branch4	95.2	86.9

Table 4

Effect of TSA on the model performance. TSA represents tiny spatial attention module

C-P2	Rank-1	mAP
No TSA	94.8	85.3
+ TSA	95.2	86.9

Weight-driven part feature fusion module. In order to verify the effect of weight-driven part feature fusion operation on the model performance, we add ablation experiments to verify the effect of weight-driven part feature operation on the model performance. It can be observed in Table 5 that using only part features improves the Rank-1 and mAP by +0.5%/+0.7% respectively compared to NO part features. This indicates that the use of the part feature can improve model performance. Rank-1 and mAP are further improved by 95.2%/86.9% after using the weight-driven part feature fusion operation module.

Table 5

Effect of weight-driven part feature fusion operation on the model performance

C-P2	Rank-1	mAP
NO part feature	94.0	84.9
+ Only part feature	94.5	85.6
+ Weight-driven part feature fusion module	95.2	86.9

Image size. To further verify the effect of image size on model performance, we added two additional sets of experiments, as shown in Table 6. From Table 6, it can be seen that with the increase of image size, the performance of the model is improved, but we also found that with the increase of image size, the training time also increases gradually. When the image size is enlarged from $288 \times 144$ to $384 \times 128$ , the training time increases nearly two times, but the improvement of Rank-1 and mAP is relatively small, so selecting the right image size will not only ensure the model accuracy but also the efficiency. It is worth mentioning that when we resize the image to $224 \times 224$ , the accuracy of the model decreases. The reason for this phenomenon is the basic properties of the pedestrian image. In short, the image length: width = 2:1 or length: width = 3:1 is more in line with the human body structure, if the image is resized into a square, the image will be stretched horizontally, which will cause damage to the features.

C-P1. We try another conv-pooling combination in the multi-scale global feature module, and we change the average pooling of branch 4 in C-P2 to max-pooling called C-P1, as in Table 6. From Table 6, we can observe that the Rank-1 and mAP of C-P2 are higher than those of C-P1 when the image sizes are the same.

Table 6

The results under different image sizes and different multi-scale global feature modules on Market-1501

Image Pixels	C-P1		C-P2

	Rank-1	mAP	Rank-1	mAP
$256 \times 128$	93.0	82.0	94.9	86.1
$288 \times 144$	93.4	83.1	95.2	86.9
$384 \times 128$	93.6	83.4	95.3	87.1
$224 \times 224$	89.5	76.3	91.3	81.0

The number of parts. Empirically, the number of parts also affects the experimental results, which is confirmed in the PCB + RPP [27] and PAR [41]. However, the optimal number of parts is not the same in different methods. For simplicity, we only construct experiments with the number of parts of 2, 4, 6, and 8 under the image size of $288 \times 144$ and Conv-pooling combination of C-P2 on the Market1501 dataset, as in Fig. 7. From Fig. 7, the best results are obtained when the number of parts is 6.

Fig. 7.

Effect of the different number of parts on model performance. It is observed that the model performance is optimal when the number of $parts = 6$ . Increasing or decreasing the number of parts will reduce the model’s performance. The reason for this phenomenon is that the too fine features of each stripe will lead to the loss of semantic information and feature redundancy.

Re-Ranking. Generally, the image feature map is obtained by CNN, subsequently, most existing methods choose L2 Euclidean distance to calculate the similarity score for Ranking or ReID tasks. To improve the accuracy of ReID, an alternative re-Ranking (RK) strategy was performed in the literature [1,33,54]. We also use L2 distance and RK to calculate the similarity scores of retrieval tasks, as shown in Table 7. From Table 7, we can find that the RK strategy can greatly improve the Rank-1 and mAP. We the use RK strategy to calculate the similarity scores in our model.

Table 7

Results are obtained by using L2 distance and RK on the Market1501 dataset

Image Pixels: $288 \times 144$	C-P2

	Rank-1	Rank-5	Rank-10	mAP
Euclidean Distance	95.2	98.3	98.9	86.9
RK	95.8	98.9	99.1	94.8

Single-query and Multi-query. Single-query uses a single image to retrieve the same pedestrian (In Tables 9, 10, and 11, those methods of using the single-query are not additionally marked). In contrast to that, multi-query retrieves the same pedestrian by using multiple images, which has higher accuracy and robustness than single-query. The multi-query method extracts multiple image features and then averages them for matching. The results of using single-query and multi-query are shown in Table 8. From Table 8, we can observe that the Multi-query strategy can greatly improve the Rank-1 and mAP, thus we use the multi-query strategy to calculate the similarity scores in our model.

Tips. Currently, only the Market1501 dataset has an additional query supplement. Thus Market1501 dataset can use a multi-query strategy, but DukeMTMC-ReID and MSMT17 datasets only use the single-query strategy.

Table 8

Results are obtained by using single-query and multi-query on the Market1501 dataset

Image Pixels: $288 \times 144$	C-P2

	Rank-1	Rank5	Rank-10	mAP
Single-query	95.2	98.3	98.9	86.9
Multi-query	96.1	99.1	99.3	90.3

4.3.3. Comparison with state of the art

In this section, we compare our model with existing models on the Market1501, DukeMTMC-ReID, and MSMT17 datasets.

Table 9
A comparison of our method with those of recent years on the Market-1501 dataset

Methods Market1501

Rank-1 Rank-5 Rank-10 mAP

BOW + kissme (ICCV2015) [48] 44.4 63.9 72.2 20.8

LOMO + XQDA (CVPR2015) [17] 43.8 – 80.5 22.2

WARCA (ECCV2016) [15] 45.2 68.1 76.0 –

SVDNet (ICCV2017) [26] 82.3 92.3 95.2 62.1

PAR (ICCV2017) [41] 81.0 92.0 94.7 63.4

PCB + RPP (ECCV2018) [27] 93.8 97.5 98.5 81.6

AlignedReID (2018) [43] 91.0 96.3 – 79.4

AlignedReID (RK) (2018) [43] 92.8 – – 89.4

HPM (AAAI2019) [6] 94.2 97.5 98.5 82.7

DHA-Net (IEEE2020) [35] 91.3 – – 76.0

CBN (ECCV2020) [55] 94.2 97.9 – 83.6

FA-Net (2021) [18] 95.0 84.6

AGW (TPAMI2021) [42] 95.1 – – 87.8

Ours 95.2 98.3 98.9 86.9

Ours (RK) 95.8 98.9 99.1 94.8

Our (Multi-Query) 96.1 99.1 99.3 90.3

Methods	Market1501
BOW + kissme (ICCV2015) [48]	44.4	63.9	72.2	20.8
LOMO + XQDA (CVPR2015) [17]	43.8	–	80.5	22.2
WARCA (ECCV2016) [15]	45.2	68.1	76.0	–
SVDNet (ICCV2017) [26]	82.3	92.3	95.2	62.1
PAR (ICCV2017) [41]	81.0	92.0	94.7	63.4
PCB + RPP (ECCV2018) [27]	93.8	97.5	98.5	81.6
AlignedReID (2018) [43]	91.0	96.3	–	79.4
AlignedReID (RK) (2018) [43]	92.8	–	–	89.4
HPM (AAAI2019) [6]	94.2	97.5	98.5	82.7
DHA-Net (IEEE2020) [35]	91.3	–	–	76.0
CBN (ECCV2020) [55]	94.2	97.9	–	83.6
FA-Net (2021) [18]	95.0			84.6
AGW (TPAMI2021) [42]	95.1	–	–	87.8
Ours	95.2	98.3	98.9	86.9
Ours (RK)	95.8	98.9	99.1	94.8
Our (Multi-Query)	96.1	99.1	99.3	90.3

On Market1501. Comparisons on Market-1501 are detailed in Table 9. The compared methods are categorized into three groups, i.e., hand-crafted methods (BOW + kissme, LOMO + XQDA, and WARCA), deep learning methods with global features (SVDNet, PAR, DHA-Net, and AGW), and deep learning methods with part features (AlignedReID, HPM, PCB + RPP, and FA-Net). We report Rank-1 = 95.2%, Rank-5 = 98.3%, Rank-10 = 98.9%, and mAP = 86.9% in Table 9. For the hand-crafted method, our method surpasses all the prior methods [15,17,48]. We also achieve a more competitive performance than the deep learning method with global features and part features. Although our mAP is not the highest compared to AGW. However, our method achieve the highest results on Rank-1, Rank5, and Rank10 (without RK). With the help of RK, Rank-1 further improved to 96.1%, and mAP further improved to 94.8%.

With the help of multi-query, Rank-1, Rank-5, and Rank-10 are further, improved to 96.1%, 99.1%, and 99.3%, and the mAP has also reached 90.3%. By using RK and the multi-query, our Rank-k and mAP are greatly improved.

On DukeMTMC-ReID. Comparisons on DukeMTMC-ReID are shown in Table 10. We report Rank-1 = 88.9%, and mAP = 78.8% (without RK) in Table 10. In the compared methods, PCB + RPP is a very robust part-based method, and many methods use it as a comparison method, our method exceeds PCB + RPP by +5.6%/+9.6% in Rank-1/mAP. With the help of RK, our method further surpasses it by a large margin of +9.6%/+20.1% in Rank-1/mAP on DukeMTMC-ReID. Our method also yields higher Rank-1 and mAP than SNR [14] which is a global feature based method.

On MSMT17. Comparisons on MSMT17 is shown in Table 11. We report Rank-1/mAP = 76.7%/56.5% (without RK) in Table 11. In the compared methods, our method exceeds PCB + RPP +8.5%/+16.1% in Rank-1/mAP, with the help of RK, our method further surpasses it by a large margin of +9.8%/+32.7% in Rank-1/mAP. Our method also yields a higher Rank-1/mAP than DI-REID [13] which is a global and part feature based method.

Table 10

A comparison of our method with those of recent years on the DukeMTMC-ReID dataset

Methods	DukeMTMC-ReID

	Rank-1	mAP
LOMO + XQDA (CVPR2015) [17]	30.8	17.0
SVDNet + Era (2017) [26]	79.3	62.4
PCB + RPP (ECCV2018) [27]	83.3	69.2
CAMA (CVPR2019) [40]	85.8	72.9
IAN (CVPR2019) [11]	87.1	73.4
AANet-50 (CVPR2019) [28]	86.4	72.6
CASN (CVPR2019) [51]	87.7	73.7
SNR (CVPR2020) [14]	85.9	73.0
FA-Net (2021) [18]	88.7	77.0
AGW (TPAMI2021) [42]	89.0	79.6
Ours	88.9	78.8
Ours (RK)	92.9	89.3

Table 11

A comparison of our method with those of recent years on the MTMC17 dataset

Methods	MSMT17

	Rank-1	mAP
PCB + RPP (ECCV2018) [27]	68.2	40.4
Auto-ReID (ICCV2019) [23]	78.2	52.5
IAN (CVPR2019) [11]	75.5	46.8
CRAN (TCSVT2019) [9]	78.7	52.4
DI-REID (CVPR2020) [13]	75.5	47.1
FA-Net (2021) [18]	76.8	51.0
AGW (TPAMI2021) [42]	68.3	49.3
Ours	76.7	56.5
Ours (RK)	78.0	73.1

4.4. Qualitative analysis

PCB [27] is a simple and strong part-based method that divides the body into horizontal stripes of the same size. Many existing methods use PCB models as the comparison method. To intuitively compare the PCB model and our model, we visualize activation maps and search results respectively, as shown in Figs 8, 9.

4.4.1. Visualization of the activation maps on Market-1501

Fig. 8.

Activation maps of pedestrians for different stages. The region covered by the color represents the scope of the model’s attention, and the shade of the color represents the intensity of the model’s attention.

In order to visualize the working mechanism of our model, we instantiated two sets of pedestrian images and compared the activation maps of stage1-stage3 of the PCB model and our model(The shade of color represents the degree of focus of the model), as shown in Fig. 8. As can be observed in Fig. 8, the first row is the visualization result of the PCB model and the second row is the visualization result of our model. For the Image1, the PCB model focuses on the book held in the pedestrian’s hand and covers a smaller region of attention, while our model tends to focus on diverse and comprehensive pedestrian features. For the Image2, factors such as background have more influence on the PCB model, while our model is rarely disturbed by the background, and is more robust to changes in factors such as the color and texture of the pedestrian’s upper and lower body clothes, and have greater capability for salient feature extraction(e.g., the clothing logo of the pedestrian in Image 2).

4.4.2. Visualization of search results on Market-1501

HPM [6] is a model based on part features proposed in recent years, which describes pedestrians by dividing parts more finely. We reproduce this model for comparison.

Fig. 9.

Retrieval results for our model. Green numbers indicate correct identifications, and red numbers indicate incorrect identifications.

In order to show the retrieval results of our models more intuitively. We reproduction three models to query the same pedestrian image and visualized the retrieval results, which are shown in Fig. 9. From Fig. 9, we can observe that the query image and the gallery image of the same pedestrian contain many factors of variation, such as lighting conditions, posture, viewpoint, and resolution, etc. The error rate of the Resnet50 model is high because it only uses global features as the description of pedestrians, and the global features are not robust to changes in pedestrian details. These details include the texture of the pedestrian clothes, logos and color shades of clothes, etc.

The accuracy of the part features based models (PCB and HPM) is higher than the global features based Resnet50. This also shows that local features are effective for ReID tasks, and our model uses enhanced global and local features, which greatly improves the accuracy of the search. This also shows that our improvements are effective.

5. Conclusion

Our method aims to enhance the robustness of the ReID model to various external factor changes, and we propose a ReID model based on a multi-scale global feature module and a weight-driven partial feature fusion module. For global features, we use multiple convolution kernels and a novel spatial attention module to extract refined global features. For part features, we divide different body parts and propose a novel adaptive weight method to weight adjacent part features to construct enhanced pedestrian part features. In addition, we introduce the Non-local module into the backbone network to enhance its feature enhancement capability, and then use two secondary modules as feature complements. Comparative experiments show that our method has high accuracy compared to other similar methods. Ablation experiments also clearly demonstrate the effectiveness of each component of our model.

Footnotes

Acknowledgements

This work was supported by the Provincial Natural Science Research Projects in Anhui Universities-Postgraduate Projects(YJS20210508), the University Synergy Innovation Program of Anhui Province under Grant GXXT-2019-007, by the National Natural Science Foundation of China under Grant 61901006, Grant 62105002, and Grant62001004, by the Anhui Provincial Natural Science Foundation under Grant 1908085QF281 and Grant2008085MF182, by the Anhui Provincial DOHURD Science Foundation under Grant 2020-YF22, and by the Educational Commission of Anhui Province of China under Grant KJ2020A0471.

References

Bai,

Bai and

Tian, Scalable person re-identification on supervised smoothed manifold, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017, pp. 2530–2539.

Bazzani,

Cristani and

Murino, Symmetry-driven accumulation of local features for human characterization and re-identification, Computer Vision and Image Understanding 117(2) (2013), 130–144. doi:10.1016/j.cviu.2012.10.008.

Deng,

Dong,

Socher,

L.-J.

Li,

Li and

Fei-Fei, Imagenet: A large-scale hierarchical image database, in: 2009 IEEE Conference on Computer Vision and Pattern Recognition, IEEE, 2009, pp. 248–255. doi:10.1109/CVPR.2009.5206848.

Fan,

Luo,

Zhang,

He,

Zhang and

Jiang, Scpnet: Spatial-channel parallelism network for joint holistic and partial person re-identification, in: Asian Conference on Computer Vision, Springer, 2018, pp. 19–34.

Farenzena,

Bazzani,

Perina,

Murino and

Cristani, Person re-identification by symmetry-driven accumulation of local features, in: 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, IEEE, 2010, pp. 2360–2367. doi:10.1109/CVPR.2010.5539926.

Fu,

Wei,

Zhou,

Shi,

Huang,

Wang,

Yao and

Huang, Horizontal pyramid matching for person re-identification, in: Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 33, 2019, pp. 8295–8302.

Gray and

Tao, Viewpoint invariant pedestrian recognition with an ensemble of localized features, in: European Conference on Computer Vision, Springer, 2008, pp. 262–275.

Hamdoun,

Moutarde,

Stanciulescu and

Steux, Person re-identification in multi-camera system by signature based on interest point descriptors collected on short video sequences, in: 2008 Second ACM/IEEE International Conference on Distributed Smart Cameras, IEEE, 2008, pp. 1–6.

Han,

Zheng,

Gao and

Sang, Complementation-reinforced attention network for person re-identification, IEEE Transactions on Circuits and Systems for Video Technology 30(10) (2019), 3433–3445. doi:10.1109/TCSVT.2019.2957467.

10.

He,

Zhang,

Ren and

Sun, Deep residual learning for image recognition, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 770–778.

11.

Hou,

Ma,

Chang,

Gu,

Shan and

Chen, Interaction-and-aggregation network for person re-identification, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, pp. 9317–9326.

12.

Hu,

Shen and

Sun, Squeeze-and-excitation networks, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. 7132–7141.

13.

Huang,

Z.-J.

Zha,

Fu,

Hong and

Li, Real-world person re-identification via degradation invariance learning, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 14084–14094.

14.

Jin,

Lan,

Zeng,

Chen and

Zhang, Style normalization and restitution for generalizable person re-identification, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 3143–3152.

15.

Jose and

Fleuret, Scalable metric learning via weighted approximate rank component analysis, in: European Conference on Computer Vision, Springer, 2016, pp. 875–890.

16.

Li,

Zhao,

Xiao and

Wang, Deepreid: Deep filter pairing neural network for person re-identification, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2014, pp. 152–159.

17.

Liao,

Hu,

Zhu and

S.Z.

Li, Person re-identification by local maximal occurrence representation and metric learning, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015, pp. 2197–2206.

18.

Liu,

Zhou,

Liu,

G.-J.

Qi,

Tian and

Li, An end-to-end foreground-aware network for person re-identification, IEEE Transactions on Image Processing 30 (2021), 2060–2071. doi:10.1109/TIP.2021.3050839.

19.

Luo,

Jiang,

Gu,

Liu,

Liao,

Lai and

Gu, A strong baseline and batch normalization neck for deep person re-identification, IEEE Transactions on Multimedia 22(10) (2019), 2597–2609. doi:10.1109/TMM.2019.2958756.

20.

Martinel,

Das,

Micheloni and

A.K.

Roy-Chowdhury, Temporal model adaptation for person re-identification, in: European Conference on Computer Vision, Springer, 2016, pp. 858–877.

21.

Matsukawa,

Okabe,

Suzuki and

Sato, Hierarchical Gaussian descriptor for person re-identification, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 1363–1372.

22.

Park and

Ham, Relation network for person re-identification, in: Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 34, 2020, pp. 11839–11847.

23.

Quan,

Dong,

Wu,

Zhu and

Yang, Auto-reid: Searching for a part-aware convnet for person re-identification, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, 2019, pp. 3750–3759.

24.

Su,

Li,

Zhang,

Xing,

Gao and

Tian, Pose-driven deep convolutional model for person re-identification, in: Proceedings of the IEEE International Conference on Computer Vision, 2017, pp. 3960–3969.

25.

Suh,

Wang,

Tang,

Mei and

K.M.

Lee, Part-aligned bilinear representations for person re-identification, in: Proceedings of the European Conference on Computer Vision (ECCV), 2018, pp. 402–419.

26.

Sun,

Zheng,

Deng and

Wang, Svdnet for pedestrian retrieval, in: Proceedings of the IEEE International Conference on Computer Vision, 2017, pp. 3800–3808.

27.

Sun,

Zheng,

Yang,

Tian and

Wang, Beyond part models: Person retrieval with refined part pooling (and a strong convolutional baseline), in: Proceedings of the European Conference on Computer Vision (ECCV), 2018, pp. 480–496.

28.

C.-P.

Tay,

Roy and

K.-H.

Yap, Aanet: Attribute attention network for person re-identifications, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, pp. 7134–7143.

29.

R.R.

Varior,

Haloi and

Wang, Gated Siamese convolutional neural network architecture for human re-identification, in: European Conference on Computer Vision, Springer, 2016, pp. 791–808.

30.

Wang,

Jiang,

Chen,

Yang and

Tang, Residual attention network for image classification, in: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017.

31.

Wang,

Zuo,

Lin,

Zhang and

Zhang, Joint learning of single-image and cross-image representations for person re-identification, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 1288–1296.

32.

Wang,

Yuan,

Chen,

Li and

Zhou, Learning discriminative features with multiple granularities for person re-identification, in: Proceedings of the 26th ACM International Conference on Multimedia, 2018, pp. 274–282. doi:10.1145/3240508.3240552.

33.

Wang,

Zhou,

Wang and

Hou, Deep ranking model by large adaptive margin learning for person re-identification, Pattern Recognition 74 (2018), 241–252. doi:10.1016/j.patcog.2017.09.024.

34.

Wang,

Girshick,

Gupta and

He, Non-local neural networks, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. 7794–7803.

35.

Wang,

Jiang,

Wu,

Ye,

Bai and

Satoh, Learning sparse and identity-preserved hidden attributes for person re-identification, IEEE Transactions on Image Processing 29 (2019), 2013–2025. doi:10.1109/TIP.2019.2946975.

36.

Wei,

Zhang,

Gao and

Tian, Person transfer gan to bridge domain gap for person re-identification, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. 79–88.

37.

Wei,

Zhang,

Yao,

Gao and

Tian, Glad: Global-local-alignment descriptor for pedestrian retrieval, in: Proceedings of the 25th ACM International Conference on Multimedia, 2017, pp. 420–428.

38.

Woo,

Park,

J.-Y.

Lee and

I.S.

Kweon, Cbam: Convolutional block attention module, in: Proceedings of the European Conference on Computer Vision (ECCV), 2018, pp. 3–19.

39.

B.N.

Xia,

Gong,

Zhang and

Poellabauer, Second-order non-local attention networks for person re-identification, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, 2019, pp. 3760–3769.

40.

Yang,

Huang,

Zhang,

Chen,

Huang and

Zhang, Towards rich feature discovery with class activation maps augmentation for person re-identification, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, pp. 1389–1398.

41.

Yao,

Zhang,

Hong,

Zhang,

Xu and

Tian, Deep representation learning with part loss for person re-identification, IEEE Transactions on Image Processing 28(6) (2019), 2860–2871. doi:10.1109/TIP.2019.2891888.

42.

Ye,

Shen,

Lin,

Xiang,

Shao and

S.C.

Hoi, Deep learning for person re-identification: A survey and outlook, IEEE Transactions on Pattern Analysis and Machine Intelligence 44(6) (2021), 2872–2893. doi:10.1109/TPAMI.2021.3054775.

43.

Zhang,

Luo,

Fan,

Xiang,

Sun,

Xiao,

Jiang,

Zhang and

Sun, Alignedreid: Surpassing human-level performance in person re-identification, 2017, arXiv preprint arXiv:1711.08184.

44.

Zhang,

Si and

Liu, Integration convolutional neural network for person re-identification in camera networks, IEEE Access 6 (2018), 36887–36896.

45.

Zhao,

Li,

Zhuang and

Wang, Deeply-learned part-aligned representations for person re-identification, in: Proceedings of the IEEE International Conference on Computer Vision, 2017, pp. 3219–3228.

46.

Zheng,

Deng,

Sun,

Jiang,

Guo,

Yu,

Huang and

Ji, Pyramidal person re-identification via multi-loss dynamic training, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, pp. 8514–8522.

47.

Zheng,

Bie,

Sun,

Wang,

Su,

Wang and

Tian, Mars: A video benchmark for large-scale person re-identification, in: European Conference on Computer Vision, Springer, 2016, pp. 868–884.

48.

Zheng,

Shen,

Tian,

Wang,

Wang and

Tian, Scalable person re-identification: A benchmark, in: Proceedings of the IEEE International Conference on Computer Vision, 2015, pp. 1116–1124.

49.

Zheng,

Yang and

A.G.

Hauptmann, 2016, Person re-identification: Past, present and future.

50.

Zheng,

Zhang,

Sun,

Chandraker,

Yang and

Tian, Person re-identification in the wild, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017, pp. 1367–1376.

51.

Zheng,

Karanam,

Wu and

R.J.

Radke, Re-identification with consistent attentive Siamese networks, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, pp. 5735–5744.

52.

W.-S.

Zheng,

Gong and

Xiang, Reidentification by relative distance comparison, IEEE Transactions on Pattern Analysis and Machine Intelligence 35(3) (2012), 653–668. doi:10.1109/TPAMI.2012.138.

53.

Zheng,

Zheng and

Yang, Unlabeled samples generated by gan improve the person re-identification baseline in vitro, in: Proceedings of the IEEE International Conference on Computer Vision, 2017, pp. 3754–3762.

54.

Zhong,

Zheng,

Cao and

Li, Re-ranking person re-identification with k-reciprocal encoding, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017, pp. 1318–1327.

55.

Zhuang,

Wei,

Xie,

Zhang,

Wu,

Ai and

Tian, Rethinking the distribution gap of person re-identification with camera-based batch normalization, in: European Conference on Computer Vision, Springer, 2020, pp. 140–157.

Person re-identification based on multi-scale global feature and weight-driven part feature

Abstract

Keywords

1. Introduction

3. Method

3.5. Loss computation

3.6. Model test

4. Experiments

4.1. Datasets

Table 1 Statistics of the datasets used for training and testing Dataset Train IDs Train Images Test IDs Query Images Cameras Total Images Market-1501 751 12,936 750 3368 6 32,217 DukeMTMC-reID 702 16,522 702 2228 8 36,441 MSMT17 1041 32,621 3060 11,659 15 126,441

4.3. Performance evaluation

4.3.1. Evaluation metrics

4.3.2. Ablation studies

4.4.1. Visualization of the activation maps on Market-1501

Footnotes

Acknowledgements

References

Table 1
Statistics of the datasets used for training and testing

Dataset Train IDs Train Images Test IDs Query Images Cameras Total Images

Market-1501 751 12,936 750 3368 6 32,217

DukeMTMC-reID 702 16,522 702 2228 8 36,441

MSMT17 1041 32,621 3060 11,659 15 126,441