Advancements in lane detection, leveraging semantic knowledge, have enhanced the detection capabilities of intelligent vehicles in various traffic scenarios. However, existing lane detection algorithms encounter difficulties in reliably extracting lane instances and adapting to simultaneous variations in lane numbers. The proposed study delves into the critical realm of visual scene understanding with a focus on the semantic knowledge of lane classes. The complexity of multi-class lane marking classification is highlighted due to the eccentric, bland, and repeatable nature of lane markings, with classification relying on relative locations. In this study, an effort has been undertaken to introduce a dataset named SemSeg-Lanes, which is derived from the BDD Lane Detection dataset. This dataset encompasses 10 distinct lane marking classes specifically designed for semantic segmentation. Various baseline models, relying on established methods for semantic segmentation, are presented for this demanding dataset. Among these, real-time networks, including PPLiteSeg variants, were incorporated into the baselines and trained to achieve a mean average precision exceeding 70%. The dataset presented enables exploration of fundamental challenges in lane marking segmentation, paving the way for applications such as lane-centric activity understanding, future event prediction, and continual learning.
LongJShelhamerEDarrellT.Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, Boston, MA, USA, 7–12 June 2015, pp.3431–3440. New York: IEEE.
2.
PoudelRPKLiwickiSCipollaR.Fast-SCNN: fast semantic segmentation network. arXiv preprint arXiv: 1902.04502, 2019.
3.
PengJLiuYTangS, et al. PP-LiteSeg: a superior real-time semantic segmentation model. arXiv preprint arXiv: 2204.02681, 2022.
4.
LinT-YDollarPGirshickR, et al. Feature pyramid networks for object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, Honolulu, HI, USA, 21–26 July 2017, pp.2117–2125. New York: IEEE.
5.
ZhaoHShiJQiX, et al. Pyramid scene parsing network. In: Proceedings of the IEEE conference on computer vision and pattern recognition, Honolulu, HI, USA, 21–26 July 2017, pp.2881–2890. New York: IEEE.
6.
FuJLiuJTianH, et al. Dual attention network for scene segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, Long Beach, CA, USA, 15–20 June 2019, pp.3146–3154. New York: IEEE.
7.
LiXZhongZWuJ, et al. Expectation maximization attention networks for semantic segmentation. In: Proceedings of the IEEE international conference on computer vision, Seoul, Korea, 27 October–2 November 2019, pp.9167–9176. New York: IEEE.
8.
HuangZWangXHuangL, et al. CCNet: criss-cross attention for semantic segmentation. In: Proceedings of the IEEE international conference on computer vision, Seoul, Korea, 27 October–2 November 2019, pp.603–612. New York: IEEE.
9.
XieEWangWYuZ, et al. SegFormer: simple and efficient design for semantic segmentation with transformers. arXiv preprint arXiv:2105.15203, 2021.
10.
StrudelRGarciaRLaptevI, et al. Segmenter: transformer for semantic segmentation. In: Proceedings of the IEEE international conference on computer vision, Montreal, QC, Canada, 10–17 October 2021, pp.7262–7272. New York: IEEE.
11.
YuanYChenXWangJ.Object-contextual representations for semantic segmentation. arXiv preprint arXiv:1909.11065, 2019.
12.
AlyM.Real time detection of lane markers in urban streets. In: 2008 IEEE intelligent vehicles symposium, Eindhoven, Netherlands, 4–6 June 2008, pp.7–12. New York: IEEE.
13.
The TuSimple lane challenge. http://benchmark.tusim ple.ai (accessed 13 November 2020).
14.
PanXShiJLuoP, et al. Spatial as deep: spatial CNN for traffic scene understanding. In: Proceedings of the AAAI conference on artificial intelligence, New Orleans, LA, USA, 2–7 February 2018, vol. 32, no. 1.
15.
MuegglerERebecqHGallegoG, et al. The event-camera dataset and simulator: event-based data for pose estimation, visual odometry, and slam. Int J Rob Res2017; 36(2): 142–149.
16.
LiHLiuHJiX, et al. CIFAR10-DVS: an event-stream dataset for object classification. Front Neurosci2017; 11: 309.
17.
HuYLiuHPfeifferM, et al. DVS benchmark datasets for object tracking, action recognition, and object recognition. Front Neurosci2016; 10: 405.
18.
HuYBinasJNeilD, et al. DDD20 end-to-end event camera driving dataset: fusing frames and events with deep learning for improved steering prediction. In: 2020 IEEE 23rd international conference on intelligent transportation systems (ITSC), Rhodes, Greece, 20–23 September 2020, pp.1–6. New York: IEEE.
19.
NeuholdGOllmannTBuloSR, et al. The mapillary vistas dataset for semantic understanding of street scenes. In: Proceedings of the international conference on computer vision (ICCV), Venice, Italy, 22–29 October 2017, pp.22–29. New York: IEEE.
20.
BrostowGJFauqueurJCipollaR.Semantic object classes in video: a high-definition ground truth database. Pattern Recognit Lett2009; 30(2): 88–97.
21.
HuangXChengXGengQ, et al. The ApolloScape dataset for autonomous driving. In: 2018 IEEE/CVF conference on computer vision and pattern recognition workshops (CVPRW), Salt Lake City, UT, USA, 18–22 June 2018, pp.954–960. New York: IEEE.
22.
BehrendtKSoussanR.Unsupervised labeled lane markers using maps. In: 2019 IEEE/CVF international conference on computer vision workshop (ICCVW), Seoul, Korea, 27–28 October 2019, pp.832–839. New York: IEEE.
23.
BehleyJGarbadeMMiliotoA, et al. SemanticKITTI: a dataset for semantic scene understanding of LiDAR sequences. In: Proceedings of the IEEE/CVF international conference on computer vision (ICCV), Seoul, Korea, 27 October–2 November 2019, pp.9297–9307. New York: IEEE.
24.
ChengWLuoHYangW, et al. DET: a high-resolution DVS dataset for lane extraction. In: 2019 IEEE/CVF conference on computer vision and pattern recognition workshops (CVPRW), Long Beach, CA, USA, 16–17 June 2019, pp.1666–1675. New York: IEEE.
25.
ZhangYZhuLFengW, et al. VIL-100: a new dataset and a baseline model for video instance lane detection. In: 2021 IEEE/CVF international conference on computer vision (ICCV), Montreal, QC, Canada, 10–17 October 2021, pp.15661–15670. New York: IEEE.
26.
YuFChenHWangX, et al. BDD100K: a diverse driving dataset for heterogeneous multitask learning. arXiv preprint arXiv:1805.04687, 2018.
27.
LiuYChuLChenG, et al. A high-efficient development toolkit for image segmentation. arXiv preprint arXiv:2101.06175, 2021.
28.
FanMLaiSHuangJ, et al. Rethinking BiSeNet for real-time semantic segmentation. arXiv preprint arXiv:2104.13188, 2021.
29.
WangJSunKChengT, et al. Deep high-resolution representation learning for visual recognition. arXiv preprint arXiv:1908.07919, 2020.
30.
TaoASapraKCatanzaroB.Hierarchical multi-scale attention for semantic segmentation. arXiv preprint arXiv:2005.10821, 2020.
31.
HongYPanHSunW, et al. Deep dual-resolution networks for real-time and accurate semantic segmentation of road scenes. arXiv preprint arXiv:2101.06085, 2021.
