This article presents a challenging multi-agent seasonal dataset collected by a fleet of Ford autonomous vehicles (AVs) at different days and times during 2017–2018. The vehicles traversed an average route of 66 km in Michigan that included a mix of driving scenarios such as the Detroit airport, freeways, city centers, university campus, and suburban neighborhoods. Each vehicle used in this data collection was a Ford Fusion outfitted with an Applanix POS-LV GNSS/INS system, four HDL-32E Velodyne 3D-lidar scanners, six Point Grey 1.3 MP cameras arranged on the rooftop for 360° coverage, and one Point Grey 5 MP camera mounted behind the windshield for the forward field of view. We present the seasonal variation in weather, lighting, construction, and traffic conditions experienced in dynamic urban environments. We also include data from multiple AVs that were driven in close proximity. This dataset can help design robust algorithms for AVs and multi-agent systems. Each log in the dataset is time-stamped and contains raw data from all the sensors, calibration values, pose trajectory, ground-truth pose, and 3D maps. All data is available in rosbag format that can be visualized, modified, and applied using the open-source Robot Operating System (ROS). We also provide the output of reflectivity-based localization for bench-marking purposes. The dataset can be freely downloaded at avdata.ford.com.
BroggiAGrisleriPZaniP (2013) Sensors technologies for intelligent vehicles perception systems: A comparison between vision and 3D-lidar. In: 16th International IEEE Conference on Intelligent Transportation Systems (ITSC 2013), pp. 887–892.
2.
CaesarHBankitiVLangAH, et al. (2019) Nuscenes: A multimodal dataset for autonomous driving. arXiv preprint arXiv:1903.11027.
3.
CampbellSO’MahonyNKrpalcovaL, et al. (2018) Sensor technology in autonomous vehicles: A review. In: 2018 29th Irish Signals and Systems Conference (ISSC), pp. 1–4.
4.
CastellanosJANeiraJTardosJD (2001) Multisensor fusion for simultaneous localization and map building. IEEE Transactions on Robotics and Automation17(6): 908–914.
5.
CastorenaJAgarwalS (2017) Ground edge based lidar localization without a reflectivity calibration for autonomous driving. IEEE Robotics and Automation Letters, in press.
6.
ChangMLambertJSangkloyP, et al. (2019) Argoverse: 3D tracking and forecasting with rich maps. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 8740–8749.
7.
CordtsMOmranMRamosS, et al. (2016) The CityScapes dataset for semantic urban scene understanding. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3213–3223.
8.
DafflonBGechterFGruerPKoukamA (2013) Vehicle platoon and obstacle avoidance: A reactive agent approach. IET Intelligent Transport Systems7(3): 257–264.
9.
DavisonAJReidIDMoltonNDStasseO (2007) MonoSLAM: Real-time single camera SLAM. IEEE Transactions on Pattern Analysis and Machine Intelligence29(6): 1052–1067.
10.
DresnerKStoneP (2004) Multiagent traffic management: A reservation-based intersection control mechanism. In: Proceedings of the Third International Joint Conference on Autonomous Agents and Multiagent Systems, Vol. 2, pp. 530–537.
11.
DresnerKStoneP (2005) Multiagent traffic management: An improved intersection control mechanism. In: Proceedings of the Fourth International Joint Conference on Autonomous Agents and Multiagent Systems, pp. 471–477.
12.
DresnerKStoneP (2008) A multiagent approach to autonomous intersection management. Journal of Artificial Intelligence Research31: 591–656.
13.
Durrant-WhyteHBaileyT (2006a) Simultaneous localization and mapping: Part I. IEEE Robotics and Automation Magazine13(2): 99–110.
14.
Durrant-WhyteHBaileyT (2006b) Simultaneous localization and mapping: Part II. IEEE Robotics and Automation Magazine13(3): 108–117.
15.
GeigerALenzPStillerCUrtasunR (2013) Vision meets robotics: The KITTI dataset. International Journal of Robotics Research32(11): 1231–1237.
16.
GeigerALenzPUrtasunR (2012) Are we ready for autonomous driving? The KITTI vision benchmark suite. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition, pp. 3354–3361.
KuipersJB (2007) Quaternions and rotation sequences: A primer with applications to orbits, aerospace, and virtual reality. In: Quaternions and Rotation Sequences. Princeton, NJ: Princeton University Press.
22.
LevinsonJMontemerloMThrunS (2007) Map-based precision vehicle localization in urban environments. In: Robotics: Science and Systems, Vol. 4.
23.
LevinsonJThrunS (2010) Robust vehicle localization in urban environments using probabilistic maps. In: IEEE International Conference on Robotics and Automation, pp. 4372–4378.
24.
LevinsonJThrunS (2014) Unsupervised calibration for multi-beam lasers. In: Experimental Robotics. Berlin: Springer, pp. 179–193.
25.
MaddernWPascoeGLinegarCNewmanP (2017) 1 year, 1,000 km: The Oxford RobotCar dataset. The International Journal of Robotics Research36(1): 3–15.
26.
MarshD (1999) Applied Geometry for Computer Graphics and CAD. Berlin: Springer-Verlag.
27.
MorasJCherfaouiVBonnifaitP (2010) A lidar perception scheme for intelligent vehicle navigation. In: 2010 11th International Conference on Control Automation Robotics & Vision. IEEE, pp. 1809–1814.
28.
NeuholdGOllmannTBulòSRKontschiederP (2017) The Mapillary Vistas dataset for semantic understanding of street scenes. In: 2017 IEEE International Conference on Computer Vision (ICCV), pp. 5000–5009.
29.
NguyenTNMichaelisBAl-HamadiATornowMMeineckeMM (2011) Stereo-camera-based urban environment perception using occupancy grid and object tracking. IEEE Transactions on Intelligent Transportation Systems13(1): 154–165.
30.
PalanisamyP (2019) Multi-agent connected autonomous driving using deep reinforcement learning. arXiv preprint arXiv: 1911.04175.
31.
PandeyGMcBrideJREusticeRM (2011) Ford Campus vision and lidar data set. The International Journal of Robotics Research30(13): 1543–1552.
32.
PandeyGMcBrideJRSavareseSEusticeRM (2012) Automatic targetless extrinsic calibration of a 3D lidar and camera by maximizing mutual information. In: Proceedings of the Twenty-Sixth AAAI Conference on Artificial Intelligence (AAAI’12). AAAI Press, pp. 2053–2059.
33.
PitropovMGarciaDRebelloJ, et al. (2020) Canadian adverse driving conditions dataset. arXiv preprint arXiv:2001.10117.
34.
QuigleyMConleyKGerkeyB, et al. (2009) ROS: An open-source robot operating system. In: ICRA Workshop on Open Source Software, Vol. 3.
35.
RedmonJDivvalaSGirshickRFarhadiA (2016) You only look once: Unified, real-time object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
36.
ReidTGHoutsSECammarataR, et al. (2019) Localization requirements for autonomous vehicles. SAE International Journal of Connected and Automated Vehicles2(3): 0012.
37.
RichardsonAStromJOlsonE (2013) Aprilcal: Assisted and repeatable camera calibration. In: 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 1814–1821.
SegalAHaehnelDThrunS (2009) Generalized-ICP. In: Proceedings of Robotics: Science and Systems, Seattle, WA.
40.
SmithRSelfMCheesemanP (1988) A stochastic map for uncertain spatial relationships. In: Proceedings of the 4th International Symposium on Robotics Research. Cambridge, MA: MIT Press, pp. 467–474.
41.
SmithRSelfMCheesemanP (1990) Estimating uncertain spatial relationships in robotics. In: Autonomous Robot Vehicles. Berlin: Springer, pp. 167–193.
42.
StudholmeCHillDHawkesD (1999) An overlap invariant entropy measure of 3D medical image alignment. Pattern Recognition32(1): 71–86.
ThrunSMontemerloMDahlkampH, et al. (2006) Stanley: The robot that won the DARPA Grand Challenge. Journal of Field Robotics23(9): 661–692.
45.
VoraAAgarwalSPandeyGMcBrideJ (2020) Aerial imagery based lidar localization for autonomous vehicles. arXiv preprint arXiv:2003.11192.
46.
VoraAGAgarwalSHoellerbauerJNShaikF (2019) High definition 3D mapping. US Patent App. 15/831,295.
47.
VuTAycardOTangoF (2014) Object perception for intelligent vehicle applications: A multi-sensor fusion approach. In: 2014 IEEE Intelligent Vehicles Symposium Proceedings, pp. 774–780.
48.
WolcottRWollherrDBussM (2017) Robust lidar localization using multiresolution Gaussian mixture maps for autonomous driving. The International Journal of Robotics Research36(3): 292–319.
49.
WolcottRWEusticeRM (2014) Visual localization within lidar maps for automated urban driving. In: 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2014). IEEE, pp. 176–183.
50.
YogamaniSHughesCHorganJ, et al. (2019) Woodscape: A multi-task, multi-camera fisheye dataset for autonomous driving. In: 2019 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 9307–9317.
