±à¼ÍƼö: |
±¾ÎÄÖ÷Òª½éÉÜÁË×Ô¶¯¼ÝÊ»ÖеÄ3DÄ¿±ê¼ì²â¸ÅÄîÐÔ¡¢ÎÊÌâºÍÄѵ㼰Ö÷Òª·½·¨µÈ, Ï£Íû¶ÔÄúµÄѧϰÓÐËù°ïÖú¡£
±¾ÎÄÀ´×ÔÓÚ³ÌÐòÔ±´ó±¾Óª£¬ÓÉ»ðÁú¹ûÈí¼þLinda±à¼¡¢ÍƼö¡£ |
|
¿ªÆª
2DÄ¿±ê¼ì²âÔÚ×Ô¶¯¼ÝÊ»ÁìÓò´æÔںܶàÎÊÌ⣬ÒòΪ×Ô¶¯¼ÝÊ»µÄ¿Õ¼äÊ×ÏÈÊÇÔÚ3D²ãÃæÉϵ쬶øÇÒÐèҪʹÓÃRGBͼÏñ¡¢RGB-DÉî¶ÈͼÏñºÍ¼¤¹âµãÔÆ£¬Êä³öÎïÌåÀà±ð¼°ÔÚÈýά¿Õ¼äÖеij¤¿í¸ß¡¢Ðýת½ÇµÈÐÅÏ¢¡£ÕâÒ»Àà¼ì²â³ÆÎª3DÄ¿±ê¼ì²â¡£
3DÄ¿±ê¼ì²â
Ëæ×ÅFaster-RCNNµÄ³öÏÖ£¬2DÄ¿±ê¼ì²â´ïµ½ÁË¿ÕǰµÄ·±ÈÙ£¬¸÷ÖÖÐµķ½·¨²»¶ÏÓ¿ÏÖ£¬°Ù¼ÒÕùÃù£¬µ«ÊÇÔÚÎÞÈ˼ÝÊ»¡¢»úÆ÷ÈË¡¢ÔöÇ¿ÏÖʵµÄÓ¦Óó¡¾°Ï£¬ÆÕͨ2D¼ì²â²¢²»ÄÜÌṩ¸ÐÖª»·¾³ËùÐèÒªµÄÈ«²¿ÐÅÏ¢£¬2D¼ì²â½öÄÜÌṩĿ±êÎïÌåÔÚ¶þάͼƬÖеÄλÖúͶÔÓ¦Àà±ðµÄÖÃÐŶȣ¬µ«ÊÇÔÚÕæÊµµÄÈýάÊÀ½çÖУ¬ÎïÌå¶¼ÊÇÓÐÈýάÐÎ×´µÄ£¬´ó²¿·ÖÓ¦Óö¼ÐèÒªÓÐÄ¿±êÎïÌåµÄ³¤¿í¸ß»¹ÓÐÆ«×ª½ÇµÈÐÅÏ¢¡£ÀýÈçÏÂͼFig.1ÖУ¬ÔÚ×Ô¶¯¼ÝÊ»³¡¾°Ï£¬ÐèÒª´ÓͼÏñÖÐÌṩĿ±êÎïÌåÈýά´óС¼°Ðýת½Ç¶ÈµÈÖ¸±ê£¬ÔÚÄñî«Í¶Ó°µÄÐÅÏ¢¶ÔÓÚºóÐø×Ô¶¯¼ÝÊ»³¡¾°ÖеÄ·¾¶¹æ»®ºÍ¿ØÖƾßÓÐÖÁ¹ØÖØÒªµÄ×÷Óá£
Ŀǰ3DÄ¿±ê¼ì²âÕý´¦ÓÚ¸ßËÙ·¢Õ¹Ê±ÆÚ£¬Ä¿Ç°Ö÷ÒªÊÇ×ÛºÏÀûÓõ¥Ä¿Ïà»ú¡¢Ë«Ä¿Ïà»ú¡¢¶àÏß¼¤¹âÀ×´ïÀ´½øÐÐ3DÄ¿±ê¼ì²â£¬´ÓĿǰ³É±¾ÉϽ²£¬¼¤¹âÀ×´ï>˫ĿÏà»ú>µ¥Ä¿Ïà»ú£¬´ÓĿǰµÄ׼ȷÂÊÉϽ²£¬¼¤¹âÀ×´ï>˫ĿÏà»ú>µ¥Ä¿Ïà»ú¡£µ«ÊÇËæ×ż¤¹âÀ×´ïµÄ²»¶Ï²úÒµ»¯·¢Õ¹£¬³É±¾ÔÚ²»¶Ï½µµÍ£¬Ä¿Ç°Ò²³öÏÖһЩʹÓõ¥Ä¿Ïà»ú¼ÓÏßÊý½ÏÉٵ魯âÀ×´ï½øÐÐ×ÛºÏʹÓõļ¼Êõ·½°¸¡£½øÐÐ3DÄ¿±ê¼ì²â´ÓʹÓõÄÊý¾Ý½øÐл®·ÖÖ÷Òª¿ÉÒÔ·ÖΪÒÔϼ¸Àࣺ
1.1¼¤¹â
Voxelnet£¬Voxelnet°Ñ¼¤¹âµãÔÆÔÚ¿Õ¼äÖоùÔÈ»®·ÖΪ²»Í¬µÄvoxel£¬ÔٰѲ»Í¬voxelÖеĵãÔÆÍ¨¹ýÌá³öµÄVFE£¨Voxel
Feature Encoding£©²ãת»»ÎªÒ»¸öͳһµÄÌØÕ÷±í´ï£¬×îºóʹÓÃRPN£¨Region Proposal
Network£©¶ÔÎïÌå½øÐзÖÀàºÍλÖûع飬ÕûÌåÁ÷³ÌÈçͼËùʾ¡£
1.2µ¥Ä¿Ïà»ú
ÒÔ¿ªÔ´µÄApolloΪÀý£¬ApolloÖÐʹÓõÄYOLO 3D£¬ÔÚApolloÖÐͨ¹ýÒ»¸ö¶àÈÎÎñÍøÂçÀ´½øÐгµµÀÏߺͳ¡¾°ÖÐÄ¿±êÎïÌå¼ì²â¡£ÆäÖеÄEncoderÄ£¿éÊÇYoloµÄDarknet£¬ÔÚÔʼDarknet»ù´¡ÉϼÓÈëÁ˸üÉîµÄ¾í»ý²ãͬʱÌí¼Ó·´¾í»ý²ã£¬²¶×½¸ü·á¸»µÄͼÏñÉÏÏÂÎÄÐÅÏ¢¡£¸ß·Ö±æ¶àͨµÀÌØÕ÷ͼ£¬²¶×½Í¼Ïñϸ½Ú£¬Éî²ãµÍ·Ö±æÂʶàͨµÀÌØÕ÷ͼ£¬±àÂë¸ü¶àͼÏñÉÏÏÂÎÄÐÅÏ¢¡£ºÍFPN£¨Feature
Paramid Network£©ÀàËÆµÄ·ÉÏßÁ¬½Ó£¬¸üºÃµÄÈÚºÏÁËͼÏñµÄϸ½ÚºÍÕûÌåÐÅÏ¢¡£Decoder·ÖΪÁ½¸ö²¿·Ö£¬Ò»²¿·ÖÊÇÓïÒå·Ö¸î£¬ÓÃÓÚ³µµÀÏß¼ì²â£¬ÁíÒ»²¿·ÖΪÎïÌå¼ì²â£¬ÎïÌå¼ì²â²¿·Ö»ùÓÚYOLO£¬Í¬Ê±»¹»áÊä³öÎïÌåµÄ·½ÏòµÈ3DÐÅÏ¢¡£
ͨ¹ýÉñ¾ÍøÂçÔ¤²â3DÕϰÎïµÄ9ά²ÎÊýÄѶȽϴó£¬ÀûÓõØÃæÆ½ÐмÙÉ裬À´½µµÍËùÐèÒªÔ¤²âµÄ3D²ÎÊý¡£1)¼ÙÉè3DÕϰÎïÖ»ÑØ×Å´¹Ö±µØÃæµÄ×ø±êÖáÓÐÐýת£¬¶øÁíÍâÁ½¸ö·½Ïò²¢Î´³öÏÖÐýת£¬Ò²¾ÍÊÇÖ»ÓÐyawÆ«ÒÆ½Ç£¬Ê£ÏµÄPitchºÍRoll¾ùΪ0¡£ÕϰÎïÖÐÐĸ߶ȺÍÏà»ú¸ß¶ÈÏ൱£¬ËùÒÔ¿ÉÒÔ¼ò»¯ÈÏΪÕϰÎïµÄZ=0£»2)¿ÉÒÔÀûÓóÉÊìµÄ2DÕϰÎï¼ì²âËã·¨£¬×¼È·Ô¤²â³öͼÏñÉÏ2DÕϰÎï¿ò(ÒÔÏñËØÎªµ¥Î»£©£»3)¶Ô3DÕϰÎïÀïµÄ6άÃèÊö£¬¿ÉÒÔÑ¡ÔñѵÁ·Éñ¾ÍøÂçÀ´Ô¤²â·½²î½ÏСµÄ²ÎÊý¡£
ÔÚApolloÖУ¬ÊµÏÖµ¥Ä¿ÉãÏñÍ·µÄ3DÕϰÎï¼ì²âÐèÒªÁ½¸ö²¿·Ö£º
1¡¢ÑµÁ·ÍøÂ磬²¢Ô¤²â³ö´ó²¿·Ö²ÎÊý£º
ͼÏñÉÏ2DÕϰÎï¿òÔ¤²â
ÕϰÎïÎïÀí³ß´ç
²»±»ÕϰÎïÔÚͼÏñÉÏλÖÃËùÓ°Ï죬²¢ÇÒͨ¹ýͼÏñÌØÕ÷¿ÉÒԺܺýâÊ͵ÄÕϰÎïyawƫת½Ç
2¡¢Í¨¹ýͼÏñ¼¸ºÎѧ¼ÆËã³öÕϰÎïÖÐÐĵãÏà¶ÔÏà»ú×ø±êϵµÄÆ«ÒÆÁ¿X·ÖÁ¿ºÍY·ÖÁ¿
1.3¼¤¹â+µ¥Ä¿Ïà»ú
AVOD£¬AVODÊäÈëRGBͼÏñ¼°BEV£¨Bird Eye View£©£¬ÀûÓÃFPNÍøÂçµÃµ½¶þÕßÈ«·Ö±æÂʵÄÌØÕ÷ͼ£¬ÔÙͨ¹ýCropºÍResizeÌáÈ¡Á½¸öÌØÕ÷ͼ¶ÔÓ¦µÄÇøÓò½øÐÐÈںϣ¬ÌôÑ¡³ö3D
proposalÀ´½øÐÐ3DÎïÌå¼ì²â£¬Õû¸öÁ÷³ÌÈçͼËùʾ¡£ ?/p>
ÎÊÌâºÍÄѵã
¾¡¹ÜĿǰ¶ÔÓÚ3DÄ¿±ê¼ì²âÒѾÓв»ÉÙµÄÑо¿£¬µ«ÊÇÔÚʵ¼ÊÓ¦ÓÃÖÐÈÎÈ»ÓÐÐí¶àµÄÎÊÌ⣬Ê×ÏÈ£¬¶ÔÎïÌåÕÚµ²¡¢½Ø¶Ï¡¢ÖÜΧ¶¯Ì¬»·¾³µÄ½¡×³ÐÔÎÊÌ⣬Æä´Î£¬ÏÖÓз½Ê½´ó¶¼ÒÀÀµÓÚÎïÌå±íÃæÎÆÀí»ò½á¹¹ÌØÕ÷£¬ÈÝÒ×Ôì³É»ìÏý£¬×îºó£¬ÔÚÂú×ã׼ȷÂÊÒªÇóµÄÌõ¼þÏ£¬Ë㷨ЧÂÊÓкܴóÎÊÌâ¡£
3D bounding boxÊÇÔÚÕæÊµÈýάÊÀ½çÖаüΧĿ±êÎïÌåµÄ×îС³¤·½Ì壬ÀíÂÛÉÏ£¬Ò»¸ö3D bounding
boxÓÐ9¸ö×ÔÓɶȣ¬3¸öÊÇλÖã¬3¸öÊÇÐýת£¬3¸öÊÇά¶È´óС¡£¶ÔÓÚ×Ô¶¯¼ÝÊ»³¡¾°ÏµÄÎïÌ壬¾ø´ó¶àÊý¶¼ÊÇˮƽ·ÅÖÃÓÚµØÃ棬ËùÒÔͨ¹ý¼ÙÉèÎïÌå¶¼·ÅÖÃÓÚˮƽµØÃ棬¿ÉÒÔÉèÖùö¶¯ºÍÇãб½Ç¶ÈÏà¶ÔÓÚË®Æ½ÃæÎªÁ㣬ͬʱµ×ÃæÊÇË®Æ½ÃæµÄÒ»²¿·Ö£¬ÕâÑù¾Í¿ÉÒÔÊ¡ÂÔµô3¸ö×ÔÓɶȣ¬»¹ÓÐ6¸ö×ÔÓɶȣ¬ËùÒÔ3DÄ¿±ê¼ì²âÒ²ÊÇÒ»¸öÄ¿±êÎïÌå6D
poseÔ¤²âÎÊÌâ¡£
Ŀǰ£¬3DÄ¿±ê¼ì²âµÄÊý¾Ý¼¯Ö÷Òª°üº¬Jura¡¢Pascal3D+¡¢LINEMOD¡¢KITTIµÈ£¬ÒÔKITTIÊý¾Ý¼¯ÎªÀý£¬ÈçͼÊÇKITTIÊý¾Ý¼¯ÖжÔÓÚÒ»¸ö3DÕϰÎïµÄ±ê×¢£º
3DÊÓ¾õÄ¿±ê¼ì²âµÄÄѵãÖ÷ÒªÔÚÓÚ£º
1£©ÕÚµ²£¬ÕÚµ²·ÖΪÁ½ÖÖÇé¿ö£¬Ä¿±êÎïÌåÏ໥ÕÚµ²ºÍÄ¿±êÎïÌå±»±³¾°ÕÚµ²
2£©½Ø¶Ï£¬²¿·ÖÎïÌ屻ͼƬ½Ø¶Ï£¬ÔÚͼƬÖÐÖ»ÄÜÏÔʾ²¿·ÖÎïÌå
3£©Ð¡Ä¿±ê£¬Ïà¶ÔÊäÈëͼƬ´óС£¬Ä¿±êÎïÌåËùÕ¼ÏñËØµã¼«ÉÙ
4£©Ðýת½Ç¶Èѧϰ£¬ÎïÌåµÄ³¯Ïò²»Í¬£¬µ«ÊǶÔÓ¦ÌØÕ÷Ïàͬ£¬Ðýת½ÇµÄÓÐЧѧϰÓнϴóÄѶȣ¬ÈçͼËùʾ ?/p>
5£©È±Ê§Éî¶ÈÐÅÏ¢£¬2DͼƬÏà¶ÔÓÚ¼¤¹âÊý¾Ý´æÔÚÐÅÏ¢³íÃÜ¡¢³É±¾µÍµÄÓÅÊÆ£¬µ«ÊÇÒ²´æÔÚȱʧÉî¶ÈÐÅÏ¢µÄȱµã
Ö÷Òª·½·¨
Ŀǰ»ùÓÚµ¥Ä¿Ïà»úµÄ3DÄ¿±ê¼ì²âµÄ·½·¨Ö÷ÒªÊǸ´ÓÃ2D¼ì²âÖеÄһϵÁз½·¨£¬Í¬Ê±¼ÓÈë¶à×ø±êµãµÄ»Ø¹é¡¢Ðýת½ÇµÄ»Ø¹é»ò·ÖÀ࣬ͬʱҲÓвÉÓÃ×Ô±àÂëÆ÷µÄ·½·¨À´½øÐÐ×Ë̬ѧϰ¡£
3.1 SSD-6D
¹Ø¼üµã£º
Viewpoint classification VS pose regression:
×÷ÕßÈÏΪ¾¡¹ÜÒÑÓÐÂÛÎÄÖ±½ÓʹÓýǶȻع飬µ«ÊÇÓÐʵÑéÖ¤Ã÷¶ÔÓÚÐýת½ÇµÄ¼ì²â£¬Ê¹Ó÷ÖÀàµÄ·½Ê½±ÈÖ±½ÓʹÓûعé¸ü¼Ó¿É¿¿£¬ÌرðÊÇʹÓÃÀëÉ¢»¯µÄviewpoints±ÈÍøÂçÖ±½ÓÊä³ö¾«È·ÊýֵЧ¹û¸üºÃ
Dealing with symmetry and view ambiguity:
¸ø¶¨Ò»¸öµÈ¾à²ÉÑùµÄÇòÌ壬¶ÔÓڶԳƵÄÄ¿±êÎïÌ壬½öÑØ×ÅÒ»Ìõ»¡Ïß²ÉÑùÊÓͼ£¬¶ÔÓÚ°ë¶Ô³ÆÎïÌ壬ÔòÍêȫʡÂÔÁíÒ»¸ö°ëÇò£¬ÈçͼËùʾ
Ч¹û£º
3.2 3D Bounding Box Estimation Using Deep Learning
and Geometry
×÷ÕßÌá³öÒ»ÖÖ´Óµ¥Ö¡Í¼ÏñÖнøÐÐ3DÄ¿±ê¼ì²âºÍ×Ë̬¹À¼ÆµÄ·½·¨£¬¸Ã·½·¨Ê×ÏÈʹÓÃÉî¶ÈÉñ¾ÍøÂç»Ø¹é³öÏà¶ÔÎȶ¨µÄ3DÄ¿±êµÄÌØÐÔ£¬ÔÙÀûÓùÀ¼Æ³öÀ´µÄ3DÌØÕ÷ºÍÓÉ2D
bounding boxת»»Îª3D bounding boxʱµÄ¼¸ºÎÔ¼ÊøÀ´²úÉú×îÖյĽá¹û¡£ÂÛÎÄÖУ¬×÷ÕßÌá³öÁËÒ»¸öÑϸñµÄ¼ÙÉ裬¼´Ò»¸ö3D
bounding boxÓ¦¸ÃÑϸñµØ±»2D bounding boxËù°üΧ£¬Ò»¸ö3D bounding
boxÓÉÖÐÐĵãµÄ£¨x, y, z£©×ø±ê¡¢ºÍÈýά³ß¶È£¨w, h, l£©ºÍÈý¸öÐýת½ÇËù±íʾ¡£Òª¹À¼ÆÈ«¾ÖµÄÎïÌå×Ë̬½ö½öͨ¹ý¼ì²âµ½µÄ2D
bounding boxÊDz»¿ÉÄܵģ¬ÈçÏÂͼËùʾ£¬¾¡¹ÜÆû³µµÄÈ«¾Ö×Ë̬һֱûÓб䣬µ«ÊÇÔÚ2D bounding
boxÖеÄ×Ë̬һֱÔڱ䡣Òò´Ë£¬×÷ÕßÑ¡Óûعé2D bounding boxÖеÄ×Ë̬ÔÙ¼ÓÉÏÔÚÏà»ú×ø±êϵÖÐÆû³µ½Ç¶ÈµÄ±ä»¯µÄ×ÛºÏÀ´½øÐÐÆû³µÈ«¾Ö×Ë̬µÄ¹À¼Æ¡£
ͬʱ£¬×÷Õß»¹Ìá³öÁËMultiBinµÄ½á¹¹À´½øÐÐ×Ë̬µÄ¹À¼Æ£¬Ê×ÏÈÀëÉ¢»¯Ðýת½Çµ½N¸öÖØµþµÄBin£¬¶ÔÓÚÿһ¸öBin£¬CNNÍøÂç¹À¼Æ³ö×Ë̬½Ç¶ÈÔÚµ±Ç°BinµÄ¸ÅÂÊ£¬Í¬Ê±¹À¼Æ³ö½Ç¶ÈÖµµÄCosºÍSinÖµ¡£ÍøÂçÕûÌå½á¹¹ÈçÏÂͼËùʾ£¬ÔÚ¹«¹²µÄÌØÕ÷ͼºóÍøÂçÓÐÈý¸ö·ÖÖ§£¬·Ö±ð¹À¼Æ3DÎïÌåµÄ³¤¿í¸ß¡¢Ã¿¸öBinµÄÖÃÐŶȺÍÿ¸öBinµÄ½Ç¶È¹À¼Æ¡£
3.3 Implicit 3D Orientation Learning for 6D Object
Detection from RGB Images
×÷ÕßÖ÷ÒªÊÇÌá³öÁËÒ»ÖÖÐÂÐ͵ĻùÓÚÈ¥Ôë×Ô±àÂëÆ÷DA£¨Denoising Autoencoder£©µÄ3DÄ¿±ê³¯Ïò¹À¼Æ·½·¨£¬Ê¹ÓÃÁËÓòËæ»ú»¯£¨Domain
Randomization£©ÔÚ3DÄ£Ð͵ÄÄ£ÄâÊÓͼÉϽøÐÐѵÁ·¡£ÔÚ½øÐмì²âʱ£¬Ê×ÏÈʹÓÃSSD£¨Single
Shot Multibox Detector£©À´½øÐÐ2DÎïÌå±ß½ç¿òµÄ»Ø¹éºÍ·ÖÀ࣬ȻºóʹÓÃÔ¤ÏÈѵÁ·µÄÉî¶ÈÍøÂç3DÄ¿±ê³¯Ïò¹À¼ÆËã·¨¶ÔÎïÌåµÄ³¯Ïò½øÐйÀ¼Æ¡£ÔÚÄ£Ð͵ÄѵÁ·ÆÚ¼ä£¬Ã»ÓÐÏÔʾµØ´Ó3D×Ë̬±ê×¢Êý¾ÝÖÐѧϰÎïÌåµÄ6D
pose£¬¶øÊÇͨ¹ýʹÓÃÓòËæ»ú»¯ÑµÁ·Ò»¸öAAE£¨Augmented Autoencoder£©´ÓÉú³ÉµÄ3DÄ£ÐÍÊÓͼÖÐѧϰÎïÌå6D
poseµÄÌØÕ÷±íʾ¡£
ÕâÖÖ´¦Àí·½Ê½ÓÐÒÔϼ¸¸öÓÅÊÆ£º
¿ÉÒÔÓÐЧ´¦ÀíÓÐÆçÒåµÄÎïÌå×Ë̬£¬ÓÈÆäÊÇÔÚÎïÌå×Ë̬¶Ô³ÆÊ±
ÓÐЧѧϰÔÚ²»Í¬»·¾³±³¾°¡¢ÕÚµ²Ìõ¼þϵÄÎïÌå3D×Ë̬±íʾ
AAE²»ÐèÒªÕæÊµµÄ×Ë̬±êעѵÁ·Êý¾Ý
Ч¹û£º
˼¿¼×ܽá
¡¤ÈçǰËùÊö£¬´¿ÊÓ¾õµ¥Ä¿3DÄ¿±ê¼ì²âÔÚ׼ȷÂÊÉÏÀëÔ¤ÆÚ»¹Óнϴó²î¾à£¬¿ÉÒÔ¿¼ÂÇÒýÈë²ÉÓÃÉî¶ÈÉñ¾ÍøÂç½áºÏÏ¡Ê輤¹âµãÔÆÉú³É³íÃܵãÔÆ¶Ô¼ì²â½á¹û½øÐÐÐÞÕý
¡¤Ä¿Ç°´ó¶àÊDzÉÓÃOne-StageµÄ·½·¨½øÐÐ3DÄ¿±êµÄ×Ë̬»Ø¹é£¬¿ÉÒÔ¿¼ÂÇʹÓÃTwo-StageµÄ·½·¨À´£¬²¢ÀûÓ÷ָîµÄMaskÐÅÏ¢
¡¤Ä¿Ç°3DÄ¿±ê¼ì²âµÄ±ê×¢Êý¾Ý½ÏÉÙ£¬¿ÉÒÔ¿¼ÂÇÒýÈë·Ç¼à¶½Ñ§Ï°
¡¤Ê¹Óøü¶àµÄ¼¸ºÎÔ¼Êø |