±¾ÎÄÖ÷Òª½²½âÔÚÊý¾ÝÍÚ¾òÖÐÈÝÒ×·¸´íÎóÒÔ¼°Ïà¹ØµÄ½â¾ö·½°¸£¬´ÓµÚ7µãÇáÐÅÔ¤ÑÔÖÐͦÓÐÒâ˼µÄ£¬×÷ÕßÓÃûÓÐÕýÈ·µÄ½áÂÛ£¬Ö»ÓÐÔ½À´Ô½×¼È·µÄ½áÂÛ×÷ΪÐÞÕý¡£
0. ȱ·¦Êý¾Ý£¨Lack Data£©
¶ÔÓÚ·ÖÀàÎÊÌâ»òÔ¤¹ÀÎÊÌâÀ´Ëµ£¬³£³£È±·¦×¼È·±ê×¢µÄ°¸Àý¡£
ÀýÈ磺
£ÆÛÕ©Õì²â£¨Fraud Detection£©£ºÔÚÉϰÙÍòµÄ½»Ò×ÖУ¬¿ÉÄÜÖ»ÓÐÇüÖ¸¿ÉÊýµÄÆÛÕ©½»Ò×£¬»¹ÓкܶàµÄÆÛÕ©½»Ò×ûÓб»ÕýÈ·±ê×¢³öÀ´£¬Õâ¾ÍÐèÒªÔÚ½¨Ä£Ç°»¨·Ñ´óÁ¿ÈËÁ¦À´ÐÞÕý¡£
£ÐÅÓÃÆÀ·Ö£¨Credit Scoring£©£ºÐèÒª¶ÔDZÔڵĸ߷çÏÕ¿Í»§½øÐг¤ÆÚ¸ú×Ù£¨±ÈÈçÁ½Ä꣩£¬´Ó¶ø»ýÀÛ×ã¹»µÄÆÀ·ÖÑù±¾¡£
1. Ì«¹Ø×¢ÑµÁ·£¨Focus on Training£©
IDMer£º¾ÍÏóÌåÓýѵÁ·ÖÐÔ½À´Ô½×¢ÖØÊµÕ½ÑµÁ·£¬ÒòΪµ¥´¿µÄ·â±ÕʽѵÁ·³£³£»áѵÁ·Ê±×´Ì¬ÉñÓ£¬±ÈÈüʱһËúºýÍ¿¡£
ʵ¼ÊÉÏ£¬Ö»ÓÐÑù±¾ÍâÊý¾ÝÉϵÄÄ£ÐÍÆÀ·Ö½á¹û²ÅÕæÕýÓÐÓã¡£¨·ñÔòµÄ»°£¬Ö±½ÓÓòÎÕÕ±íºÃÁË£¡£©
ÀýÈ磺
£°©Ö¢¼ì²â£¨Cancer detection£©£ºMD AndersonµÄÒ½ÉúºÍÑо¿ÈËÔ±£¨1993£©Ê¹ÓÃÉñ¾ÍøÂçÀ´½øÐа©Ö¢¼ì²â£¬¾ªÆæµØ·¢ÏÖ£¬ÑµÁ·Ê±¼äÔ½³¤£¨´Ó¼¸ÌìÑÓ³¤ÖÁÊýÖÜ£©£¬¶ÔѵÁ·¼¯µÄÐÔÄܸÄÉÆ·Ç³£Çá΢£¬µ«ÔÚ²âÊÔ¼¯ÉϵÄÐÔÄÜÈ´Ã÷ÏÔϽµ¡£
£»úÆ÷ѧϰ»ò¼ÆËã»ú¿ÆÑ§Ñо¿Õß³£³£ÊÔͼÈÃÄ£ÐÍÔÚÒÑÖªÊý¾ÝÉϱíÏÖ×îÓÅ£¬ÕâÑù×öµÄ½á¹ûͨ³£»áµ¼Ö¹ý¶ÈÄâºÏ£¨overfit£©¡£
½â¾ö·½·¨£º
½â¾öÕâ¸öÎÊÌâµÄµäÐÍ·½·¨ÊÇÖØ³éÑù£¨Re-Sampling£©¡£ÖسéÑù¼¼Êõ°üÀ¨£ºbootstrap¡¢cross-validation¡¢jackknife¡¢leave-one-out¡µÈµÈ¡£
2. Ö»ÒÀÀµÒ»Ïî¼¼Êõ£¨Rely on One Technique£©
IDMer£ºÕâ¸ö´íÎóºÍµÚ10ÖÖ´íÎóÓÐÏà֮ͨ´¦£¬Çëͬʱ²ÎÕÕÆä½â¾ö·½·¨¡£Ã»ÓжԱÈÒ²¾ÍûÓÐËùνµÄºÃ»µ£¬±çÖ¤·¨µÄ˼ÏëÔÚ´ËÌåÏÖÎÞÒÅ¡£
¡°µ±Ð¡º¢×ÓÊÖÄÃÒ»°Ñ´¸×Óʱ£¬Õû¸öÊÀ½ç¿´ÆðÀ´¾ÍÊÇһö¶¤×Ó¡£¡±ÒªÏëÈù¤×÷¾¡Éƾ¡ÃÀ£¬¾ÍÐèÒªÒ»Ì×ÍêÕûµÄ¹¤¾ßÏä¡£
²»Òª¼òµ¥µØÐÅÀµÄãÓõ¥¸ö·½·¨·ÖÎöµÄ½á¹û£¬ÖÁÉÙÒªºÍ´«Í³·½·¨£¨±ÈÈçÏßÐԻعé»òÏßÐÔÅбð·ÖÎö£©×ö¸ö±È½Ï¡£
Ñо¿½á¹û£º°´ÕÕ¡¶Éñ¾ÍøÂç¡·ÆÚ¿¯µÄͳ¼Æ£¬ÔÚ¹ýÈ¥3ÄêÀ´£¬Ö»ÓÐ1/6µÄÎÄÕÂÖÐ×öµ½ÁËÉÏÊöÁ½µã¡£Ò²¾ÍÊÇ˵£¬ÔÚ¶ÀÁ¢ÓÚѵÁ·Ñù±¾Ö®ÍâµÄ²âÊÔ¼¯ÉϽøÐÐÁË¿ª¼¯²âÊÔ£¬²¢ÓëÆäËü¹ã·º²ÉÓõķ½·¨½øÐÐÁ˶Աȡ£
½â¾ö·½·¨£º
ʹÓÃһϵÁкõŤ¾ßºÍ·½·¨¡££¨Ã¿ÖÖ¹¤¾ß»ò·½·¨¿ÉÄÜ×î¶à´øÀ´5%~10%µÄ¸Ä½ø£©¡£
3. Ìá´íÁËÎÊÌ⣨Ask the Wrong Question£©
IDMer£ºÒ»°ãÔÚ·ÖÀàËã·¨Öж¼»á¸ø³ö·ÖÀྫ¶È×÷ΪºâÁ¿Ä£ÐͺûµµÄ±ê×¼£¬µ«ÔÚʵ¼ÊÏîÄ¿ÖÐÎÒÃÇÈ´¼¸ºõ²»¿´Õâ¸öÖ¸±ê¡£ÎªÊ²Ã´£¿ÒòΪÄDz»ÊÇÎÒÃǹØ×¢µÄÄ¿±ê¡£
a)ÏîÄ¿µÄÄ¿±ê£ºÒ»¶¨ÒªËø¶¨ÕýÈ·µÄÄ¿±ê
ÀýÈ磺
ÆÛÕ©Õì²â£¨¹Ø×¢µÄÊÇÕýÀý£¡£©£¨ShannonʵÑéÊÒÔÚ¹ú¼Ê³¤Í¾µç»°ÉϵķÖÎö£©£º²»ÒªÊÔͼÔÚÒ»°ãµÄͨ»°ÖÐ°ÑÆÛÕ©ºÍ·ÇÆÛÕ©ÐÐΪ·ÖÀà³öÀ´£¬ÖصãÓ¦·ÅÔÚÈçºÎÃèÊöÕý³£Í¨»°µÄÌØÕ÷£¬È»ºó¾Ý´Ë·¢ÏÖÒ쳣ͨ»°ÐÐΪ¡£
b)Ä£Ð͵ÄÄ¿±ê£ºÈüÆËã»úÈ¥×öÄãÏ£ÍûËü×öµÄÊÂ
´ó¶àÊýÑо¿ÈËÔ±»á³ÁÃÔÓÚÄ£Ð͵ÄÊÕÁ²ÐÔÀ´¾¡Á¿½µµÍÎó²î£¬ÕâÑùÈÃËûÃÇ¿ÉÒÔ»ñµÃÊýѧÉϵÄÃÀ¸Ð¡£µ«¸üÓ¦¸ÃÈüÆËã»ú×öµÄÊÂÇéÓ¦¸ÃÊÇÈçºÎ¸ÄÉÆÒµÎñ£¬¶ø²»Êǽö½ö²àÖØÄ£ÐͼÆËãÉϵľ«¶È¡£
4. Ö»¿¿Êý¾ÝÀ´Ëµ»°£¨Listen (only) to the Data£©
IDMer£º¡°ÈÃÊý¾Ý˵»°¡±Ã»ÓÐ´í£¬¹Ø¼üÊÇ»¹Òª¼ÇµÃÁíÒ»¾ä»°£º¼æÌýÔòÃ÷£¬Æ«ÌýÔò°µ£¡Èç¹ûÊý¾Ý£«¹¤¾ß¾Í¿ÉÒÔ½â¾öÎÊÌâµÄ»°£¬»¹ÒªÈË×öÊ²Ã´ÄØ£¿
4a.Ͷ»úÈ¡ÇɵÄÊý¾Ý£ºÊý¾Ý±¾ÉíÖ»ÄܰïÖú·ÖÎöÈËÔ±ÕÒµ½Ê²Ã´ÊÇÏÔÖøµÄ½á¹û£¬µ«Ëü²¢²»ÄܸæËßÄã½á¹ûÊǶԻ¹ÊÇ´í¡£
4b.¾¹ýÉè¼ÆµÄʵÑ飺ijЩʵÑéÉè¼ÆÖвôÔÓÁËÈËΪµÄ³É·Ö£¬ÕâÑùµÄʵÑé½á¹ûÒ²³£³£²»¿ÉÐÅ¡£
5. ʹÓÃÁËδÀ´µÄÐÅÏ¢£¨Accept Leaks from the Future£©
IDMer£º¿´ËƲ»¿ÉÄÜ£¬È´ÊÇʵ¼ÊÖкÜÈÝÒ×·¸µÄ´íÎó£¬ÌرðÊÇÄãÃæ¶Ô³ÉǧÉÏÍò¸ö±äÁ¿µÄʱºò¡£ÈÏÕæ¡¢×Ðϸ¡¢ÓÐÌõÀíÊÇÊý¾ÝÍÚ¾òÈËÔ±µÄ»ù±¾ÒªÇó¡£
Ô¤±¨£¨Forecast£©Ê¾Àý£ºÔ¤±¨Ö¥¼Ó¸çÒøÐÐÔÚijÌìµÄÀûÂÊ£¬Ê¹ÓÃÉñ¾ÍøÂ罨죬ģÐ͵Ä׼ȷÂÊ´ïµ½95%¡£µ«ÔÚÄ£ÐÍÖÐȴʹÓÃÁ˸ÃÌìµÄÀûÂÊ×÷ΪÊäÈë±äÁ¿¡£
½ðÈÚÒµÖеÄÔ¤±¨Ê¾Àý£ºÊ¹ÓÃ3ÈÕµÄÒÆ¶¯Æ½¾ùÀ´Ô¤±¨£¬µ«È´°ÑÒÆ¶¯Æ½¾ùµÄÖеãÉèÔÚ½ñÌì¡£
½â¾ö·½·¨£º
Òª×Ðϸ²é¿´ÄÇЩÈýá¹û±íÏÖµÃÒì³£ºÃµÄ±äÁ¿£¬ÕâЩ±äÁ¿ÓпÉÄÜÊDz»Ó¦¸ÃʹÓ㬻òÕß²»Ó¦¸ÃÖ±½ÓʹÓõġ£
¸øÊý¾Ý¼ÓÉÏʱ¼ä´Á£¬±ÜÃâ±»ÎóÓá£
6. ÅׯúÁ˲»¸ÃºöÂԵݸÀý£¨Discount Pesky Cases£©
IDMer£ºµ½µ×ÊÇ¡°ÄþΪ¼¦Í·£¬²»Îª·ïβ¡±£¬»¹ÊÇ¡°´óÒþÒþÓÚÊУ¬Ð¡ÒþÒþÓÚÒ°¡±£¿²»Í¬µÄÈËÉú̬¶È¿ÉÒÔÓÐͬÑù¾«²ÊµÄÈËÉú£¬²»Í¬µÄÊý¾ÝÒ²¿ÉÄÜÔ̺¬Í¬ÑùÖØÒªµÄ¼ÛÖµ¡£
Òì³£Öµ¿ÉÄܻᵼÖ´íÎóµÄ½á¹û£¨±ÈÈç¼Û¸ñÖеÄСÊýµã±ê´íÁË£©£¬µ«Ò²¿ÉÄÜÊÇÎÊÌâµÄ´ð°¸£¨±ÈÈç³ôÑõ¶´£©¡£ËùÒÔÐèÒª×Ðϸ¼ì²éÕâЩÒì³£¡£
Ñо¿ÖÐ×îÈü¤¶¯µÄ»°Óï²»ÊÇ¡°°¡¹þ£¡¡±£¬¶øÊÇ¡°Õâ¾ÍÓÐµãÆæ¹ÖÁË¡¡¡±
Êý¾ÝÖеIJ»Ò»ÖÂÐÔÓпÉÄÜ»áÊǽâ¾öÎÊÌâµÄÏßË÷£¬ÉîÍÚÏÂÈ¥Ò²Ðí¿ÉÒÔ½â¾öÒ»¸ö´óµÄÒµÎñÎÊÌâ¡£
ÀýÈ磺
ÔÚÖ±ÓÊÓªÏúÖУ¬ÔÚ¶Ô¼ÒÍ¥µØÖ·µÄºÏ²¢ºÍÇåÏ´¹ý³ÌÖз¢ÏÖµÄÊý¾Ý²»Ò»Ö£¬·´¶ø¿ÉÄÜÊÇеÄÓªÏú»ú»á¡£
½â¾ö·½·¨£º
¿ÉÊÓ»¯¿ÉÒÔ°ïÖúÄã·ÖÎö´óÁ¿µÄ¼ÙÉèÊÇ·ñ³ÉÁ¢¡£
7. ÇáÐÅÔ¤²â£¨Extrapolate£©
IDMer£ºÒÀÈ»ÊDZçÖ¤·¨ÖеĹ۵㣬ÊÂÎï¶¼ÊDz»¶Ï·¢Õ¹±ä»¯µÄ¡£
ÈËÃdz£³£ÔÚ¾Ñé²»¶àµÄʱºòÇáÒ׵óöһЩ½áÂÛ¡£
¼´±ã·¢ÏÖÁËһЩ·´Àý£¬ÈËÃÇÒ²²»Ì«Ô¸Òâ·ÅÆúÔÏȵÄÏë·¨¡£
ά¶ÈÖäÓÔÚµÍά¶ÈÉϵÄÖ±¾õ£¬·ÅÔÚ¸ßά¶È¿Õ¼äÖУ¬³£³£ÊǺÁÎÞÒâÒåµÄ¡£
½â¾ö·½·¨£º
½ø»¯ÂÛ¡£Ã»ÓÐÕýÈ·µÄ½áÂÛ£¬Ö»ÓÐÔ½À´Ô½×¼È·µÄ½áÂÛ¡£
8. ÊÔͼ»Ø´ðËùÓÐÎÊÌ⣨Answer Every Inquiry£©
IDMer£ºÓеãÏñÎÒÅÀɽʱ¹ÄÀø×Ô¼ºµÄÒ»¾ä»°¡°ÎÒ²»ÖªµÀʲôʱºòÄܵÇÉÏɽ·å£¬µ«ÎÒÖªµÀÅÀÒ»²½¾ÍÀëÖÕµã½üÒ»²½¡£¡±
¡°²»ÖªµÀ¡±ÊÇÒ»ÖÖÓÐÒâÒåµÄÄ£Ðͽá¹û¡£
Ä£ÐÍÒ²ÐíÎÞ·¨100%׼ȷ»Ø´ðÎÊÌ⣬µ«ÖÁÉÙ¿ÉÒÔ°ïÎÒÃǹÀ¼Æ³öÏÖijÖÖ½á¹ûµÄ¿ÉÄÜÐÔ¡£
9. Ëæ±ãµØ½øÐгéÑù£¨Sample Casually£©
9a ½µµÍ³éÑùˮƽ¡£ÀýÈ磬MDÖ±Óʹ«Ë¾½øÐÐÏìÓ¦Ô¤²â·ÖÎö£¬µ«·¢ÏÖÊý¾Ý¼¯ÖеIJ»ÏìÓ¦¿Í»§Õ¼±ÈÌ«¸ß£¨×ܹ²Ò»°ÙÍòÖ±Óʿͻ§£¬ÆäÖг¬¹ý99%µÄÈËδ¶ÔÓªÏú×ö³öÏìÓ¦£©¡£ÓÚÊǽ¨Ä£ÈËÔ±×öÁËÈçϳéÑù£º°ÑËùÓÐÏìÓ¦Õß·ÅÈëÑù±¾¼¯£¬È»ºóÔÚËùÓв»ÏìÓ¦ÕßÖнøÐÐϵͳ³éÑù£¬¼´Ã¿¸ô10È˳éÒ»¸ö·ÅÈëÑù±¾¼¯£¬Ö±µ½Ñù±¾¼¯´ïµ½10ÍòÈË¡£µ«Ä£Ð;ÓÈ»µÃ³öÈçϹæÔò£º·²ÊǾÓסÔÚKetchikan¡¢WrangellºÍWard
Cove AlaskaµÄÈ˶¼»áÏìÓ¦ÓªÏú¡£ÕâÏÔÈ»ÊÇÓÐÎÊÌâµÄ½áÂÛ¡££¨ÎÊÌâ¾Í³öÔÚÕâÖÖ³éÑù·½·¨ÉÏ£¬ÒòΪÔʼÊý¾Ý¼¯ÒѾ°´ÕÕÓÊÕþ±àÂëÅÅÐò£¬ÉÏÃæÕâÈý¸öµØÇøÖв»ÏìÓ¦ÕßδÄܱ»³éÈ¡µ½Ñù±¾¼¯ÖУ¬¹Ê´ËµÃ³öÁËÕâÖÖ½áÂÛ£©¡£
½â¾ö·½·¨£º¡°ºÈǰҡһҡ£¡¡±ÏÈ´òÂÒÔʼÊý¾Ý¼¯ÖеÄ˳Ðò£¬´Ó¶ø±£Ö¤³éÑùµÄËæ»úÐÔ¡£
9b Ìá¸ß³éÑùˮƽ¡£ÀýÈ磬ÔÚÐÅÓÃÆÀ·ÖÖУ¬ÒòΪΥԼ¿Í»§µÄÕ¼±ÈÒ»°ã¶¼·Ç³£µÍ£¬ËùÒÔÔÚ½¨Ä£Ê±³£³£»áÈËΪµ÷¸ßÎ¥Ô¼¿Í»§µÄÕ¼±È£¨±ÈÈç°ÑÕâЩΥԼ¿Í»§µÄÈ¨ÖØÌá¸ß5±¶£©¡£½¨Ä£Öз¢ÏÖ£¬Ëæ×ÅÄ£ÐÍÔ½À´Ô½¸´ÔÓ£¬ÅбðÎ¥Ô¼¿Í»§µÄ׼ȷÂÊÒ²Ô½À´Ô½¸ß£¬µ«¶ÔÕý³£¿Í»§µÄÎóÅÐÂÊÒ²ËæÖ®Éý¸ß¡££¨ÎÊÌâ³öÔÚÊý¾Ý¼¯µÄ»®·ÖÉÏ¡£ÔÚ°ÑÔʼÊý¾Ý¼¯»®·ÖΪѵÁ·¼¯ºÍ²âÊÔ¼¯Ê±£¬ÔʼÊý¾Ý¼¯ÖÐÎ¥Ô¼¿Í»§µÄÈ¨ÖØÒѾ±»Ìá¸ß¹ýÁË£©
½â¾ö·½·¨£ºÏȽøÐÐÊý¾Ý¼¯»®·Ö£¬È»ºóÔÙÌá¸ßѵÁ·¼¯ÖÐÎ¥Ô¼¿Í»§µÄÈ¨ÖØ¡£
10. Ì«ÏàÐÅ×î¼ÑÄ£ÐÍ£¨Believe the Best Model£©
IDMer£º»¹ÊÇÄǾäÀÏ»°£¡°Ã»ÓÐ×îºÃ£¬Ö»ÓиüºÃ£¡¡±
¿É½âÊÍÐÔ²¢²»Ò»¶¨×ÜÊDZØÒªµÄ¡£¿´ÆðÀ´²¢²»ÍêÈ«ÕýÈ·»òÕß¿ÉÒÔ½âÊ͵ÄÄ£ÐÍ£¬ÓÐʱҲ»áÓÐÓá£
¡°×î¼Ñ¡±Ä£ÐÍÖÐʹÓõÄһЩ±äÁ¿£¬»á·ÖÉ¢ÈËÃÇÌ«¶àµÄ×¢ÒâÁ¦¡££¨²»¿É½âÊÍÐÔÓÐʱҲÊÇÒ»¸öÓŵ㣩
Ò»°ãÀ´Ëµ£¬ºÜ¶à±äÁ¿¿´ÆðÀ´±Ë´Ë¶¼ºÜÏàËÆ£¬¶ø×î¼ÑÄ£Ð͵Ľṹ¿´ÉÏȥҲǧ²îÍò±ð£¬ÎÞ¼£¿ÉÑ¡£µ«Ðè×¢ÒâµÄÊÇ£¬½á¹¹ÉÏÏàËÆ²¢²»Òâζ׏¦ÄÜÉÏÒ²ÏàËÆ¡£
½â¾ö·½·¨£º°Ñ¶à¸öÄ£Ðͼ¯×°ÆðÀ´¿ÉÄÜ»á´øÀ´¸üºÃ¸üÎȶ¨µÄ½á¹û¡£ |