±à¼ÍƼö: |
±¾ÎÄÖ÷Òª½éÉÜÁ˼ǼʹÓÃspring
batch×öÊý¾ÝÇ¨ÒÆÊ±Ê±Óöµ½µÄÒ»¸ö¹Ø¼üÎÊÌ⣺Êý¾ÝÇ¨ÒÆÁ¿´óʱÈçºÎ±£Ö¤ÄÚ´æµÈÏà¹ØÄÚÈÝ¡£
±¾ÎÄÀ´×ÔÓÚ CSDN £¬ÓÉ»ðÁú¹ûÈí¼þAnna±à¼¡¢ÍƼö¡£ |
|
¸ÅÊö
±¾Æª²©¿ÍÊǼǼʹÓÃspring batch×öÊý¾ÝÇ¨ÒÆÊ±Ê±Óöµ½µÄÒ»¸ö¹Ø¼üÎÊÌ⣺Êý¾ÝÇ¨ÒÆÁ¿´óʱÈçºÎ±£Ö¤ÄÚ´æ¡£µ±ÎÒÃÇÔÚʹÓÃspring
batchʱ£¬ÎÒÃDZØÐëÅäÖÃÈý¸ö¶«Î÷: reader£¬processor£¬ºÍwriter¡£ÆäÖУ¬readerÓÃÓÚ´ÓÊý¾Ý¿âÖжÁÊý¾Ý£¬µ±Êý¾ÝÁ¿½ÏСʱ£¬readerµÄÂß¼²»»á¶ÔÄÚ´æ´øÀ´Ì«¶àѹÁ¦£¬µ«Êǵ±ÎÒÃÇҪȥ¶ÁµÄÊý¾ÝÁ¿·Ç³£´óµÄʱºò£¬ÎÒÃǾͲ»µÃ²»¿¼ÂÇÄÚ´æµÈ·½ÃæµÄÎÊÌ⣬ÒòΪÈôÊý¾ÝÁ¿·Ç³£´ó£¬Äڴ棬ִÐÐʱ¼äµÈµÈ¶¼»áÊܵ½Ó°Ïì¡£¹ØÓÚspring
batchµÄ»ù´¡ÖªÊ¶ºÍ½éÉÜÇë²Î¿¼ÕâÆª²©¿Í£º Åú´¦Àí¿ò¼Üspring batch½éÉܼ°Ê¹Óá£
ÎÊÌâÊÇʲô
ÔÚÉÏÃæµÄÄÚÈݵ±ÖÐÎÒÃÇÒѾÌáµ½ÁË,ÎÒÃÇÃæÁÙµÄÎÊÌâÊÇÊý¾ÝÇ¨ÒÆÁ¿´óʱµÄÄÚ´æÎÊÌâ¡£µ«ÊÇÕâÑùµÄÃèÊö·Ç³£Áýͳ£¬Òò´Ë²©Ö÷¾ö¶¨½«ÕâÒ»²¿·Öµ¥¶ÀÁà³öÀ´Ëµ¡£
ÔÚѧϰÁËspring batchµÄ֪ʶ֮ºóÎÒÃÇÓ¦¸ÃºÜÇå³þµÄÒ»µãÊÇ£¬Ã¿Ò»¸öspring batchµÄstep¶¼°üº¬ÈçϵIJ¿·Ö:

¼´¶ÁÊý¾Ý£¬´¦ÀíÊý¾Ý£¬Ð´Êý¾Ý¡£ÕâÈý¸ö²½ÖèÀïÃæ×î¿ÉÄܻᵼÖÂÄÚ´æ±ä´óÎÊÌâµÄÎÞÒÉÊǶÁÊý¾Ý»·½Ú¡£¶ÁÊý¾Ý×÷Ϊspring
batchµÄÊý¾ÝÊäÈ룬ÊÇÕû¸öspring batch jobµÄ¿ªÍ·Âß¼¡£
ÈôÎÒÃǵÄÊý¾ÝÁ¿²»´ó£¬ÈçÖ»Óм¸Ê®ÍòÌõ£¬ÄÇÎÒÃÇÎÞÒɲ»»áÃæÁÙÄÚ´æÎÊÌ⣬¼´±ãÒ»´Î½«ËùÓÐÊý¾Ý¼ÓÔØµ½ÄÚ´æµ±ÖУ¬Õ¼µÄÄÚ´æÒ²²»»á·Ç³£¶à£¬ÇÒspring
batchÊý¾ÝÇ¨ÒÆµÄËٶȷdz£Ö®¿ì£¬¼¸Ê®ÍòÌõµÄÊý¾ÝÍùÍùÊǼ¸Ê®ÃëµÄʱ¼ä¾Í¿ÉÒÔÇ¨ÒÆÍê³É¡£µ«Êǵ±Êý¾ÝÁ¿±ä´óÖ®ºó£¬ÎÊÌâ¾Í²»Ò»ÑùÁË¡£µ±ÎÒÃǵÄÊý¾ÝÁ¿´ïµ½Êý°ÙÍò»òÉÏǧÍòʱ£¬ÈôÒ»´ÎÐÔ½«ËùÓÐÊý¾ÝÈ«²¿¶Áµ½ÄÚ´æµ±ÖУ¬Ôò»áÕ¼¾ÝÔ¶Ô¶³¬³öÕý³£·¶Î§µÄ·Ç³£´óµÄÄÚ´æ¡£¸ÃÎÊÌâʾÒâͼÈçÏÂËùʾ:

ÎÒÃÇдµÄÈκγÌÐò¶¼»áÓÐÒ»¸öÔËÐÐÄÚ´æ,¼ÙÉèÕâ¸öÄÚ´æµÄ×ÜÈÝÁ¿ÏÖÔÚÖ»ÓÐ4g£¬¶øÎÒÃÇÊý¾Ý¿âÀïÐèÒª²Ù×÷µÄÊý¾ÝÓÐ8g£¬ÄÇôÎÞÒÉ£¬Ò»´ÎÐԵĽ«Êý¾Ý¶Á³öÀ´¾Í»á³ö´í¡£Õâ±ãÊÇÐèÒª¿¼ÂǵÃÎÊÌâ¡£
SpringÌṩµÄreaderʵÏÖ
springÌṩÁ˷dz£·á¸»µÄReaderʵÏÖ£¬ÆäÖбȽϳ£ÓõĴÓÊý¾Ý¿â¶ÁÊý¾ÝµÄÓÐJdbcCursorItemReader,JdbcPagingItemReaderµÈ¡£
JdbcCursorItemReader
ʹÓÃJdbcCursorItemReaderµÄʾÀý´úÂëÈçÏ£º
@Bean
public JdbcCursorItemReader<CustomerCredit>
itemReader() {
return new JdbcCursorItemReaderBuilder<CustomerCredit>()
.dataSource(this.dataSource)
.name("creditReader")
.sql("select ID, NAME, CREDIT from CUSTOMER")
.rowMapper(new CustomerCreditRowMapper())
.build();
}
|
JdbcCursorItemReaderµÄºÃ´¦ÔÚÓÚʹÓüòµ¥£¬µ«ÊÇÎÒÃÇ´ÓËüµÄsql¾ÍÄÜ·¢ÏÖ£¬JdbcCursorItemReader»áÒ»´Î°ÑËùÓеÄÊý¾ÝÈ«²¿ÄûØÀ´£¬µ±Êý¾ÝÁ¿¹ý´ó¶ø·þÎñÆ÷ÄÚ´æ²»¹»Ê±£¬¾Í»áÓöµ½ÏÂÃæÎÞ·¨·ÖÅäÄÚ´æµÄÎÊÌ⣺

±¨´íÐÅϢΪ:Resource exhaustion event:The
JVM was unable to allocate memory from the heap. Òâ˼¾ÍÊÇÐèÒª·ÖÅäÄÚ´æµÄÊý¾ÝÌ«¶à£¬µ«ÊÇÎÞ·¨ÕÒµ½×ã¹»µÄÄÚ´æÁË¡£
·´Ó³ÔÚÄÚ´æÀ¶ÑÄÚ´æ»á³ÊÏÖ³öÈçϵÄÇé¿ö:

Ëæ×Åÿһ´ÎÊý¾Ý¶ÁÈ룬¶ÑÄÚ´æ¶¼»áÔö´ó£¬ÔÒò¾ÍÔÚÓÚJdbcCursorItemReaderÒ»´ÎÐÔ¶Á»ØÁËËùÓеÄÊý¾Ý£¬·µ»ØÖ®ºó¾Í»á´æÔÚÒ»¸ö¶ÔÏóÀïÃæ£¬¶øÕâ¸ö¶ÔÏóµÄ³ß´ç¹ý´ó£¬Òò´ËÖ±½Ó½øÈëÁËÀÏÄê´ú¡£ÔÚÊý¾ÝÇ¨ÒÆÍê³É֮ǰ£¬ÕâЩÊý¾Ý¶¼²»»á±»»ØÊÕ¡£ÈçÏÂͼËùʾ:

ºÁÎÞÒÉÎÊ£¬µ±ÎÒÃǵÄÊý¾ÝÁ¿´óʱ²»Ó¦¸ÃʹÓÃÕâÖÖÀàÐ͵ÄreaderÀ´¶ÁÈ¡Êý¾Ý¡£
JdbcPagingItemReader
JdbcPagingItemReaderµÄ×÷ÓúÍËüµÄÃû×ÖÒ»Ñù£¬Ëü¿ÉÒÔ·ÖÒ³¶ÁÈ¡Êý¾Ý£¬µ«ÊÇʹÓÃÆðÀ´Ïà±ÈÓÚJdbcCursorItemReader¸ü¼Ó¸´ÔÓ£¬Ê¾Àý´úÂëÈçÏÂ:
@Bean
public JdbcPagingItemReader itemReader(DataSource
dataSource, PagingQueryProvider queryProvider)
{
Map<String, Object> parameterValues = new
HashMap<>();
parameterValues.put("status", "NEW");
return new JdbcPagingItemReaderBuilder<CustomerCredit>()
.name("creditReader")
.dataSource(dataSource)
.queryProvider(queryProvider)
.parameterValues(parameterValues)
.rowMapper(customerCreditMapper())
.pageSize(1000)
.build();
}
@Bean
public SqlPagingQueryProviderFactoryBean queryProvider()
{
SqlPagingQueryProviderFactoryBean provider = new
SqlPagingQueryProviderFactoryBean();
provider.setSelectClause("select id, name,
credit");
provider.setFromClause("from customer");
provider.setWhereClause("where status=:status");
provider.setSortKey("id");
}
|
¿ÉÒÔ¿´µ½ÎÒÃÇÄܹ»ÉèÖÃpageµÄ´óС£¬JdbcPagingItemReader½«¸ù¾ÝÕâ¸öÒ³µÄ´óС£¬Ã¿´Î¶ÁÈ¡Õâô¶àµÄÊý¾Ý£¬Òò´ËÕâЩÊý¾Ý·µ»Ø±£´æµÄ¶ÔÏ󣬾ÍÖ»»áÊÇС¶ÔÏó£¬Òò´ËËûÃDz»»áÖ±½ÓÔÚÀÏÄê´úÀï·ÖÅ䣬¶øÊÇÏÈ·ÖÅäÔÚÄêÇá´ú£¬Ëæ×ÅÄêÇá´ú²»¶Ï±ä´ó£¬minor
gcÒ²²»¶Ï½øÐУ¬»ØÊÕµôÒѾ´¦ÀíÍêµÄÊý¾Ý£¬ÀÏÄê´úµÄÄÚ´æÊ¹ÓÃÁ¿²»»áÓÐÈκÎÔö´ó£¬ÀàËÆÏÂͼ£º
ÀÏÄê´úÄÚ´æ²»»áÓÐÈκα仯£¬ÄêÇá´ø»áËæ×Å·þÎñÆ÷Êý¾ÝÇ¨ÒÆ½øÐжøÔö´óͬʱ±»»ØÊÕ¡£
ÔÚʹÓÃJdbcPagingItemReaderʱ£¬ÓÐÒ»¸ö±ØÐë×¢ÒâµÄµØ·½¾ÍÊÇÅÅÐò¹Ø¼ü×ÖÊDZØÐëÖ¸¶¨µÄ£¬ÔÒòÔÚÓÚÅÅÐòÊÇ·ÖҳʵÏÖÔÀíµÄ¼¼Êõ»ù´¡¡£sortKeyºÍÎÒÃÇÖ¸¶¨µÄÆäËû×Ö¾äÒ»Æð¹¹½¨³öSQLÓï¾ä³öÀ´¡£ÔÚsortKeyÉϱØÐëʹÓÃunique
key constraintÔ¼Êø£¬ÒòΪֻÓÐÕâÑù²ÅÄܵÃÒÔÈ·±£Ö´ÐÐÖ®¼ä²»»á¶ªÊ§ÈκÎÊý¾Ý¡£ÕâÒ²¿ÉÒÔ˵ÊÇJdbcCursorItemReaderÏà¶Ô±ãÀûµÄÒ»µãÓÅÊÆ¡£
×ܽá
Êý¾ÝÁ¿Ð¡Ê±Ñ¡ÔñµÄ·½°¸²î±ð²»»áºÜ´ó£¬µ±Êý¾ÝÁ¿´óʱ£¬ÎªÁËÓкõÄÄÚ´æ±íÏÖÔòʹÓ÷ÖÒ³µÄreaderÊDZØÒªµÄ¡£µ«Í¬Ê±£¬ÒòΪҪʵÏÖ·ÖÒ³£¬Ò²»á´øÀ´Ò»Ð©²»¿É±ÜÃâµÄÏÞÖÆ¡£
|