[SpringBoot] mybatis jdbcTemplate jpa/hibernate batch insert throughput 비교

• 성능 차이 관련 글
  • https://stackoverrun.com/ko/q/3938448
    • 요약 : JDBC로 batch insert를 사용하는 것이 대부분 더 나은 퍼포먼스를 얻을 수 있다.
  • https://homoefficio.github.io/2020/01/25/Spring-Data에서-Batch-Insert-최적화/
    • 요약 : Spring Data JPA/Hibernate 방식과 Spring Data JDBC 방식을 비교하였을 때 Spring Data JDBC 방식이 빠르다.

---

성능 테스트

테스트 환경

• H/W
  • 프로세서 : Intel(R) Core(TM) i7-8700 CPU @ 3.20GHz (12 CPU), ~3.2GHz
  • 메모리 : 32GB
  • OS : Windows 10 Pro 64bit
• IDE : Eclipse (2020-03)
• Build : Maven
• Compiler : JDK 8
• Spring Boot 2.2.6
• PostgreSQL 12.2
• Spring JDBC 5.2.5
• Spring Data JPA 2.2.6
• Hibernate Core 5.4.12.Final
• Hibernate Commons Annotations 5.1.0.Final
• Consumer Thread 3개
• LMAX Exchange에서 개발한 Concurrency Ring Buffer 'Disruptor' 사용

결과

• 테스트 툴 : JMeter
  • Thread 50
  • Loop Count 10,000
• Spring JDBC - jdbcTemplate.batchUpdate 사용
  • Throughput : 19726.2/sec
• Spring JPA - JpaRepository 사용
  • Throughput : 11594.5/sec
• Mybatis - foreach 태그 list insert 사용
  • Throughput : 14115.5/sec
• 테스트 데이터 : 1309.0 bytes 크기의 JSON 타입

jdbcTemplate.batchUpdate 사용 코드

List<DocUsageDTO> batchArgs = new ArrayList<DocUsageDTO>();

@Override
public void onEvent(AsyncLogEvent event, long sequence, boolean endOfBatch) throws Exception {
	if (sequence % CONSUMER_THREADS == threadId) {
		list.add(event);
	}

	if (endOfBatch) {
		insertData(list.size());
		list.clear();
	}
}

public void insertData(int listSize) throws SQLException {
	StringBuffer query = new StringBuffer();
	query.append("INSERT INTO doc_usage VALUES (");
	for (int i = 0; i < PARAMETERS; i++) {
		query.append("?,");
	}
	query.append("?);");

	for(AsyncLogEvent e : list) {
		batchArgs.add(e.getValue());
	}

	jdbcTemplate.batchUpdate(query.toString(), batchArgs, listSize,
			new ParameterizedPreparedStatementSetter<DocUsageDTO>() {
				@Override
				public void setValues(PreparedStatement ps, DocUsageDTO arg) throws SQLException {
					// parameter 값 설정
				}
			});
	
	batchArgs.clear();
	for (AsyncLogEvent e : list) {
		e.getResult().setResult(e.getValue());
	}
}

 

 

Spring JPA - JpaRepository 사용 코드

List<DocUsageDTO> batchArgs = new ArrayList<DocUsageDTO>();

@Override
public void onEvent(AsyncLogEvent event, long sequence, boolean endOfBatch) throws Exception {
	if(sequence % CONSUMER_THREADS == threadId) {
		list.add(event);
	}

	if(endOfBatch) {
		insertata();
		list.clear();
	}
}

public void insertData() throws SQLException {
	for(AsyncLogEvent e : list) {
		batchArgs.add(e.getValue());
	}
	
	repository.saveAll(batchArgs);
	
	for(AsyncLogEvent e : list) {
		e.getResult().setResult(e.getValue());
	}
 
	batchArgs.clear();
}

application.yml

# Server Configuration
server:
  tomcat:
    maxThreads: 200

# Database Configuration
spring: 
  datasource: 
    driver-class-name: org.postgresql.Driver
    url: jdbc:postgresql://localhost:5432/test
    username: 
    password: 
    
    # Connection pool
    hikari:
      connectionTimeout: 30000
      validationTimeout: 1000
      maxPoolSize: 30
      maxLifetime: 300000
      maximumPoolSize: 100
      auto-commit: true
   
spring.jpa:
  database: PostgreSQL
  properties.hibernate:
    dialect: org.hibernate.dialect.PostgreSQLDialect

Mybatis 사용 코드

DocUsageInsertService.java

private final List<AsyncLogEvent> list = new ArrayList<AsyncLogEvent>();
private final List<DocUsageDTO> dataList = new ArrayList<DocUsageDTO>();

@Override
public void onEvent(AsyncLogEvent event, long sequence, boolean endOfBatch) throws Exception {
		if (sequence % CONSUMER_THREADS == threadId) {
				list.add(event);
		}

		if(endOfBatch && list.size() != 0) {
				insertData();
				list.clear();
		}
}

public void insertData() throws SQLException {
		for(AsyncLogEvent e : list) {
			dataList.add(e.getValue());
		}

		docUsageMapper.insertDocUsage(dataList);

		for(AsyncLogEvent e : list) {
			e.getResult().setResult(e.getValue());
		}

		dataList.clear();
}

 

DocUsageMapper.xml

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE mapper PUBLIC "-//mybatis.org//DTD Mapper 3.0//EN" "http://mybatis.org/dtd/mybatis-3-mapper.dtd">

<mapper namespace="com.mybatis.test.mapper.DocUsageMapper">
	<insert id="insertDocUsage" parameterType="java.util.List">
		INSERT INTO doc_usage VALUES
		<foreach collection="list" item="item" separator=",">
			(#{item.arg1}, #{item.arg2}, #{item.arg2}... #{item.arg44})
		</foreach>
	</insert>
</mapper>