mybatis操作mariadb驱动的mysql批量插入报错java.nio.BufferOverflowException: null,
mybatis操作mariadb驱动的mysql批量插入报错java.nio.BufferOverflowException: null
业务中需要定期向数据库中写入统计数据,数据量比较大。在往数据库中写入时有时会报缓冲区溢出异常。最后的异常栈如下:
Caused by: java.nio.BufferOverflowException: null
at java.nio.HeapByteBuffer.put(HeapByteBuffer.java:189)
at java.nio.ByteBuffer.put(ByteBuffer.java:859)
at org.mariadb.jdbc.internal.packet.send.SendExecutePrepareStatementPacket.send(SendExecutePrepareStatementPacket.java:105)
at org.mariadb.jdbc.internal.protocol.AbstractQueryProtocol.executePreparedQuery(AbstractQueryProtocol.java:578)
at org.mariadb.jdbc.MariaDbServerPreparedStatement.executeInternal(MariaDbServerPreparedStatement.java:279)
at org.mariadb.jdbc.MariaDbServerPreparedStatement.execute(MariaDbServerPreparedStatement.java:369)
at com.alibaba.druid.pool.DruidPooledPreparedStatement.execute(DruidPooledPreparedStatement.java:493)
… 31 common frames omitted
经过一番痛苦的源码解读,终于找到了异常的发生原因。要从异常栈中的SendExecutePrepareStatementPacket类的send方法说起,源码如下:
public int send(final OutputStream os) throws IOException {
PacketOutputStream buffer = (PacketOutputStream) os;
buffer.startPacket(0, true);
buffer.buffer.put((byte) 0x17);
buffer.buffer.putInt(statementId);
buffer.buffer.put((byte) 0x00); //CURSOR TYPE NO CURSOR TODO implement when using cursor
buffer.buffer.putInt(1); //Iteration count
//create null bitmap
if (parameterCount > 0) {
int nullCount = (parameterCount + 7) / 8;
byte[] nullBitsBuffer = new byte[nullCount];
for (int i = 0; i < parameterCount; i++) {
if (parameters[i] instanceof NullParameter) {
nullBitsBuffer[i / 8] |= (1 << (i % 8));
}
}
buffer.buffer.put(nullBitsBuffer);/*Null Bit Map*/
//check if parameters type (using setXXX) have change since previous request, and resend new header type if so
boolean mustSendHeaderType = false;
if (parameterTypeHeader[0] == null) {
mustSendHeaderType = true;
} else {
for (int i = 0; i < this.parameterCount; i++) {
if (!parameterTypeHeader[i].equals(parameters[i].getMariaDbType())) {
mustSendHeaderType = true;
break;
}
}
}
if (mustSendHeaderType) {
buffer.buffer.put((byte) 0x01);
//Store types of parameters in first in first package that is sent to the server.
for (int i = 0; i < this.parameterCount; i++) {
parameterTypeHeader[i] = parameters[i].getMariaDbType();
parameters[i].writeBufferType(buffer);
}
} else {
buffer.buffer.put((byte) 0x00);
}
}
for (int i = 0; i < parameterCount; i++) {
if (parameters[i] instanceof NotLongDataParameterHolder) {
((NotLongDataParameterHolder) parameters[i]).writeBinary(buffer);
}
}
buffer.finishPacket();
return 0;
}
源码中的nullBitsBuffer数组的尺寸是由传入的参数的数据量决定的。而buffer.buffer.put(nullBitsBuffer)语句中的buffer.buffer由如下代码构造:
public PacketOutputStream(OutputStream outputStream) {
this.outputStream = outputStream;
buffer = ByteBuffer.allocate(1024).order(ByteOrder.LITTLE_ENDIAN);
this.seqNo = -1;
useCompression = false;
}
因此nullBitsBuffer尺寸超过1024后就会报错。需要确保每次提交数据库的数据量不能过大;