Kafka community, Getting *java.lang.OutOfMemoryError: Requested array size exceeds VM * *limit,* when Atlas hook in Hive tries to send Kafka message. Let me know if any workaround possible on this issue or this is because of JDK issue https://bugs.openjdk.java.net/browse/JDK-8154035.
2018-09-10 15:12:57,961 ERROR HiveHook.java:160 - HiveHook.run(): failed to process operation QUERY *java.lang.OutOfMemoryError: Requested array size exceeds VM limit *at java.lang.StringCoding$StringEncoder.encode(StringCoding.java:300) at java.lang.StringCoding.encode(StringCoding.java:344) at java.lang.String.getBytes(String.java:918) at org.apache.kafka.common.serialization.StringSerializer.serialize(StringSerializer.java:47) at org.apache.kafka.common.serialization.StringSerializer.serialize(StringSerializer.java:28) at org.apache.kafka.common.serialization.ExtendedSerializer$Wrapper.serialize(ExtendedSerializer.java:65) at org.apache.kafka.common.serialization.ExtendedSerializer$Wrapper.serialize(ExtendedSerializer.java:55) at org.apache.kafka.clients.producer.KafkaProducer.doSend(KafkaProducer.java:783) at org.apache.kafka.clients.producer.KafkaProducer.send(KafkaProducer.java:760) at org.apache.kafka.clients.producer.KafkaProducer.send(KafkaProducer.java:648) at org.apache.atlas.kafka.KafkaNotification.sendInternalToProducer(KafkaNotification.java:197) at org.apache.atlas.kafka.KafkaNotification.sendInternal(KafkaNotification.java:182) at org.apache.atlas.notification.AbstractNotification.send(AbstractNotification.java:117) at org.apache.atlas.hook.AtlasHook.notifyEntitiesInternal(AtlasHook.java:143) at org.apache.atlas.hook.AtlasHook.notifyEntities(AtlasHook.java:128) at org.apache.atlas.hook.AtlasHook.notifyEntities(AtlasHook.java:181) at org.apache.atlas.hive.hook.HiveHook.run(HiveHook.java:157) at org.apache.atlas.hive.hook.HiveHook.run(HiveHook.java:52) at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1619) at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1303) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1170) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1165) at org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:197) at org.apache.hive.service.cli.operation.SQLOperation.access$300(SQLOperation.java:76) at org.apache.hive.service.cli.operation.SQLOperation$2$1.run(SQLOperation.java:255) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1869) at org.apache.hive.service.cli.operation.SQLOperation$2.run(SQLOperation.java:266) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
