Angel, What OS are you running on the VM? Some Linux variant?
The Linux kernel will get aggressive with processes that get really greedy with resources, especially memory. You might check your ‘dmesg’ logs to look for reports of the kernel asserting itself. It will summarily kill user processes if it deems it necessary to keep the OS stable. I’d be curious what the group thinks are the memory requirements for the heap but I suspect it is less that 700 MB if you’re just running some light, experimental code. Brian > On Feb 15, 2023, at 10:52 PM, Angel Motta <angelmo...@gmail.com> wrote: > > I tried with another virtual machine with 1GB of RAM and the jvm heap size > setting to 700MB and It worked! Kafka started successfully. > It looks like the default setting of 1GB heap size in the server.properties > is the minimum? or near the minimum right? > > I wanted the minimum resources (memory) because this is for development > purposes so I think I will have to consider this as the minimum. > I wanted to try the vanilla installation to get a little bit familiar > before trying the docker mode installation and I hope it works well :) > > Any comment about it is welcomed. I couldn't find any log indicating the > cause of the kill but I think It was an out of memory error so I share my > experience for anyone having similar issues. > > >> On Wed, Feb 15, 2023 at 9:58 PM Angel Motta <angelmo...@gmail.com> wrote: >> >> I deleted the content of the log directory and format again as is >> indicated in the quickstart but same problem. Kafka process is killed >> during startup in the same way. >> >> >> On Wed, Feb 15, 2023 at 9:11 PM sunil chaudhari < >> sunilmchaudhar...@gmail.com> wrote: >> >>> Ok sorry to misinterpret your message. >>> However if you are learning and its your first time, you can clear the >>> data from the logs directory defined in server.properties. >>> And start fresh instance and then see what happens. >>> I hope you have followed all documents properly. >>> >>>> On Thu, 16 Feb 2023 at 7:07 AM, Angel Motta <angelmo...@gmail.com> wrote: >>> >>>> Hello Sunil >>>> Thanks for your message. Only to clarify that I never could successfully >>>> start Kafka. This my first attempt to start Kafka and I get the "killed" >>>> message as shown in the log. >>>> Knowing this fact is necessary to remove all data and logs? If this >>> helps, >>>> could you tell me how to do this? >>>> >>>> >>>> >>>> On Wed, Feb 15, 2023 at 8:29 PM sunil chaudhari < >>>> sunilmchaudhar...@gmail.com> >>>> wrote: >>>> >>>>> Remove all data and logs. >>>>> And start it. >>>>> Next time when you want to stop then dont kill the process with kill >>>>> command. >>>>> Stop it gracefully using kafka-server-stop under /bin >>>>> Kafka needs stop signal to do some cleanup operations before it >>> stops. So >>>>> kill is not the option. >>>>> >>>>> On Thu, 16 Feb 2023 at 6:49 AM, Angel Motta <angelmo...@gmail.com> >>>> wrote: >>>>> >>>>>> Hello everyone! >>>>>> New Kafka user here. I am learning Kafka and trying to run it for >>> the >>>>>> first time (part of my CS undergrad thesis) >>>>>> I have the minimum server in AWS EC2 with 500MB of RAM and whenever >>> I >>>> try >>>>>> to start Kafka (kraft mode) I get "killed" as a last message. >>>>>> >>>>>> First I had to change the heap size from default (1GB) because I >>>> received >>>>>> an error about it so now It is 256M and now I see the logs of >>> startup >>>>>> process but the process is killed. >>>>>> >>>>>> How can I know the cause of this error? >>>>>> I paste the last part of the log here. I just followed the >>> quickstart >>>>>> https://kafka.apache.org/quickstart >>>>>> I also quickly saw the log directory (server, controller) but any >>> hint >>>> to >>>>>> me. >>>>>> >>>>>> Thanks in advance for your assistance. >>>>>> >>>>>> [2023-02-16 01:00:54,082] INFO [LogLoader >>>> partition=__cluster_metadata-0, >>>>>> dir=/tmp/kraft-combined-logs] Producer state recovery took 3ms for >>>>> snapshot >>>>>> load and 0ms for segment recovery from offset 28 >>>> (kafka.log.UnifiedLog$) >>>>>> >>>>>> [2023-02-16 01:00:54,131] INFO Initialized snapshots with IDs >>>> SortedSet() >>>>>> from /tmp/kraft-combined-logs/__cluster_metadata-0 >>>>>> (kafka.raft.KafkaMetadataLog$) >>>>>> >>>>>> [2023-02-16 01:00:54,183] INFO [raft-expiration-reaper]: Starting >>>>>> (kafka.raft.TimingWheelExpirationService$ExpiredOperationReaper) >>>>>> >>>>>> [2023-02-16 01:00:54,523] INFO [RaftManager nodeId=1] Completed >>>>> transition >>>>>> to ResignedState(localId=1, epoch=9, voters=[1], >>>> electionTimeoutMs=1695, >>>>>> unackedVoters=[], preferredSuccessors=[]) >>>>>> (org.apache.kafka.raft.QuorumState) >>>>>> >>>>>> [2023-02-16 01:00:54,533] INFO [RaftManager nodeId=1] Completed >>>>> transition >>>>>> to CandidateState(localId=1, epoch=10, retries=1, >>>> electionTimeoutMs=1539) >>>>>> (org.apache.kafka.raft.QuorumState) >>>>>> >>>>>> [2023-02-16 01:00:54,549] INFO [RaftManager nodeId=1] Completed >>>>> transition >>>>>> to Leader(localId=1, epoch=10, epochStartOffset=28, >>>>>> highWatermark=Optional.empty, voterStates={1=ReplicaState(nodeId=1, >>>>>> endOffset=Optional.empty, lastFetchTimestamp=-1, >>>>> lastCaughtUpTimestamp=-1, >>>>>> hasAcknowledgedLeader=true)}) (org.apache.kafka.raft.QuorumState) >>>>>> >>>>>> [2023-02-16 01:00:54,615] INFO [kafka-raft-outbound-request-thread]: >>>>>> Starting (kafka.raft.RaftSendThread) >>>>>> >>>>>> [2023-02-16 01:00:54,620] INFO [kafka-raft-io-thread]: Starting >>>>>> (kafka.raft.KafkaRaftManager$RaftIoThread) >>>>>> >>>>>> [2023-02-16 01:00:54,694] INFO [RaftManager nodeId=1] High watermark >>>> set >>>>> to >>>>>> LogOffsetMetadata(offset=29, >>>>>> >>>> metadata=Optional[(segmentBaseOffset=0,relativePositionInSegment=3044)]) >>>>>> for the first time for epoch 10 based on indexOfHw 0 and voters >>>>>> [ReplicaState(nodeId=1, >>> endOffset=Optional[LogOffsetMetadata(offset=29, >>>>>> >>>>> >>>> >>> metadata=Optional[(segmentBaseOffset=0,relativePositionInSegment=3044)])], >>>>>> lastFetchTimestamp=-1, lastCaughtUpTimestamp=-1, >>>>>> hasAcknowledgedLeader=true)] (org.apache.kafka.raft.LeaderState) >>>>>> >>>>>> [2023-02-16 01:00:54,723] INFO [RaftManager nodeId=1] Registered the >>>>>> listener org.apache.kafka.image.loader.MetadataLoader@1633422834 >>>>>> (org.apache.kafka.raft.KafkaRaftClient) >>>>>> >>>>>> [2023-02-16 01:00:54,761] INFO [RaftManager nodeId=1] Registered the >>>>>> listener >>>>>> >>>>>> >>>>> >>>> >>> org.apache.kafka.controller.QuorumController$QuorumMetaLogListener@629125780 >>>>>> (org.apache.kafka.raft.KafkaRaftClient) >>>>>> >>>>>> [2023-02-16 01:00:54,780] INFO [MetadataLoader 1] Publishing initial >>>>>> snapshot at offset 27 to SnapshotGenerator >>>>>> (org.apache.kafka.image.loader.MetadataLoader) >>>>>> >>>>>> [2023-02-16 01:00:54,798] INFO [ThrottledChannelReaper-Fetch]: >>> Starting >>>>>> (kafka.server.ClientQuotaManager$ThrottledChannelReaper) >>>>>> >>>>>> [2023-02-16 01:00:54,799] INFO [ThrottledChannelReaper-Produce]: >>>> Starting >>>>>> (kafka.server.ClientQuotaManager$ThrottledChannelReaper) >>>>>> >>>>>> [2023-02-16 01:00:54,808] INFO [ThrottledChannelReaper-Request]: >>>> Starting >>>>>> (kafka.server.ClientQuotaManager$ThrottledChannelReaper) >>>>>> >>>>>> [2023-02-16 01:00:54,813] INFO >>>>> [ThrottledChannelReaper-ControllerMutation]: >>>>>> Starting (kafka.server.ClientQuotaManager$ThrottledChannelReaper) >>>>>> >>>>>> [2023-02-16 01:00:54,849] INFO [ExpirationReaper-1-AlterAcls]: >>> Starting >>>>>> (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper) >>>>>> >>>>>> [2023-02-16 01:00:54,865] INFO [SocketServer >>> listenerType=CONTROLLER, >>>>>> nodeId=1] Enabling request processing. (kafka.network.SocketServer) >>>>>> >>>>>> [2023-02-16 01:00:54,874] INFO [BrokerServer id=1] Transition from >>>>> SHUTDOWN >>>>>> to STARTING (kafka.server.BrokerServer) >>>>>> >>>>>> [2023-02-16 01:00:54,877] INFO [BrokerServer id=1] Starting broker >>>>>> (kafka.server.BrokerServer) >>>>>> >>>>>> [2023-02-16 01:00:54,918] INFO [ThrottledChannelReaper-Fetch]: >>> Starting >>>>>> (kafka.server.ClientQuotaManager$ThrottledChannelReaper) >>>>>> >>>>>> [2023-02-16 01:00:54,918] INFO [ThrottledChannelReaper-Produce]: >>>> Starting >>>>>> (kafka.server.ClientQuotaManager$ThrottledChannelReaper) >>>>>> >>>>>> [2023-02-16 01:00:54,919] INFO [ThrottledChannelReaper-Request]: >>>> Starting >>>>>> (kafka.server.ClientQuotaManager$ThrottledChannelReaper) >>>>>> >>>>>> [2023-02-16 01:00:54,919] INFO >>>>> [ThrottledChannelReaper-ControllerMutation]: >>>>>> Starting (kafka.server.ClientQuotaManager$ThrottledChannelReaper) >>>>>> >>>>>> [2023-02-16 01:00:54,957] INFO [BrokerToControllerChannelManager >>>> broker=1 >>>>>> name=forwarding]: Starting >>>> (kafka.server.BrokerToControllerRequestThread) >>>>>> >>>>>> [2023-02-16 01:00:54,972] INFO [BrokerToControllerChannelManager >>>> broker=1 >>>>>> name=forwarding]: Recorded new controller, from now on will use node >>>>>> localhost:9093 (id: 1 rack: null) >>>>>> (kafka.server.BrokerToControllerRequestThread) >>>>>> >>>>>> [2023-02-16 01:00:55,059] INFO Updated connection-accept-rate max >>>>>> connection creation rate to 2147483647 >>> (kafka.network.ConnectionQuotas) >>>>>> >>>>>> [2023-02-16 01:00:55,065] INFO Awaiting socket connections on >>>>> 0.0.0.0:9092 >>>>>> . >>>>>> (kafka.network.DataPlaneAcceptor) >>>>>> >>>>>> [2023-02-16 01:00:55,074] INFO [SocketServer listenerType=BROKER, >>>>> nodeId=1] >>>>>> Created data-plane acceptor and processors for endpoint : >>>>>> ListenerName(PLAINTEXT) (kafka.network.SocketServer) >>>>>> >>>>>> [2023-02-16 01:00:55,082] INFO [BrokerToControllerChannelManager >>>> broker=1 >>>>>> name=alterPartition]: Starting >>>>>> (kafka.server.BrokerToControllerRequestThread) >>>>>> >>>>>> [2023-02-16 01:00:55,082] INFO [BrokerToControllerChannelManager >>>> broker=1 >>>>>> name=alterPartition]: Recorded new controller, from now on will use >>>> node >>>>>> localhost:9093 (id: 1 rack: null) >>>>>> (kafka.server.BrokerToControllerRequestThread) >>>>>> >>>>>> [2023-02-16 01:00:55,101] INFO [ExpirationReaper-1-Produce]: >>> Starting >>>>>> (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper) >>>>>> >>>>>> [2023-02-16 01:00:55,101] INFO [ExpirationReaper-1-Fetch]: Starting >>>>>> (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper) >>>>>> >>>>>> [2023-02-16 01:00:55,101] INFO [ExpirationReaper-1-DeleteRecords]: >>>>> Starting >>>>>> (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper) >>>>>> >>>>>> [2023-02-16 01:00:55,112] INFO [ExpirationReaper-1-ElectLeader]: >>>> Starting >>>>>> (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper) >>>>>> >>>>>> [2023-02-16 01:00:55,152] INFO [ExpirationReaper-1-Heartbeat]: >>> Starting >>>>>> (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper) >>>>>> >>>>>> [2023-02-16 01:00:55,153] INFO [ExpirationReaper-1-Rebalance]: >>> Starting >>>>>> (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper) >>>>>> >>>>>> [2023-02-16 01:00:55,282] INFO [RaftManager nodeId=1] Registered the >>>>>> listener kafka.server.metadata.BrokerMetadataListener@1398770581 >>>>>> (org.apache.kafka.raft.KafkaRaftClient) >>>>>> >>>>>> [2023-02-16 01:00:55,287] INFO [BrokerToControllerChannelManager >>>> broker=1 >>>>>> name=heartbeat]: Starting >>>> (kafka.server.BrokerToControllerRequestThread) >>>>>> >>>>>> [2023-02-16 01:00:55,287] INFO [BrokerToControllerChannelManager >>>> broker=1 >>>>>> name=heartbeat]: Recorded new controller, from now on will use node >>>>>> localhost:9093 (id: 1 rack: null) >>>>>> (kafka.server.BrokerToControllerRequestThread) >>>>>> >>>>>> [2023-02-16 01:00:55,301] INFO [BrokerLifecycleManager id=1] >>>> Incarnation >>>>>> K3MwUk8zQUGgIXaeg0yY_w of broker 1 in cluster >>> zodne5JyThyBUzTtQnIADg is >>>>> now >>>>>> STARTING. (kafka.server.BrokerLifecycleManager) >>>>>> >>>>>> [2023-02-16 01:00:55,393] INFO [ExpirationReaper-1-AlterAcls]: >>> Starting >>>>>> (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper) >>>>>> >>>>>> [2023-02-16 01:00:55,405] INFO [BrokerServer id=1] Waiting for >>> broker >>>>>> metadata to catch up. (kafka.server.BrokerServer) >>>>>> >>>>>> [2023-02-16 01:00:55,537] INFO [BrokerLifecycleManager id=1] >>>> Successfully >>>>>> registered broker 1 with broker epoch 30 >>>>>> (kafka.server.BrokerLifecycleManager) >>>>>> >>>>>> [2023-02-16 01:00:55,546] INFO [BrokerLifecycleManager id=1] The >>> broker >>>>> has >>>>>> caught up. Transitioning from STARTING to RECOVERY. >>>>>> (kafka.server.BrokerLifecycleManager) >>>>>> >>>>>> [2023-02-16 01:00:55,549] INFO [BrokerMetadataListener id=1] >>> Starting >>>> to >>>>>> publish metadata events at offset 30. >>>>>> (kafka.server.metadata.BrokerMetadataListener) >>>>>> >>>>>> [2023-02-16 01:00:55,552] INFO [BrokerMetadataPublisher id=1] >>>> Publishing >>>>>> initial metadata at offset OffsetAndEpoch(offset=30, epoch=10) with >>>>>> metadata.version 3.4-IV0. >>>> (kafka.server.metadata.BrokerMetadataPublisher) >>>>>> >>>>>> [2023-02-16 01:00:55,553] INFO Loading logs from log dirs >>>>>> ArraySeq(/tmp/kraft-combined-logs) (kafka.log.LogManager) >>>>>> >>>>>> [2023-02-16 01:00:55,563] INFO [BrokerLifecycleManager id=1] The >>> broker >>>>> is >>>>>> in RECOVERY. (kafka.server.BrokerLifecycleManager) >>>>>> >>>>>> [2023-02-16 01:00:55,565] INFO Attempting recovery for all logs in >>>>>> /tmp/kraft-combined-logs since no clean shutdown file was found >>>>>> (kafka.log.LogManager) >>>>>> >>>>>> [2023-02-16 01:00:55,591] INFO Loaded 0 logs in 38ms. >>>>>> (kafka.log.LogManager) >>>>>> >>>>>> [2023-02-16 01:00:55,594] INFO Starting log cleanup with a period of >>>>> 300000 >>>>>> ms. (kafka.log.LogManager) >>>>>> >>>>>> [2023-02-16 01:00:55,599] INFO Starting log flusher with a default >>>> period >>>>>> of 9223372036854775807 ms. (kafka.log.LogManager) >>>>>> >>>>>> Killed >>>>>> >>>>> >>>> >>> >>
