Remove all data and logs. And start it. Next time when you want to stop then dont kill the process with kill command. Stop it gracefully using kafka-server-stop under /bin Kafka needs stop signal to do some cleanup operations before it stops. So kill is not the option.
On Thu, 16 Feb 2023 at 6:49 AM, Angel Motta <angelmo...@gmail.com> wrote: > Hello everyone! > New Kafka user here. I am learning Kafka and trying to run it for the > first time (part of my CS undergrad thesis) > I have the minimum server in AWS EC2 with 500MB of RAM and whenever I try > to start Kafka (kraft mode) I get "killed" as a last message. > > First I had to change the heap size from default (1GB) because I received > an error about it so now It is 256M and now I see the logs of startup > process but the process is killed. > > How can I know the cause of this error? > I paste the last part of the log here. I just followed the quickstart > https://kafka.apache.org/quickstart > I also quickly saw the log directory (server, controller) but any hint to > me. > > Thanks in advance for your assistance. > > [2023-02-16 01:00:54,082] INFO [LogLoader partition=__cluster_metadata-0, > dir=/tmp/kraft-combined-logs] Producer state recovery took 3ms for snapshot > load and 0ms for segment recovery from offset 28 (kafka.log.UnifiedLog$) > > [2023-02-16 01:00:54,131] INFO Initialized snapshots with IDs SortedSet() > from /tmp/kraft-combined-logs/__cluster_metadata-0 > (kafka.raft.KafkaMetadataLog$) > > [2023-02-16 01:00:54,183] INFO [raft-expiration-reaper]: Starting > (kafka.raft.TimingWheelExpirationService$ExpiredOperationReaper) > > [2023-02-16 01:00:54,523] INFO [RaftManager nodeId=1] Completed transition > to ResignedState(localId=1, epoch=9, voters=[1], electionTimeoutMs=1695, > unackedVoters=[], preferredSuccessors=[]) > (org.apache.kafka.raft.QuorumState) > > [2023-02-16 01:00:54,533] INFO [RaftManager nodeId=1] Completed transition > to CandidateState(localId=1, epoch=10, retries=1, electionTimeoutMs=1539) > (org.apache.kafka.raft.QuorumState) > > [2023-02-16 01:00:54,549] INFO [RaftManager nodeId=1] Completed transition > to Leader(localId=1, epoch=10, epochStartOffset=28, > highWatermark=Optional.empty, voterStates={1=ReplicaState(nodeId=1, > endOffset=Optional.empty, lastFetchTimestamp=-1, lastCaughtUpTimestamp=-1, > hasAcknowledgedLeader=true)}) (org.apache.kafka.raft.QuorumState) > > [2023-02-16 01:00:54,615] INFO [kafka-raft-outbound-request-thread]: > Starting (kafka.raft.RaftSendThread) > > [2023-02-16 01:00:54,620] INFO [kafka-raft-io-thread]: Starting > (kafka.raft.KafkaRaftManager$RaftIoThread) > > [2023-02-16 01:00:54,694] INFO [RaftManager nodeId=1] High watermark set to > LogOffsetMetadata(offset=29, > metadata=Optional[(segmentBaseOffset=0,relativePositionInSegment=3044)]) > for the first time for epoch 10 based on indexOfHw 0 and voters > [ReplicaState(nodeId=1, endOffset=Optional[LogOffsetMetadata(offset=29, > metadata=Optional[(segmentBaseOffset=0,relativePositionInSegment=3044)])], > lastFetchTimestamp=-1, lastCaughtUpTimestamp=-1, > hasAcknowledgedLeader=true)] (org.apache.kafka.raft.LeaderState) > > [2023-02-16 01:00:54,723] INFO [RaftManager nodeId=1] Registered the > listener org.apache.kafka.image.loader.MetadataLoader@1633422834 > (org.apache.kafka.raft.KafkaRaftClient) > > [2023-02-16 01:00:54,761] INFO [RaftManager nodeId=1] Registered the > listener > > org.apache.kafka.controller.QuorumController$QuorumMetaLogListener@629125780 > (org.apache.kafka.raft.KafkaRaftClient) > > [2023-02-16 01:00:54,780] INFO [MetadataLoader 1] Publishing initial > snapshot at offset 27 to SnapshotGenerator > (org.apache.kafka.image.loader.MetadataLoader) > > [2023-02-16 01:00:54,798] INFO [ThrottledChannelReaper-Fetch]: Starting > (kafka.server.ClientQuotaManager$ThrottledChannelReaper) > > [2023-02-16 01:00:54,799] INFO [ThrottledChannelReaper-Produce]: Starting > (kafka.server.ClientQuotaManager$ThrottledChannelReaper) > > [2023-02-16 01:00:54,808] INFO [ThrottledChannelReaper-Request]: Starting > (kafka.server.ClientQuotaManager$ThrottledChannelReaper) > > [2023-02-16 01:00:54,813] INFO [ThrottledChannelReaper-ControllerMutation]: > Starting (kafka.server.ClientQuotaManager$ThrottledChannelReaper) > > [2023-02-16 01:00:54,849] INFO [ExpirationReaper-1-AlterAcls]: Starting > (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper) > > [2023-02-16 01:00:54,865] INFO [SocketServer listenerType=CONTROLLER, > nodeId=1] Enabling request processing. (kafka.network.SocketServer) > > [2023-02-16 01:00:54,874] INFO [BrokerServer id=1] Transition from SHUTDOWN > to STARTING (kafka.server.BrokerServer) > > [2023-02-16 01:00:54,877] INFO [BrokerServer id=1] Starting broker > (kafka.server.BrokerServer) > > [2023-02-16 01:00:54,918] INFO [ThrottledChannelReaper-Fetch]: Starting > (kafka.server.ClientQuotaManager$ThrottledChannelReaper) > > [2023-02-16 01:00:54,918] INFO [ThrottledChannelReaper-Produce]: Starting > (kafka.server.ClientQuotaManager$ThrottledChannelReaper) > > [2023-02-16 01:00:54,919] INFO [ThrottledChannelReaper-Request]: Starting > (kafka.server.ClientQuotaManager$ThrottledChannelReaper) > > [2023-02-16 01:00:54,919] INFO [ThrottledChannelReaper-ControllerMutation]: > Starting (kafka.server.ClientQuotaManager$ThrottledChannelReaper) > > [2023-02-16 01:00:54,957] INFO [BrokerToControllerChannelManager broker=1 > name=forwarding]: Starting (kafka.server.BrokerToControllerRequestThread) > > [2023-02-16 01:00:54,972] INFO [BrokerToControllerChannelManager broker=1 > name=forwarding]: Recorded new controller, from now on will use node > localhost:9093 (id: 1 rack: null) > (kafka.server.BrokerToControllerRequestThread) > > [2023-02-16 01:00:55,059] INFO Updated connection-accept-rate max > connection creation rate to 2147483647 (kafka.network.ConnectionQuotas) > > [2023-02-16 01:00:55,065] INFO Awaiting socket connections on 0.0.0.0:9092 > . > (kafka.network.DataPlaneAcceptor) > > [2023-02-16 01:00:55,074] INFO [SocketServer listenerType=BROKER, nodeId=1] > Created data-plane acceptor and processors for endpoint : > ListenerName(PLAINTEXT) (kafka.network.SocketServer) > > [2023-02-16 01:00:55,082] INFO [BrokerToControllerChannelManager broker=1 > name=alterPartition]: Starting > (kafka.server.BrokerToControllerRequestThread) > > [2023-02-16 01:00:55,082] INFO [BrokerToControllerChannelManager broker=1 > name=alterPartition]: Recorded new controller, from now on will use node > localhost:9093 (id: 1 rack: null) > (kafka.server.BrokerToControllerRequestThread) > > [2023-02-16 01:00:55,101] INFO [ExpirationReaper-1-Produce]: Starting > (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper) > > [2023-02-16 01:00:55,101] INFO [ExpirationReaper-1-Fetch]: Starting > (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper) > > [2023-02-16 01:00:55,101] INFO [ExpirationReaper-1-DeleteRecords]: Starting > (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper) > > [2023-02-16 01:00:55,112] INFO [ExpirationReaper-1-ElectLeader]: Starting > (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper) > > [2023-02-16 01:00:55,152] INFO [ExpirationReaper-1-Heartbeat]: Starting > (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper) > > [2023-02-16 01:00:55,153] INFO [ExpirationReaper-1-Rebalance]: Starting > (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper) > > [2023-02-16 01:00:55,282] INFO [RaftManager nodeId=1] Registered the > listener kafka.server.metadata.BrokerMetadataListener@1398770581 > (org.apache.kafka.raft.KafkaRaftClient) > > [2023-02-16 01:00:55,287] INFO [BrokerToControllerChannelManager broker=1 > name=heartbeat]: Starting (kafka.server.BrokerToControllerRequestThread) > > [2023-02-16 01:00:55,287] INFO [BrokerToControllerChannelManager broker=1 > name=heartbeat]: Recorded new controller, from now on will use node > localhost:9093 (id: 1 rack: null) > (kafka.server.BrokerToControllerRequestThread) > > [2023-02-16 01:00:55,301] INFO [BrokerLifecycleManager id=1] Incarnation > K3MwUk8zQUGgIXaeg0yY_w of broker 1 in cluster zodne5JyThyBUzTtQnIADg is now > STARTING. (kafka.server.BrokerLifecycleManager) > > [2023-02-16 01:00:55,393] INFO [ExpirationReaper-1-AlterAcls]: Starting > (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper) > > [2023-02-16 01:00:55,405] INFO [BrokerServer id=1] Waiting for broker > metadata to catch up. (kafka.server.BrokerServer) > > [2023-02-16 01:00:55,537] INFO [BrokerLifecycleManager id=1] Successfully > registered broker 1 with broker epoch 30 > (kafka.server.BrokerLifecycleManager) > > [2023-02-16 01:00:55,546] INFO [BrokerLifecycleManager id=1] The broker has > caught up. Transitioning from STARTING to RECOVERY. > (kafka.server.BrokerLifecycleManager) > > [2023-02-16 01:00:55,549] INFO [BrokerMetadataListener id=1] Starting to > publish metadata events at offset 30. > (kafka.server.metadata.BrokerMetadataListener) > > [2023-02-16 01:00:55,552] INFO [BrokerMetadataPublisher id=1] Publishing > initial metadata at offset OffsetAndEpoch(offset=30, epoch=10) with > metadata.version 3.4-IV0. (kafka.server.metadata.BrokerMetadataPublisher) > > [2023-02-16 01:00:55,553] INFO Loading logs from log dirs > ArraySeq(/tmp/kraft-combined-logs) (kafka.log.LogManager) > > [2023-02-16 01:00:55,563] INFO [BrokerLifecycleManager id=1] The broker is > in RECOVERY. (kafka.server.BrokerLifecycleManager) > > [2023-02-16 01:00:55,565] INFO Attempting recovery for all logs in > /tmp/kraft-combined-logs since no clean shutdown file was found > (kafka.log.LogManager) > > [2023-02-16 01:00:55,591] INFO Loaded 0 logs in 38ms. > (kafka.log.LogManager) > > [2023-02-16 01:00:55,594] INFO Starting log cleanup with a period of 300000 > ms. (kafka.log.LogManager) > > [2023-02-16 01:00:55,599] INFO Starting log flusher with a default period > of 9223372036854775807 ms. (kafka.log.LogManager) > > Killed >
