Hi, Rohit, The log for RocksDB should show whether the compact happened or not. You can find the log in the container's runtime state-store directory here: <container-runtime-dir-depending-on-installation-script>/state/<store-name>/<Partition_num>/LOG
My guess is that you might not have enough entries / writes to trigger the compaction yet. You can reach out to RocksDB community for questions regarding to when and how the compaction is triggered. Best, -Yi On Fri, Mar 11, 2016 at 11:36 AM, Rohit Bansal <bansalrohi...@gmail.com> wrote: > I waited for a considerable timing, around 15-20 minutes. There were no > logs of compaction though. How can I check if compaction occured or not ? > > On Sat, Mar 12, 2016 at 1:04 AM, Yi Pan <nickpa...@gmail.com> wrote: > > > Hi, Rohit, > > > > RocksDB TTL is best-effort: > > https://github.com/facebook/rocksdb/wiki/Time-to-Live. And the expired > > records are removed during compaction. > > > > I wonder in your test case whether the compaction has ever happened. How > > long did you wait to see whether records are gone or not? > > > > -Yi > > > > On Thu, Mar 10, 2016 at 9:50 PM, Rohit Bansal <bansalrohi...@gmail.com> > > wrote: > > > > > Navina, > > > > > > Please find my configurations file below - > > > > > > # Licensed to the Apache Software Foundation (ASF) under one > > > # or more contributor license agreements. See the NOTICE file > > > # distributed with this work for additional information > > > # regarding copyright ownership. The ASF licenses this file > > > # to you under the Apache License, Version 2.0 (the > > > # "License"); you may not use this file except in compliance > > > # with the License. You may obtain a copy of the License at > > > # > > > # http://www.apache.org/licenses/LICENSE-2.0 > > > # > > > # Unless required by applicable law or agreed to in writing, > > > # software distributed under the License is distributed on an > > > # "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY > > > # KIND, either express or implied. See the License for the > > > # specific language governing permissions and limitations > > > # under the License. > > > > > > job.factory.class=org.apache.samza.job.local.ThreadJobFactory > > > job.name=stream-poc > > > job.coordinator.system=kafka > > > job.coordinator.replication.factor=1 > > > > > > task.class=samza.stream.poc.Consolidate > > > task.inputs=stream.consolidate > > > task.consumer.batch.size=100 > > > task.window.ms=20000 > > > > > > # Serializers > > > > > > > > > serializers.registry.json.class=org.apache.samza.serializers.JsonSerdeFactory > > > > > > > > > serializers.registry.string.class=org.apache.samza.serializers.StringSerdeFactory > > > > > > # Systems > > > > > > > > > systems.stream.samza.factory=org.apache.samza.system.kafka.KafkaSystemFactory > > > systems.stream.samza.key.serde=string > > > systems.stream.samza.msg.serde=string > > > systems.stream.consumer.zookeeper.connect=localhost:2181 > > > systems.stream.producer.bootstrap.servers=localhost:9092 > > > > > > # Kafka System > > > > > > > > > systems.kafka.samza.factory=org.apache.samza.system.kafka.KafkaSystemFactory > > > systems.kafka.samza.msg.serde=string > > > systems.kafka.samza.key.serde=string > > > systems.kafka.consumer.zookeeper.connect=localhost:2181 > > > systems.kafka.producer.bootstrap.servers=localhost:9092 > > > > > > > > > > > > stores.rocksdb.factory=org.apache.samza.storage.kv.RocksDbKeyValueStorageEngineFactory > > > stores.rocksdb.key.serde=string > > > stores.rocksdb.msg.serde=string > > > stores.rocksdb.rocksdb.ttl.ms=2000 > > > > > > > > > *My test case :-* > > > > > > *Putting data in store (with TTL 2 seconds) and then getting data from > > > store and printing in console in window() method which is executed in > > > windowing operation in every 20 seconds. Data is printed in all > windowing > > > operation executions and is not getting deleted. It should have printed > > > null.* > > > > > > *Thanks* > > > > > > On Fri, Mar 11, 2016 at 12:31 AM, Navina Ramesh < > > > nram...@linkedin.com.invalid> wrote: > > > > > > > Hi Rohit, > > > > Are you running samza 0.10.0 or the latest from trunk? > > > > > > > > It will be helpful to paste your config here. Please do not attach it > > to > > > > the email because the mail server filters attachments. > > > > > > > > Thanks! > > > > Navina > > > > > > > > On Thu, Mar 10, 2016 at 3:05 AM, Rohit Bansal < > bansalrohi...@gmail.com > > > > > > > wrote: > > > > > > > > > Hi, > > > > > > > > > > My messages are not getting deleted even after I have mentioned > 2000 > > ms > > > > as > > > > > TTL in properties file. Do I need to add any other properties or > make > > > > > changes to any configuration for TTL to evict messages from db ? I > > can > > > > > attach my task file and properties file if needed. > > > > > > > > > > -- > > > > > > > > > > Regards > > > > > > > > > > Rohit Bansal > > > > > > > > > > +91-7838732062 > > > > > > > > > > > > > > > > > > > > > -- > > > > Navina R. > > > > > > > > > > > > > > > > -- > > > > > > Regards > > > > > > Rohit Bansal > > > > > > +91-7838732062 > > > > > > > > > -- > > Regards > > Rohit Bansal > > +91-7838732062 >
