addu390 commented on issue #384: URL: https://github.com/apache/flink-agents/issues/384#issuecomment-3978774049
@wenjin272 I can take this on. I have a question on the batching strategy, should we keep it simple with a configurable batch size on `CompactionConfig`, or go with something like a token budget approach (estimate tokens per item, cap each batch to a context window limit)? Batch size is simpler but less precise and could under/over batch unnecessarily. I'm thinking batch size with a sensible default to begin with since we don't have a tokenizer handy anyway. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
