davidradl commented on code in PR #27385: URL: https://github.com/apache/flink/pull/27385#discussion_r2660815162
########## flink-models/flink-model-triton/README.md: ########## @@ -0,0 +1,174 @@ +# Flink Triton Model Integration + +This module provides integration between Apache Flink and NVIDIA Triton Inference Server, enabling real-time model inference within Flink streaming applications. + +## Features + +- **REST API Integration**: Communicates with Triton Inference Server via HTTP/REST API +- **Asynchronous Processing**: Non-blocking inference requests for high throughput +- **Flexible Configuration**: Comprehensive configuration options for various use cases +- **Error Handling**: Built-in retry mechanisms and error handling +- **Resource Management**: Efficient HTTP client pooling and resource management + +## Configuration Options + +### Required Options + +| Option | Type | Description | +|--------|------|-------------| +| `endpoint` | String | Full URL of the Triton Inference Server endpoint (e.g., `http://localhost:8000/v2/models`) | +| `model-name` | String | Name of the model to invoke on Triton server | +| `model-version` | String | Version of the model to use (defaults to "latest") | + +### Optional Options + +| Option | Type | Default | Description | +|--------|------|---------|-------------| +| `timeout` | Long | 30000 | Request timeout in milliseconds | +| `max-retries` | Integer | 3 | Maximum number of retries for failed requests | +| `batch-size` | Integer | 1 | Batch size for inference requests | +| `priority` | Integer | - | Request priority level (0-255, higher values = higher priority) | Review Comment: what does `-` mean? Is there a concept of not having a priority or is this the same as 0? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
