Alanxtl commented on code in PR #692: URL: https://github.com/apache/dubbo-go-pixiu/pull/692#discussion_r2250288764
########## pkg/cluster/retry/noretry/no_retry.go: ########## @@ -0,0 +1,47 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package noretry + +import ( + "github.com/apache/dubbo-go-pixiu/pkg/cluster/retry" + "github.com/apache/dubbo-go-pixiu/pkg/model" +) + +func init() { + retry.RegisterRetryPolicy(model.RetryerNoRetry, newNoRetryPolicy) +} + +type NoRetryPolicy struct { + firstTime bool +} + +func (n *NoRetryPolicy) Attempt(err error) bool { + if !n.firstTime { + n.firstTime = true + return true // Allow the first attempt + } + return false +} + +func (n *NoRetryPolicy) Reset() { + n.firstTime = false +} + +func newNoRetryPolicy(config map[string]any) (retry.Retryer, error) { Review Comment: 这个函数要注册到`retry`的工厂函数里面 `pkg/cluster/retry/retry.go:49` `pkg/cluster/retry/noretry/no_retry.go:26` 由工厂函数生成实例，要是弄成成员函数的话工厂函数就不好弄了，而且`NoRetryPolicy`本身也不是单例的，没必要弄成成员函数 ########## docs/ai/endpoint.md: ########## @@ -0,0 +1,153 @@ + +## LLM Gateway Endpoint Configuration + +English | [中文](./endpoint_CN.md) + +This document explains how to configure upstream endpoints for Large Language Models (LLMs) within your gateway's routing configuration. + +### Endpoint Structure + +Each endpoint within a cluster is defined by an id and can contain an llm_meta block for custom behavior.clusters: + +```yaml +clusters: + - name: "my_llm_cluster" + endpoints: + - id: "provider-1-main" + llm_meta: + provider: "deepseek" + # ... other LLM-specific configuration goes here ... + - id: "provider-2-fallback" + llm_meta: + provider: "openai" + # ... other LLM-specific configuration goes here ... +``` + +### `llm_meta` Configuration Fields + +The llm_meta block holds all the configuration specific to how the gateway should treat this LLM endpoint. + +`provider` + +- Type: `string` +- Description: A name to identify the LLM provider check [here]() for all supported llm providers. This is primarily for routing for specific llm provider. + +`fallback` + +- Type: `boolean` +- Description: Determines if the gateway should proceed to the next endpoint in the cluster if all retry attempts on this endpoint fail.true: If this endpoint fails, the gateway will attempt the next available endpoint.false: If this endpoint fails, the process stops, and the last error is returned to the client. This should be set to false for the last endpoint in a fallback chain. Review Comment: done -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
