Gabriel39 commented on code in PR #70693:
URL: https://github.com/apache/doris/pull/70693#discussion_r4637638458


##########
fe/fe-core/src/main/java/org/apache/doris/tls/server/OssFeServerStarterProvider.java:
##########
@@ -73,36 +77,51 @@ private OssThriftServerStarter(int port, ServerSocket 
listener) {
 
         @Override
         public void start() throws Exception {
-            if (Config.enable_tls && 
TlsProtocolSet.isProtocolIncluded(TlsProtocolSet.Protocol.THRIFT)) {
-                throw new UnsupportedOperationException("FE thrift TLS 
requires TLS module");
+            boolean tls = 
PrivateRpcTls.enabled(TlsProtocolSet.Protocol.THRIFT);
+            if (tls && 
ThriftServer.THREADED_SELECTOR.equalsIgnoreCase(Config.thrift_server_type)) {
+                throw new UnsupportedOperationException("Private RPC TLS 
requires a blocking Thrift server");
             }
-            FrontendServiceImpl service = new 
FrontendServiceImpl(ExecuteEnv.getInstance());
-            Logger feServiceLogger = 
LogManager.getLogger(FrontendServiceImpl.class);
-            FrontendService.Iface instance = (FrontendService.Iface) 
Proxy.newProxyInstance(
-                    FrontendServiceImpl.class.getClassLoader(),
-                    FrontendServiceImpl.class.getInterfaces(),
-                    (proxy, method, args) -> {
-                        long begin = System.currentTimeMillis();
-                        String name = method.getName();
-                        if (MetricRepo.isInit) {
-                            
MetricRepo.THRIFT_COUNTER_RPC_ALL.getOrAdd(name).increase(1L);
-                        }
-                        feServiceLogger.debug("receive request for {}", name);
-                        try {
-                            return method.invoke(service, args);
-                        } finally {
-                            ConnectContext.remove();
-                            feServiceLogger.debug("finish process request for 
{}", name);
+            ServerSocket transportListener = listener;
+            boolean started = false;
+            try {
+                if (tls) {
+                    transportListener = PrivateRpcTls.thriftServerSocket(new 
InetSocketAddress(

Review Comment:
   Fixed in f93e5d1c86d18129b54115ea6d00f6a8a076fab7. The native writer now 
supplies the local BE's built-in Thrift TLS policy, node identity and bounded 
connect timeout, overriding sink-supplied values. The isolated Java plugin 
authenticates that FE connection, verifies its hostname and never falls back to 
plaintext. Reconnect and master redirection retain the same policy; 
native-compatible PKCS#1 keys are supported using the existing pinned BC 
version.
   
   Real Thrift/TLS tests exercise initial allocation and the writer's actual 
rotation callback, including a preallocated first block, plus redirection and 
negative authentication/timeout cases. The callback tests reproduce the 
original failure. All 27 selected MaxCompute tests and 33 related native ASAN 
tests pass. The MaxCompute storage session is mocked; the FE callback uses a 
real listener. A separate isolated-class-loader probe using only the deployed 
plugin directory also authenticates with PKCS#1 and reuses its connection.



##########
regression-test/suites/rpc_p0/test_remote_doris_private_rpc_tls.groovy:
##########
@@ -0,0 +1,188 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+import groovy.json.JsonSlurper
+import org.apache.doris.regression.suite.ConnectionInfo
+
+import java.nio.file.Files
+import java.sql.DriverManager
+import java.sql.SQLException
+import java.util.concurrent.TimeUnit
+
+suite("test_remote_doris_private_rpc_tls", "p0,nonConcurrent") {
+    def logRoot = new File(System.getProperty("DORIS_HOME"), 
"regression-test/log").toPath()
+    Files.createDirectories(logRoot)
+    def logs = Files.createTempDirectory(logRoot, "private-rpc-tls-").toFile()
+    def helper = new File(context.file.parentFile, 
"private_rpc_tls_fixture.py")
+    // Keep lifecycle and port-allocation checks in the default regression 
entry point as well.
+    def fixtureTests = new ProcessBuilder("python3", "-m", "unittest", 
"discover",

Review Comment:
   Fixed in f93e5d1c86d18129b54115ea6d00f6a8a076fab7. The suite checks for 
Linux before creating logs, running helper tests or launching engines, and logs 
an explicit unsupported-host skip. The Linux NonConcurrent path remains enabled 
by default, without a test-specific opt-in flag or supplied remote endpoint 
settings.
   
   A simulated-macOS admission test fails at the process boundary before the 
fix and passes afterward. Normal default-profile Linux replay on the fresh 
FE/ASAN BE packages passes all six result blocks, five expected refusals and 
nine helper tests, with zero skipped scripts.



##########
fe/fe-core/src/main/java/org/apache/doris/qe/runtime/PipelineExecutionTaskBuilder.java:
##########
@@ -81,12 +90,13 @@ private Map<BackendWorker, MultiFragmentsPipelineTask> 
buildMultiFragmentTasks(
             ByteString serializeFragments = 
workerToSerializeFragments.get(worker);
 
             Backend backend = worker.getBackend();
+            // Remote catalog workers keep their independent cluster's channel 
policy.
             fragmentTasks.put(
                     worker,
                     new MultiFragmentsPipelineTask(
                             coordinatorContext,
                             backend,
-                            backendServiceProxy,
+                            isInternalWorker(worker) ? backendServiceProxy : 
BackendServiceProxy.getInstance(false),

Review Comment:
   Implemented in f93e5d1c86d18129b54115ea6d00f6a8a076fab7. BackendServiceProxy 
now binds an immutable BackendRpcTarget from the address and worker catalog 
identity. CoordinatorContext retains one binding per worker, and 
MultiFragmentsPipelineTask and ResultReceiver share it for prepare, start, 
cancel, fetch and failed-channel removal. The binding retains its policy and 
proxy shard while the existing proxy cache continues to handle DNS turnover and 
channel recreation; it does not pin a physical channel. Legacy constructors 
remain available.
   
   Complete execution-graph admission remains in the coordinator before 
dispatch and fragment serialization. Binding itself opens no channel. A 
repeated-fetch test reproduces the former re-selection, and a shared-worker 
test verifies all five operations use the same binding; same-address 
local/remote isolation and rejected-graph tests remain. The official 60-test 
FE/BE Java selection and real default TLS/plaintext mixed-join regression pass.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to