damccorm commented on PR #37528: URL: https://github.com/apache/beam/pull/37528#issuecomment-3862477231
> I see a few flakes related to #37528 but https://github.com/apache/beam/actions/runs/21734367880/job/62696049665?pr=37528 maybe looks concerning > > ``` > self = <apache_beam.ml.inference.model_manager_test.TestModelManager testMethod=test_single_model_convergence_with_fluctuations> > > def test_single_model_convergence_with_fluctuations(self): > """ > Tests that the estimator converges to the true usage with fluctuations. > """ > model_name = "fluctuating_model" > model_cost = 3000.0 > load_cost = 2500.0 > # Fix random seed for reproducibility > random.seed(42) > > def loader(): > self.mock_monitor.allocate(load_cost) > return model_name > > model = self.manager.acquire_model(model_name, loader) > self.manager.release_model(model_name, model) > initial_est = self.manager._estimator.get_estimate(model_name) > self.assertEqual(initial_est, load_cost) > > def run_inference(): > model = self.manager.acquire_model(model_name, loader) > noise = model_cost - load_cost + random.uniform(-300.0, 300.0) > self.mock_monitor.allocate(noise) > time.sleep(0.1) > self.mock_monitor.free(noise) > self.manager.release_model(model_name, model) > return > > with ThreadPoolExecutor(max_workers=8) as executor: > futures = [executor.submit(run_inference) for _ in range(100)] > > for f in futures: > f.result() > > est_cost = self.manager._estimator.get_estimate(model_name) > > self.assertAlmostEqual(est_cost, model_cost, delta=100.0) > E AssertionError: 2768.996662756712 != 3000.0 within 100.0 delta (231.00333724328812 difference) > ``` > > I don't _think_ this is caused by this PR, but any ideas on what might be causing this? Actually, I see it here as well https://github.com/apache/beam/pull/37506 So it is probably unrelated, but may need follow up as well. Rerunning checks now -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
