Re: Error "record is out of upper bound"

2023-03-06 Thread Svetak Sundhar via user
For posterity here met with Nick over GVC, the issue is coming from the file Bruno identified above, and is a warning for their custom histogram logic. Svetak Sundhar Technical Solutions Engineer, Data s vetaksund...@google.com On Fri, Mar 3, 2023 at 1:40 PM Bruno Volpato via user wrote:

Re: Error "record is out of upper bound"

2023-03-03 Thread Bruno Volpato via user
Found some more info, sorry for the chopping. Are you using *bigqueryio* or *bigquery_tools* somehow? If so, biguquery_tools defines a histogram using 20 buckets of 3 seconds each to export latencies (see https://github.com/apache/beam/blob/master/sdks/python/apache_beam/io/gcp/bigquery_tools.py#L

Re: Error "record is out of upper bound"

2023-03-03 Thread Bruno Volpato via user
Hi Nick, This seems to come from utils/histogram.py . Any chance that you are initializing it in a way that defines bounds up to 60,000 but invoking record() with out of bounds value? Best, Bruno On Fr

Re: Error "record is out of upper bound"

2023-03-03 Thread Svetak Sundhar via user
Hi Nick-- Thanks for your email. Could you file a ticket with GCP Support, providing a job ID and project ID to look at? We have some tooling that can help us further investigate the issue. If not, we could maybe hop on a call and investigate the issue. Thanks, Svetak Sundhar Technical Solu

Error "record is out of upper bound"

2023-03-03 Thread Nick Edwards
Hello! I’m investigating a warning we’re receiving through Google Cloud’s Monitoring. The warning reads "record is out of upper bound 6: " and then includes a 5 or 6 digit integer. I’m having trouble finding any info on this sort of behavior. Any insight or assistance you can offer is greatly a