Re: Newbie iceberg questions

2021-01-28 Thread Ryan Blue
Replies inlin. On Thu, Jan 28, 2021 at 9:42 AM kkishore iiith wrote: > Hello Community, > > I am solving the problem of handling late arrived data in one of our > systems. Currently, we wait for 8 hours for the late data to arrive before > starting processing the current hour data. > > We have t

Newbie iceberg questions

2021-01-28 Thread kkishore iiith
Hello Community, I am solving the problem of handling late arrived data in one of our systems. Currently, we wait for 8 hours for the late data to arrive before starting processing the current hour data. We have three stages in our pipeline A -> B -> C where B waits for 8 hours for A's hourly dat