Re: [DISCUSS][FLIP-536] Add latest checkpoint details endpoint to Rest API

Hi Chesnay, thanks for your input, > That seems like something we'd maybe want to introduce consistently for all checkpoint-related endpoints. Do you believe we should scope it out of this FLIP and follow up with another FLIP, also I only see it might impact the /jobs/:jobid/checkpoints endpoint s

Re: [DISCUSS][FLIP-536] Add latest checkpoint details endpoint to Rest API

I think the idea of filtering is interesting but I do wonder if we should introduce it as part of this FLIP. That seems like something we'd maybe want to introduce consistently for all checkpoint-related endpoints. I'm also not sure about returning a 404 if no checkpoints exists (especially wi

Re: [DISCUSS][FLIP-536] Add latest checkpoint details endpoint to Rest API

Hi Poorvank yes the idea is to do the latest checkpoint Id lookup from the history and use it to return the checkpoint details. > Possible to consider adding type (savepoint/checkpoint) filtering: Since cache returns AbstractCheckpointStats yeah that's a good idea, I believe it might be useful

Re: [DISCUSS][FLIP-536] Add latest checkpoint details endpoint to Rest API

Hi Ahmed, Thank you for the FLIP. +1 (non-binding) for this feature. I have two implementation questions: 1. Approach for finding latest checkpoints: Since the FLIP mentions "utilizing existing CheckpointStatsCache,

Re: [DISCUSS][FLIP-536] Add latest checkpoint details endpoint to Rest API

Hi David, Thanks for the feedback, I guess an alternative approach would be adding paging and sorting to the checkpointing stats query, however this will still require 2 REST api calls to get the latest checkpoint details as the stats endpoint only gives a summary not the details, I am open to addi

Re: [DISCUSS][FLIP-536] Add latest checkpoint details endpoint to Rest API

Hi Ahmed, Thanks for submitting this Flip. What do you think of having /jobs/:jobid/checkpoints with query params to specify sorted criteria and direction and the number of returned elements (page size). This would appear to be more of a standard (and flexible) way of doing a search. To get the