jerryshao commented on code in PR #13468:
URL: https://github.com/apache/gravitino/pull/13468#discussion_r4128720521
##########
core/src/main/java/org/apache/gravitino/job/JobManager.java:
##########
@@ -508,6 +501,10 @@ public JobEntity runJob(String metalake, String
jobTemplateName, Map<String, Str
// Check if the job template exists, will throw NoSuchJobTemplateException
if it does not exist.
JobTemplateEntity jobTemplateEntity = getJobTemplate(metalake,
jobTemplateName);
+ // Reject a job with missing parameters before creating anything for it.
+ JobTemplateResolver jobTemplateResolver = new
JobTemplateResolver(jobTemplateEntity);
+ jobTemplateResolver.checkJobConf(jobConf);
Review Comment:
Good catch, thanks — `builtin-iceberg-remove-orphan-files` now uses
`{{older_than:-}}`, `{{location:-}}`, `{{dry_run:-false}}` and
`{{spark_conf:-}}`, and is bumped to v2. I did the same for
`builtin-iceberg-rewrite-manifests`, which had the same problem.
On the test:
`TestBuiltInJobTemplateProvider#testTemplatesOnlyRequireTheParametersTheyHaveTo`
now pins the required parameters of every built-in template (the catalog
connection keys everywhere, plus `where_clause` for rewrite-data-files, and
none for SparkPi), so a template that keeps a bare placeholder for an optional
argument fails the build. It reads them through a new
`JobTemplateResolver#requiredParameters()`. Two job tests modelled the
substitution themselves and missed default values, so that model moved into a
shared `TemplateArguments` test helper. The docs table lists both jobs now.
_🤖 Addressed by [Claude Code](https://claude.com/claude-code)_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
