wenjin272 commented on code in PR #1091:
URL: https://github.com/apache/flink-agents/pull/1091#discussion_r4003692697
##########
runtime/src/main/java/org/apache/flink/agents/runtime/skill/repository/SkillMaterializer.java:
##########
@@ -58,6 +61,27 @@ public final class SkillMaterializer {
private static final int JAR_URL_PREFIX_LEN = "jar:".length();
+ // --- Size caps for download and extraction (issue #1072) ---
+
+ /** Maximum number of bytes accepted from a single HTTP download. */
+ public static final long MAX_DOWNLOAD_BYTES = 512L * 1024 * 1024; // 512
MiB
+
+ /**
+ * Maximum uncompressed size of any single entry during zip extraction.
Declared sizes in the
+ * zip central directory are attacker-controlled; this cap is enforced
against actual bytes
+ * written, not {@link ZipEntry#getSize()}.
+ */
+ public static final long MAX_EXTRACT_ENTRY_BYTES = 200L * 1024 * 1024; //
200 MiB
+
+ /**
+ * Maximum cumulative uncompressed bytes written across all entries during
a single zip
+ * extraction. Enforced against actual bytes written.
+ */
+ public static final long MAX_EXTRACT_TOTAL_BYTES = 1024L * 1024 * 1024; //
1 GiB
+
+ /** Maximum number of entries permitted in a single zip archive. */
+ public static final int MAX_EXTRACT_ENTRIES = 10_000;
Review Comment:
These limits are currently hard-coded, but realistic skill archive sizes and
TaskManager temporary-disk budgets can vary significantly between deployments.
Could we expose them through a single configuration object, with aligned Java,
Python, and YAML semantics? The values should remain positive and should not
support an unlimited setting.
The proposed defaults also seem quite permissive for a skill archive: a 512
MiB download, 200 MiB single entry, 1 GiB extraction, or 10,000 entries can
still consume substantial disk space and inodes, especially when multiple
skills are materialized concurrently. Unless we have usage data supporting
these values, could we start with more conservative defaults—for example, 64
MiB download, 64 MiB per entry, 256 MiB total extraction, and 1,000
entries—while allowing deployments to raise them explicitly?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
