wenjin272 commented on code in PR #1091:
URL: https://github.com/apache/flink-agents/pull/1091#discussion_r4003692697
##########
runtime/src/main/java/org/apache/flink/agents/runtime/skill/repository/SkillMaterializer.java:
##########
@@ -58,6 +61,27 @@ public final class SkillMaterializer {
private static final int JAR_URL_PREFIX_LEN = "jar:".length();
+ // --- Size caps for download and extraction (issue #1072) ---
+
+ /** Maximum number of bytes accepted from a single HTTP download. */
+ public static final long MAX_DOWNLOAD_BYTES = 512L * 1024 * 1024; // 512
MiB
+
+ /**
+ * Maximum uncompressed size of any single entry during zip extraction.
Declared sizes in the
+ * zip central directory are attacker-controlled; this cap is enforced
against actual bytes
+ * written, not {@link ZipEntry#getSize()}.
+ */
+ public static final long MAX_EXTRACT_ENTRY_BYTES = 200L * 1024 * 1024; //
200 MiB
+
+ /**
+ * Maximum cumulative uncompressed bytes written across all entries during
a single zip
+ * extraction. Enforced against actual bytes written.
+ */
+ public static final long MAX_EXTRACT_TOTAL_BYTES = 1024L * 1024 * 1024; //
1 GiB
+
+ /** Maximum number of entries permitted in a single zip archive. */
+ public static final int MAX_EXTRACT_ENTRIES = 10_000;
Review Comment:
These limits are currently hard-coded, but realistic skill archive sizes and
TaskManager temporary-disk budgets can vary significantly between deployments.
Could we expose them through a single configuration object, with aligned Java,
Python, and YAML semantics? The values should remain positive and should not
support an unlimited setting.\n\nThe proposed defaults also seem quite
permissive for a skill archive: a 512 MiB download, 200 MiB single entry, 1 GiB
extraction, or 10,000 entries can still consume substantial disk space and
inodes, especially when multiple skills are materialized concurrently. Unless
we have usage data supporting these values, could we start with more
conservative defaults—for example, 64 MiB download, 64 MiB per entry, 256 MiB
total extraction, and 1,000 entries—while allowing deployments to raise them
explicitly?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
