GitHub user dosubot[bot] added a comment to the discussion: Failure on creation of Snowflake dataset due to staging S3 download errors (works in SQL Lab, regression with new connector)
<!-- Greeting --> Hi @oliverbell-klv! I'm [Dosu](https://go.dosu.dev/dosubot) and Iām helping the superset team. <!-- Answer --> This is a known challenge with Superset's Snowflake integration: metadata reflection and dataset creation use SQLAlchemy's inspector methods (like `get_columns`), which can trigger result sets that Snowflake stages in S3. If your infrastructure intermittently fails to access the staging S3 bucket, these operations will fail, even if SQL Lab queries work (since they often return small results directly and avoid S3 staging) [source](https://github.com/apache/superset/issues/33937). Superset does not have built-in retry logic or self-healing for Snowflake-specific network failures. Mitigation relies on tuning pool and timeout settings in the connector's configuration or your infrastructure setup. You can adjust these via the database's "Extra" field (`engine_params` and `connect_args`). Also, enabling the `CLIENT_SESSION_KEEP_ALIVE` parameter in Snowflake may improve connection stability [source](https://github.com/apache/superset/issues/33937). If the issue persists, review your AWS/container networking and resource limits, and enable detailed logging (`LOG_LEVEL = logging.DEBUG`) to capture more diagnostic info. There are no Superset-specific config flags or version pins that directly address S3 staging download errors in metadata reflection. This is primarily an infrastructure/networking issue surfaced by how Superset interacts with the Snowflake connector for metadata queries. <!-- Dosu Comment Footer --> *To reply, just mention [@dosu](https://go.dosu.dev/dosubot).* --- <sup>How did I do? [Good](https://app.dosu.dev/response-feedback/373a262b-1f87-43f5-a960-d7e96f49ddd9?feedback_type=great_response) | [Irrelevant](https://app.dosu.dev/response-feedback/373a262b-1f87-43f5-a960-d7e96f49ddd9?feedback_type=irrelevant_answer) | [Incorrect](https://app.dosu.dev/response-feedback/373a262b-1f87-43f5-a960-d7e96f49ddd9?feedback_type=incorrect_sources) | [Verbose](https://app.dosu.dev/response-feedback/373a262b-1f87-43f5-a960-d7e96f49ddd9?feedback_type=too_verbose) | [Hallucination](https://app.dosu.dev/response-feedback/373a262b-1f87-43f5-a960-d7e96f49ddd9?feedback_type=hallucination) | [Report š](https://app.dosu.dev/response-feedback/373a262b-1f87-43f5-a960-d7e96f49ddd9?feedback_type=bug_report) | [Other](https://app.dosu.dev/response-feedback/373a262b-1f87-43f5-a960-d7e96f49ddd9?feedback_type=other)</sup> [](https://app.dosu.dev/a28d3c7e-a9d3-459e-9fb6-3a6f9ff4f357/ask?utm_source=github)& nbsp;[](https://go.dosu.dev/discord-bot) [](https://twitter.com/intent/tweet?text=%40dosu_ai%20helped%20me%20solve%20this%20issue!&url=https%3A//github.com/apache/superset/issues/34736) GitHub link: https://github.com/apache/superset/discussions/34778#discussioncomment-14169369 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
