GitHub user MorieeLM created a discussion: Async Queries redis connection error
I have hosted superset on kubernetes and keep getting these errors when trying
to generate charts on superset web; the user interface hangs with a display
message of "waiting for postgresql" and no charts are gemerated: kubectl logs
-n superset -l app=superset-worker --tail=20
Defaulted container "superset" out of: superset, wait-for-postgres-redis (init)
R = retval = fun(*args, kwargs)
File "/app/superset/initialization/init.py", line 114, in call
return task_base.call(self, *args, kwargs)
File "/app/.venv/lib/python3.10/site-packages/celery/app/trace.py", line 736,
in protected_call
return self.run(args, **kwargs)
File "/app/superset/tasks/async_queries.py", line 110, in
load_chart_data_into_cache
async_query_manager.update_job(
File "/app/superset/async_events/async_query_manager.py", line 306, in
update_job
self._cache.xadd(scoped_stream_name, event_data, "", self._stream_limit)
File "/app/superset/async_events/cache_backend.py", line 71, in xadd
return self._cache.xadd(stream_name, event_data, event_id, maxlen)
File "/app/.venv/lib/python3.10/site-packages/redis/commands/core.py", line
3534, in xadd
return self.execute_command("XADD", name, *pieces)
File "/app/.venv/lib/python3.10/site-packages/redis/client.py", line 1266, in
execute_command
conn = self.connection or pool.get_connection(command_name, **options)
File "/app/.venv/lib/python3.10/site-packages/redis/connection.py", line
1461, in get_connection
connection.connect()
File "/app/.venv/lib/python3.10/site-packages/redis/connection.py", line 713,
in connect
raise ConnectionError(self._error_message(e))
redis.exceptions.ConnectionError: Error 111 connecting to localhost:6379.
Connection refused.
please assist with a workaround for this.
I have configOverrides defined as follows: # Proxy and feature flags
ENABLE_PROXY_FIX = True
FEATURE_FLAGS = {
"DASHBOARD_NATIVE_FILTERS": True,
"DASHBOARD_CROSS_FILTERS": True,
"GLOBAL_ASYNC_QUERIES": True,
}
# Celery configuration (don't override the default one, just update it)
CELERY_CONFIG = {
'broker_url': f'redis://{REDIS_HOST}:{REDIS_PORT}/{REDIS_CELERY_DB}',
'result_backend':
f'redis://{REDIS_HOST}:{REDIS_PORT}/{REDIS_CELERY_DB}',
'imports': [
'superset.sql_lab',
'superset.tasks.scheduler',
],
'worker_prefetch_multiplier': 10,
'task_acks_late': True,
'task_annotations': {
'sql_lab.get_sql_results': {
'rate_limit': '100/s',
},
}
}
# Cache configuration
CACHE_CONFIG = {
'CACHE_TYPE': 'RedisCache',
'CACHE_DEFAULT_TIMEOUT': 300,
'CACHE_KEY_PREFIX': 'superset_',
'CACHE_REDIS_HOST': REDIS_HOST,
'CACHE_REDIS_PORT': REDIS_PORT,
'CACHE_REDIS_DB': REDIS_DB,
'CACHE_REDIS_URL': f'redis://{REDIS_HOST}:{REDIS_PORT}/{REDIS_DB}',
}
RESULTS_BACKEND = RedisCache(
host=REDIS_HOST,
port=REDIS_PORT,
db=REDIS_DB,
key_prefix='superset_results'
)
# Global async queries configuration
GLOBAL_ASYNC_QUERIES_JWT_SECRET =
os.environ.get('GLOBAL_ASYNC_QUERIES_JWT_SECRET')
GLOBAL_ASYNC_QUERIES_REDIS_CONFIG = {
'host': REDIS_HOST,
'port': REDIS_PORT,
'db': REDIS_DB,
}
GLOBAL_ASYNC_QUERIES_TRANSPORT = 'redis'
GLOBAL_ASYNC_QUERIES_REDIS_STREAM_PREFIX = 'async-events-'
GLOBAL_ASYNC_QUERIES_REDIS_STREAM_LIMIT = 1000
GLOBAL_ASYNC_QUERIES_REDIS_STREAM_LIMIT_FIREHOSE = 100
GitHub link: https://github.com/apache/superset/discussions/34650
----
This is an automatically sent email for [email protected].
To unsubscribe, please send an email to:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
