dosubot[bot] commented on issue #28445:
URL: https://github.com/apache/superset/issues/28445#issuecomment-2345083245
<!-- Answer -->
The `cache_config` for Superset is as follows:
```python
# Default cache for Superset objects
CACHE_CONFIG: CacheConfig = {
"CACHE_TYPE": "RedisCache",
"CACHE_DEFAULT_TIMEOUT": 300,
"CACHE_KEY_PREFIX": "superset_",
"CACHE_REDIS_HOST": os.getenv("REDIS_HOST", "redis"),
"CACHE_REDIS_PORT": os.getenv("REDIS_PORT", "6379"),
"CACHE_REDIS_DB": os.getenv("REDIS_RESULTS_DB", "1"),
}
# Cache for datasource metadata and query results
DATA_CACHE_CONFIG: CacheConfig = CACHE_CONFIG
# Cache for dashboard filter state. `CACHE_TYPE` defaults to
`SupersetMetastoreCache`
# that stores the values in the key-value table in the Superset metastore,
as it's
# required for Superset to operate correctly, but can be replaced by any
# `Flask-Caching` backend.
FILTER_STATE_CACHE_CONFIG: CacheConfig = {
"CACHE_TYPE": "SupersetMetastoreCache",
"CACHE_DEFAULT_TIMEOUT": int(timedelta(days=90).total_seconds()),
# Should the timeout be reset when retrieving a cached value?
"REFRESH_TIMEOUT_ON_RETRIEVAL": True,
# The following parameter only applies to `MetastoreCache`:
# How should entries be serialized/deserialized?
"CODEC": JsonKeyValueCodec(),
}
# Cache for explore form data state. `CACHE_TYPE` defaults to
`SupersetMetastoreCache`
# that stores the values in the key-value table in the Superset metastore,
as it's
# required for Superset to operate correctly, but can be replaced by any
# `Flask-Caching` backend.
EXPLORE_FORM_DATA_CACHE_CONFIG: CacheConfig = {
"CACHE_TYPE": "SupersetMetastoreCache",
"CACHE_DEFAULT_TIMEOUT": int(timedelta(days=7).total_seconds()),
# Should the timeout be reset when retrieving a cached value?
"REFRESH_TIMEOUT_ON_RETRIEVAL": True,
# The following parameter only applies to `MetastoreCache`:
# How should entries be serialized/deserialized?
"CODEC": JsonKeyValueCodec(),
}
```
Superset uses [Flask-Caching](https://flask-caching.readthedocs.io/) for
caching purposes. Flask-Caching supports various caching backends, including
Redis (recommended), Memcached, SimpleCache (in-memory), or the local
filesystem. Custom cache backends are also supported.
For example, to configure the filter state cache using Redis:
```python
FILTER_STATE_CACHE_CONFIG = {
'CACHE_TYPE': 'RedisCache',
'CACHE_DEFAULT_TIMEOUT': 86400,
'CACHE_KEY_PREFIX': 'superset_filter_cache',
'CACHE_REDIS_URL': 'redis://localhost:6379/0'
}
```
Dependencies for dedicated cache stores include:
- For Redis: the [redis](https://pypi.python.org/pypi/redis) Python package
- For Memcached: the [pylibmc](https://pypi.org/project/pylibmc/) client
library
If the required caches are undefined, Superset falls back to using a
built-in cache that stores data in the metadata database. For example, to use
the built-in cache to store chart data:
```python
DATA_CACHE_CONFIG = {
"CACHE_TYPE": "SupersetMetastoreCache",
"CACHE_KEY_PREFIX": "superset_results",
"CACHE_DEFAULT_TIMEOUT": 86400,
}
```
The cache timeout for charts can be overridden by settings for an individual
chart, dataset, or database. Setting the cache timeout to `-1` disables caching
for charting data.
Caching for SQL Lab query results is configured using `RESULTS_BACKEND` and
requires a cachelib object.
For caching thumbnails, enable the feature flag and configure a cache system
and celery workers. For example, to store images on S3:
```python
from flask import Flask
from s3cache.s3cache import S3Cache
class CeleryConfig(object):
broker_url = "redis://localhost:6379/0"
imports = (
"superset.sql_lab",
"superset.tasks.thumbnails",
)
result_backend = "redis://localhost:6379/0"
worker_prefetch_multiplier = 10
task_acks_late = True
CELERY_CONFIG = CeleryConfig
def init_thumbnail_cache(app: Flask) -> S3Cache:
return S3Cache("bucket_name", 'thumbs_cache/')
THUMBNAIL_CACHE_CONFIG = init_thumbnail_cache
THUMBNAIL_SELENIUM_USER = "Admin"
```
Using the above example, cache keys for dashboards will be
`superset_thumb__dashboard__{ID}`. You can override the base URL for selenium
using `WEBDRIVER_BASEURL`. Additional selenium web drive configuration can be
set using `WEBDRIVER_CONFIGURATION`.
The `CacheManager` class in `superset/utils/cache_manager.py` is responsible
for initializing and managing these caches. It ensures that the required caches
are properly configured and initialized, falling back to the built-in cache if
necessary. The `ExploreFormDataCache` class extends the `Cache` class to handle
specific logic for explore form data, such as renaming data keys and adding
default datasource types
<sup>[[1]](https://github.com/apache/superset/blob/master/superset/config.py)</sup><sup>[[2]](https://github.com/apache/superset/blob/master/docs/docs/configuration/cache.mdx)</sup><sup>[[3]](https://github.com/apache/superset/blob/master/superset/utils/cache_manager.py)</sup><sup>[[4]](https://github.com/apache/superset/blob/master/docker/pythonpath_dev/superset_config.py)</sup>.
<!-- Dosu Comment Footer -->
*To continue talking to [Dosu](https://dosu.dev), mention @dosu.*
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
