cberge908 commented on issue #30844: URL: https://github.com/apache/superset/issues/30844#issuecomment-2462315583
> The issue you're experiencing with the SQLite database not updating in Superset when the underlying file is changed externally is likely related to how file locks and caching are handled. Superset, when using SQLite, might lock the file or cache the data, preventing it from reflecting updates made to the file externally. > > Here are a few suggestions to address this issue: > > 1. **File Locking**: Ensure that the file locking mechanism is not causing the issue. SQLite can have problems with file locks, especially when used over network filesystems like NFS or Blobfuse2. Consider using a different database backend that is more suitable for concurrent access and updates, such as PostgreSQL or MySQL. This actually helped - we've now changed our process slightly. We now export the flat table into a csv on the blob. We're then ingesting this periodically into a table in another DB, which is already connected to our Superset instance (Clickhouse). With this we were able to avoid using SQLite with Superset and also the use of Blobfuse2. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
