But... is it not like that in any other Java/Scala/Python/... app that uses
dependencies that also have their own dependencies?
If you want to provide a library, maybe you should give the user the option
to decide if they want an all-in-one ubber jar with shaded (more difficult
to debug) dependenc
> I may not need anything from spark but if I'll declare a dependency
in Jackson or guava with a different version than spark already use and
package- I might break things...
In that case I would recommend you to use assembly / assemblyShadeRules
for sbt-assembly or maven-shade-plugin for maven an
I think SHS only uses LevelDB/RocksDB to store intermediate data, supporting
re-parsing to rebuild the cache should be fine enough.
Also share my experience about using LevelDB/RocksDB for SHS, it seems LevelDB
has native memory leak issues, at least for the SHS use case, I need to reboot
the S
IMHO, it's probably dependent on how long the rewrite will take, from
reading the event log. If loading the state from LevelDB and rewriting to
RocksDB is quite much faster, then we may want to support this for a couple
minor releases to not force users to lose their cache. If there is no such
diff
