Thanks for the response. Lots of food for thought. I was using Teradata as just one example. We're also supporting Snowflake, Vertica, Netezza, and others that escape me at the moment. I think there's one or two that's a real stretch but someone managed to shoehorn it into our framework. No Access yet, thank God, although I wouldn't be surprised if someone asked to be able to open an Access file located on a Google drive or Dropbox folder.
The point about code breaking on new releases is valid but we're enterprise software and don't update our libraries. For instance we just moved off the bundled PostgreSQL 8.4 driver and only did so because we need to support PostgreSQL 10 servers and that driver is simply too old. Now we're up to the 2-year-old 9.4.1207 since 9.4.1208 breaks someone's ill-advised workaround and fixing that will introduce customer-facing changes. That's an extreme example since we also support Redshift and the latter's driver also registers itself as a PostgreSQL driver and that was causing problems. That was our choice but we're sometimes driven by the oldest version of software still used by a customer. (*cough* Hadoop *cough*) What that means here is that our horizon is probably a few years out. I agree that having to update our own patches every few months may not be worth the effort but if the math changes if we don't have to worry about a merge for several years. By then it would be a fresh effort anyway. We're only using the basic functionality of jOOQ so it's not as painful as if we were cutting edge. (*) Hadoop is weird since we're focused to use the libraries installed on the system instead of our bundled libraries. I know about the <optional> tag in maven dependencies and that we can use it to write software that uses newer functionality but gracefully falls back on an older library where the method or class doesn't exist but, so far, the pain of staying at 2.7 is less than the pain of adding the add'l checks to support the newer libraries if they're available. Bear On Mon, Mar 19, 2018 at 3:02 AM, Lukas Eder <[email protected]> wrote: > Hi Bear, > > Thank you very much for your message and for taking the time to review > your jOOQ usage. I'm sure there's always room for improving the integration > - jOOQ, over the past 10 years, has solved many niche problems. But not > everyone is aware of the possibility of using jOOQ in such a situation. I > will further comment inline: > > 2018-03-18 20:49 GMT+01:00 The bear in Boulder <[email protected]>: > >> One of the two issues is adding support for additional databases. In some >> cases (on the blog) you've said that they're not economically feasible >> since there's so few potential customers (e.g., Teradata), >> > > Just to be sure, you do mean Teradata, not Teradata Aster > > I'm not sure if the referred claim still true in the case of Teradata. At > some point, I think that Teradata simply didn't offer a reasonable price > plan for developers. We cannot afford a USD 30,000 / year subscription of a > database only to run some integration tests. But today, their website > offers a free tier for developers with non-production usage, akin to > Oracle's OTN license: > https://www.teradata.com/products-and-services/ > database-deployment-license-options/ > > It's certainly not unreasonable to re-think a future integration of > Teradata given that an analytics database is the perfect match for jOOQ: > https://github.com/jOOQ/jOOQ/issues/629 > > >> in other cases you've said that they're too far from the SQL standards to >> be a good fit (Hive). >> > > Yes indeed... Not only are they far from the SQL standards, but there are > simply too many of these SQL-ish databases. The expectation from formal > jOOQ support is that we integration tests such an integration and that > doesn't really scale for us. > > >> In our case we're only using a handful of methods so it's not hard for us >> to start with DEFAULT and just tweak the bits we need. >> > > That's probably true in the case of Hive, which doesn't offer much SQL > functionality. Not sure about Teradata. > > >> However there's no way to register them with jOOQ - we have to use >> DEFAULT and make the tweaks in various, often ugly, ways. Besides being >> ugly and hard to maintain since one class may have tweaks for multiple >> unsupported databases) it's harder to test since we're working at a much >> higher abstraction layer than Converter, DataType, and Binding. >> > > Indeed, there had been discussions in the past about externalising feature > flags from jOOQ's internals. E.g. we could specify a flag that indicates > whether a database supports > > - select statements without from clause > - nested set operations > - the values() clause > - multi row values() (e.g. in insert) > - derived column lists > - etc. > > And if the feature is not supported, there could be additional flags > indicating how it should be emulated. Some ideas about how emulations work > in jOOQ in this blog post: > https://blog.jooq.org/2018/03/13/top-10-sql-dialect- > emulations-implemented-in-jooq/ > > We'll not implement this for the simple reason that such feature flags > will become public API and will need to be maintained backwards-compatibly > as such, which will be cause a lot of pain moving jOOQ forward. Just > recently, I've implemented a fix that improved the existing nested set > operation emulation (e.g. the possibility of writing things like (((select > 1) union all (select 2)) except (select 3)). That change required quite a > bit of internal refactoring and broke quite a bit of internal API. It was > difficult enough to get right and not create any regressions as jOOQ is > designed today. It would have been impossible if public feature flags had > to be maintained backwards compatibly as well. > > >> (It goes without saying that what I'm about to suggest requires a decent >> understanding of the jOOQ internals and things may break in subsequent >> releases. That's a trade-off between whether it will be cheaper for us to >> track the jOOQ changes and make appropriate changes or for us to continue >> doing all of this externally.) >> > > Oh yes, and there *will* be breaking, at least when upgrading to the next > minor release :-) > > Is it possible to add a few USER_DEFINED entries in SQLDialect? It would >> be us to remember that USER_DEFINED means TERADATA. This would leave slots >> in the caches in DefaultDataType that should go a long way towards >> providing the necessary hooks we need in order to implement our own >> databases. The USER_DEFINED entries would default to DEFAULT. >> > > I'm assuming you mean for the official jOOQ build to add such USER_DEFINED > entries. No, that doesn't make sense. We do not want to officially > encourage this kind of patching. In addition to not wanting to encourage > it, we certainly do not want to maintain it. Anything that is published in > an API like jOOQ can reasonably be expected to be maintained backwards > compatibly. > > Sure, right now, it might scratch your immediate itch, and you won't mind > if we slightly break things 3 years down the line. But your peers and > successors certainly will. If they have to maintain any solution based on > such a USER_DEFINED hack, they will not forgive you, nor us :-) > > Besides, the reason why you suggested it is because you browsed the jOOQ > internals and found a Map[n] type where n = SQLDialect.values().length. You > concluded that by adding a new SQLDialect, you could patch this map to > contain more entries, which you could then fill with your own data. It is > really never a good idea to leak such internals into an API, even if it is > very clearly documented as a hack. > > So, let's immediately forget about this idea. > > >> The alternative is to simply fork the code, add our own entries to >> SQLDialect, and go on from there. >> > > That's always an option, although rest assured, you're very unlikely to > upgrade your version of jOOQ if your fork starts to deviate substantially > from the official jOOQ build. > > >> We may have to do that anyway for other reasons >> > > What are those reasons? Anything that could be added to jOOQ, out of the > box? > > Cheers, > Lukas > > -- > You received this message because you are subscribed to a topic in the > Google Groups "jOOQ User Group" group. > To unsubscribe from this topic, visit https://groups.google.com/d/ > topic/jooq-user/qa8VMSIscAc/unsubscribe. > To unsubscribe from this group and all its topics, send an email to > [email protected]. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "jOOQ User Group" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
