Hi,
From experience working on two storage engines on MySQL and being main
developer for the second one, and one "storage engine" on big company, I
would say MySQL storage engine API has some legacy problems and clearly
some missing functionality. e.g.
- Some part of the storage engines are always the same, parameters are
not fully given to functions or you really can't return proper error
messages
- There is interface to get e.g. row type, but still you need to hack
handler.cc/handler.h because some values are hard coded there (show
create table)
- Storage engine API clearly shows that transaction concept has being
introduced after the first API version was designed, still you can't
really return errors on commit operation
- Storage engine can announce some that it can do some functionality to
upper layer, but this is very limited. Storage engines should have more
control what is done on MySQL parser/optimiser level and what is done on
storage engine layer. E.g. for columnar storage engine, I would love to
have opportunity to inform MySQL that I can do join's myself with all
conditions pushed down to attributes, but that I can't do e.g. external
sorting.
- For in-memory storage engine there should be API where MySQL data type
and in-memory data types are exactly the same to avoid conversions,
similarly in-memory buffer pool should contain directly usable types for
MySQL.
Thus, as a conclusion, we really would need a better and cleaner API
where storage engine has more control.
R: Jan
The storage engine API has never been my favorite part of MySQL. I
haven't written a storage engine but I used to maintain a custom
storage engine and migrated that from 4.0 to 5.0 so I had some
experience with the API.
Is there a different API that would make it easier for innovation to
come to MySQL/MariaDB? For example there is a lot going on in the
write-optimized database space. Until recently we had nothing for
MySQL open-source. Now we have open source TokuDB (great news). I
haven't read their source but I have been told they use something that
was derived from the BerkeleyDB API.
Would the BerkeleyDB API make it easier to get new storage engines
into MySQL? For example, we don't have an engine optimized for
in-memory workloads. And while InnoDB will have great cache hit rates
when all data is in-memory. It might write much more data to disk than
required.
Do we need a better API or is the current code good enough? I hope to
hear from people who have done serious storage engine work.
--
Mark Callaghan
mdcal...@gmail.com <mailto:mdcal...@gmail.com>
_______________________________________________
Mailing list: https://launchpad.net/~maria-developers
Post to : maria-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~maria-developers
More help : https://help.launchpad.net/ListHelp
_______________________________________________
Mailing list: https://launchpad.net/~maria-developers
Post to : maria-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~maria-developers
More help : https://help.launchpad.net/ListHelp
