2020-11-10 01:03:33 UTC - Joshua Dunham: Hey Everyone, is it possible to use kafka as the storage (instead of CouchDB)? https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1604970213258100?thread_ts=1604970213.258100&cid=C3TPCAQG1 ---- 2020-11-10 01:09:09 UTC - Rodric Rabbah: Storage in what sense? https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1604970549258500?thread_ts=1604970213.258100&cid=C3TPCAQG1 ---- 2020-11-10 01:09:30 UTC - Joshua Dunham: users/perms and activations https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1604970570258700?thread_ts=1604970213.258100&cid=C3TPCAQG1 ---- 2020-11-10 01:09:49 UTC - Joshua Dunham: I don't have a handle on all the tables in couch yet. https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1604970589258900?thread_ts=1604970213.258100&cid=C3TPCAQG1 ---- 2020-11-10 01:10:14 UTC - Joshua Dunham: But I'm currently trying to migrate OW kafka to Pulsar and the kafka proto support. https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1604970614259100?thread_ts=1604970213.258100&cid=C3TPCAQG1 ---- 2020-11-10 01:10:54 UTC - Joshua Dunham: It struck me that Pulsar would be a good choice for primary storage through it's apache bookkeeper service. https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1604970654259300?thread_ts=1604970213.258100&cid=C3TPCAQG1 ---- 2020-11-10 01:29:34 UTC - Rodric Rabbah: activations grow quickly as they’re meta data, results and logs associated with every function run https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1604971774259600?thread_ts=1604970213.258100&cid=C3TPCAQG1 ---- 2020-11-10 01:30:11 UTC - Rodric Rabbah: there is a kafka topic for activations but it’s meant as a buffer that’s drained into a more permanent store, like elastic so that logs and activations may be indexed and dashboards created https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1604971811259800?thread_ts=1604970213.258100&cid=C3TPCAQG1 ---- 2020-11-10 01:30:24 UTC - Rodric Rabbah: i don’t know enough about pulsar to comment on suitability here https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1604971824260000?thread_ts=1604970213.258100&cid=C3TPCAQG1 ---- 2020-11-10 01:30:53 UTC - Rodric Rabbah: there is an abstraction in OpenWhisk to plug in different data stores, for users, and whisk entities like actios https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1604971853260200?thread_ts=1604970213.258100&cid=C3TPCAQG1 ---- 2020-11-10 01:31:09 UTC - Rodric Rabbah: one would have to implement a new interface to change the backing store https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1604971869260400?thread_ts=1604970213.258100&cid=C3TPCAQG1 ---- 2020-11-10 01:31:24 UTC - Rodric Rabbah: there’s an open pr that does this for mongo db as an example https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1604971884260600?thread_ts=1604970213.258100&cid=C3TPCAQG1 ---- 2020-11-10 01:32:43 UTC - Rodric Rabbah: thank you @Dominic Kim for leading this release :clap: :tada: tada : Dominic Kim, seonghyun, chetanm, Brendan Doyle congapartyparrot : seonghyun, chetanm thumbsup_all : Joshua Dunham https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1604971963261100?thread_ts=1604971963.261100&cid=C3TPCAQG1 ---- 2020-11-10 01:33:05 UTC - Dominic Kim: Thank you :slightly_smiling_face: https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1604971985261400?thread_ts=1604971963.261100&cid=C3TPCAQG1 ---- 2020-11-10 01:37:19 UTC - Joshua Dunham: Cool https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1604972239262000?thread_ts=1604970213.258100&cid=C3TPCAQG1 ---- 2020-11-10 01:38:17 UTC - Joshua Dunham: bookKeeper is more than adequate. It's part Presto part Druid. Since Pulsar speaks Kafka it would be slick for it to pull double duty for longterm store AND be a feed. https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1604972297262200?thread_ts=1604970213.258100&cid=C3TPCAQG1 ---- 2020-11-10 01:38:45 UTC - Joshua Dunham: I'll ask over in that channel what the devs think. +1 : Rodric Rabbah https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1604972325262400?thread_ts=1604970213.258100&cid=C3TPCAQG1 ---- 2020-11-10 02:14:03 UTC - Joshua Dunham: Is the invoker the only component that utilizes couch (besides CLI clients?) https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1604974443262700?thread_ts=1604970213.258100&cid=C3TPCAQG1 ---- 2020-11-10 03:26:07 UTC - Rodric Rabbah: Controller also https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1604978767263100?thread_ts=1604970213.258100&cid=C3TPCAQG1 ---- 2020-11-10 23:41:33 UTC - Brendan Doyle: I would be concerned with putting the actual actions through kafka to store. As far as I'm aware, you shouldn't be using kafka for like 50mb in a message https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1605051693263600?thread_ts=1604970213.258100&cid=C3TPCAQG1 ----
