William thank you for your feedback! I think I'll dive into the WAN connection between clusters. There are a lot of considerations to make like fragmentation, intermittent connectivity, maybe leverage the circuit breaker pattern. A reasonable amount of discussion and research is needed, but I'll try specing this out and sharing in the mailing list before moving forward.
Product examples however seems like a great started (at least for me), and likely a great buildup towards the item above. Do you have product examples already in mind? The Jira issue does not specify any. A couple examples come to mind, but likely belong on another thread. I'll catch up on the procedures in the mean time. Joao On Tue, Jul 28, 2015 at 3:06 PM Gregory Chase <gch...@pivotal.io> wrote: > +1 > > On Tue, Jul 28, 2015 at 2:59 PM, Dan Smith <dsm...@pivotal.io> wrote: > > > I think Williams ideas and those listed on "How to Contribute" page are a > > good place to start. > > > > I'd also like to suggest that we start tagging bugs with "starter" that > > might be good bugs for someone relatively new to the geode code base. I > see > > several other apache projects doing something similar. > > > > Cassandra has a lhc (low hanging fruit) label: > > http://wiki.apache.org/cassandra/HowToContribute > > > > Spark has a starter label: > > > > > https://cwiki.apache.org/confluence/display/SPARK/Contributing+to+Spark#ContributingtoSpark-PreparingtoContributeCodeChanges > > > > -Dan > > > > > > On Tue, Jul 28, 2015 at 11:37 AM, arghya sadhu <arghy...@gmail.com> > wrote: > > > > > Hi William, > > > > > > Unfortunately I have no knowledge of apache mesos/marathon ,but I am > > > interested to know about it and to spend time on that. > > > > > > Thanks, > > > Arghya > > > On Jul 28, 2015 10:42 PM, "William Markito" <wmark...@pivotal.io> > wrote: > > > > > > > Replying the last 3 messages: > > > > > > > > @Arghya: That's great. Are you already familiar with > mesos/marathon, > > > > etc? If not that would be > > > > a good starting point before moving fwd with implementation for > Geode. > > > > > > > > @Eric: That's right. Although please see my notes on this thread... > > > > > > > > @Christian: Thanks for creating that. What do you mean by non-OQL ? > > > > > > > > > > > > So talking specifically about Zeppelin and Geode, technically it's > > > already > > > > possible today > > > > using the Geode spark connector. I'm going to blog about it soon, but > > in > > > a > > > > nutshell > > > > the connector allows you to execute OQL within a spark/sql context > and > > > > register them as RDDs: > > > > > > > > For example: > > > > > > > > %spark > > > > val deviceLocations = sqlContext.gemfireOQL("select d.x, d.y, > > d.deviceId > > > > from /DeviceLocations d") > > > > deviceLocations.registerTempTable("deviceLocations") > > > > > > > > Then allows you to execute spark SQL such as in this image: > > > > http://img42.com/En7hO > > > > > > > > That said, OQL is a more direct access to Geode and both access would > > > have > > > > different use cases... > > > > > > > > > > > > On Tue, Jul 28, 2015 at 6:46 AM, Christian Tzolov < > ctzo...@pivotal.io> > > > > wrote: > > > > > > > > > Glad i came across this discussion because i've oppened a ticket to > > add > > > > > Geode OQL Interpreter for Zeppelin: > > > > > https://issues.apache.org/jira/browse/ZEPPELIN-189 > > > > > > > > > > I'm working on PostgreSQL/HAWQ/GPDB Interpreter [ZEPPELIN-70] and i > > > have > > > > > been exploring a Geode Interpreter support as well. > > > > > > > > > > Any ideas for non-OQL Interpreters use cases/support? > > > > > > > > > > Cheers, > > > > > Christian > > > > > > > > > > > > > > > > > > > > > > > > > On 28 July 2015 at 14:04, Eric Pederson <eric...@gmail.com> wrote: > > > > > > > > > > > Hi William - on the Zeppelin thing, do you mean a OQL > > > > runner/visualizer a > > > > > > la Databrowser? That would be cool. > > > > > > > > > > > > > > > > > > -- Eric > > > > > > > > > > > > On Tue, Jul 28, 2015 at 1:08 AM, William Markito < > > > wmark...@pivotal.io> > > > > > > wrote: > > > > > > > > > > > > > Hello João, > > > > > > > > > > > > > > Let me try to shed some light here... > > > > > > > > > > > > > > On Mon, Jul 27, 2015 at 8:42 PM, João Peixoto < > > > > joao.harti...@gmail.com > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > Dear Geode devs, > > > > > > > > > > > > > > > > I've been following (and to some extent participating) the > > > mailing > > > > > > lists > > > > > > > > and I find the project really interesting. > > > > > > > > > > > > > > > > > > > > > > And we're happy with that! :) > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I would like to contribute but I feel a little lost. I know > > that > > > > > there > > > > > > > is A > > > > > > > > LOT going on right now with the donation to Apache and all. > > > > > > > > > > > > > > > > I've been looking into a couple issues that I could tackle to > > > begin > > > > > > with: > > > > > > > > * https://issues.apache.org/jira/browse/GEODE-36 : Changing > > the > > > > > shell > > > > > > to > > > > > > > > "geode". Seems like an easy enough task, since you seem to > use > > > > Spring > > > > > > > > Shell. After looking at the code it is not that simple. You > use > > > > > Spring > > > > > > > > Shell 1.0.0 and JLine (older version). Spring Shell 1.1.0 is > > not > > > > > > > compatible > > > > > > > > and would require a lot of changes, but I'm not sure of the > > > current > > > > > > > > implementation decisions nor who to ask about it. > > > > > > > > > > > > > > > > > > > > > > I'd hold a little bit on the rename to Geode for now but the > > > > > > upgrade/update > > > > > > > to new version of the Spring shell would be an awesome > > > contribution, > > > > > but > > > > > > > not sure if that's the easiest way to start... > > > > > > > > > > > > > > > > > > > > > > > > > > > > > * https://issues.apache.org/jira/browse/GEODE-34 : > > Experimenting > > > > > with > > > > > > > > Reactor/Reactive Streams. Much more "researchy" and for all > > > intents > > > > > and > > > > > > > > purposes, "hardcore". Specially not knowing the internal > > > structure > > > > > > (apart > > > > > > > > from studying the code) nor who to ask about it, like an > > > overview. > > > > > > > > Definitely sounds fun though. > > > > > > > > > > > > > > > > > > > > > > Also agree. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On this subject I'd like to ask your advice. Should I wait > for > > > > Geode > > > > > to > > > > > > > be > > > > > > > > out of incubating status before diving into the project > itself? > > > is > > > > > > there > > > > > > > > some sort of implementation decisions notebook or general > > > internal > > > > > > > > architecture diagram/document? Or should I just power through > > it? > > > > > > > > > > > > > > > > > > > > > > > *Not at all!* In fact getting contributions while we're > > incubating > > > > are > > > > > > very > > > > > > > important and a key metric towards our graduation. > > > > > > > > > > > > > > So first of all, at the How to contribute > > > > > > > < > > > https://cwiki.apache.org/confluence/display/GEODE/How+to+Contribute > > > > > > > > > > > page > > > > > > > we listed a few ideas that would be interesting and some are > > > actually > > > > > > > fairly simple ways to contribute without that much knowledge of > > > Geode > > > > > > > internals. But other ideas not listed there are also welcome! > > > > > > > > > > > > > > Giving some other examples: > > > > > > > > > > > > > > - Geode has an huge number of features and when we decided > to > > > open > > > > > > > source GemFire we *left out* of the package the product > > examples > > > > on > > > > > > > purpose, given that they would need to be updated and > > > refactored. > > > > > That > > > > > > > work > > > > > > > I've not started yet and would be tracked in GEODE-33 > > > > > > > <https://issues.apache.org/jira/browse/GEODE-33>. > > > > > > > - Currently I'm looking into Apache Mesos and Marathon > > > integration > > > > > or > > > > > > at > > > > > > > least a tutorial on how to run a Geode cluster on Mesos. We > > > could > > > > > > work > > > > > > > together if you think that's interesting. GEODE-49 > > > > > > > <https://issues.apache.org/jira/browse/GEODE-49> > > > > > > > - Creating a Apache Zeppelin interpreter should also an > > > > interesting > > > > > > > add-on to the project. > > > > > > > - A WAN integration (integrating Geode clusters > geographically > > > > > > > separated) mechanism using Apache ActiveMQ or Apollo would > be > > an > > > > > > > interesting and yet challenging feature. Problems like > > keeping > > > > > > systems > > > > > > > in > > > > > > > sync, taking care of consistency, etc.. > > > > > > > - Provide a flag/setting on Geode that would enable exposing > > JMX > > > > > over > > > > > > > REST through something like Jolokia, considering security, > > > etc... > > > > > > > - Provide extra functionalities in GFSH adding functions or > > > > > integrated > > > > > > > security through OAUTH. > > > > > > > - Enhance our backup/restore experience and procedures. > Here > > > for > > > > > > > example one could leverage JClouds, Spring Cloud or some > other > > > > > "cloud" > > > > > > > API > > > > > > > and provide integration with AWS Glacier > > > > > > > <http://aws.amazon.com/glacier/> (or > > > > > > > Azure Backup) doing backup/restore on a cheap cloud storage > > > > service. > > > > > > > > > > > > > > Some of these ideas would actually require extra discussions > > here, > > > > > given > > > > > > > that I'm pretty much sharing my personal list of *interesting* > > > stuff > > > > to > > > > > > add > > > > > > > to the project, but should probably be well accepted and other > > > > > committers > > > > > > > would pair or act as champions on the work helping you as > > > > possible/when > > > > > > > needed. > > > > > > > > > > > > > > Examples, documentation and bugs also definitely important and > > > > valuable > > > > > > > contributions as well. > > > > > > > > > > > > > > Regarding Geode internals and more details please check these > > > links: > > > > > > > > > > > > > > - > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/GEODE/Geode+Internal+Architecture > > > > > > > - > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/GEODE/Core+Distributed+System+Concepts > > > > > > > - > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/GEODE/Source+code+launch+points > > > > > > > > > > > > > > Hope that helps. > > > > > > > > > > > > > > > > > > > > > > Sorry for the long post. Thanks in advance > > > > > > > > Joao > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > William Markito Oliveira > > > > > > > Enterprise Architect > > > > > > > -- For questions about Apache Geode, please write to > > > > > > > *dev@geode.incubator.apache.org > > > > > > > <dev@geode.incubator.apache.org>* > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > Christian Tzolov <http://www.linkedin.com/in/tzolov> | Solution > > > > Architect, > > > > > EMEA Practice Team | Pivotal <http://pivotal.io/> > > > > > ctzo...@pivotal.io|+31610285517 > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > William Markito Oliveira > > > > Enterprise Architect > > > > -- For questions about Apache Geode, please write to > > > > *dev@geode.incubator.apache.org > > > > <dev@geode.incubator.apache.org>* > > > > > > > > > > > > > -- > Greg Chase > > Director of Big Data Communities > http://www.pivotal.io/big-data > > Pivotal Software > http://www.pivotal.io/ > > 650-215-0477 > @GregChase > Blog: http://geekmarketing.biz/ >
