We may directly send request to GA from Ignite node: https://developers.google.com/analytics/devguides/collection/protocol/v1/ <https://developers.google.com/analytics/devguides/collection/protocol/v1/> Latest version can be received from maven central: https://repo1.maven.org/maven2/org/apache/ignite/ignite-core/maven-metadata.xml <https://repo1.maven.org/maven2/org/apache/ignite/ignite-core/maven-metadata.xml>
> 2 окт. 2017 г., в 12:51, Dmitriy Setrakyan <dsetrak...@apache.org> написал(а): > > Denis, > > I am not sure I understand. We already do have CGI enabled for > download.cgi. Is there something else we need? > > D. > > On Mon, Oct 2, 2017 at 8:35 AM, Denis Magda <dma...@gridgain.com> wrote: > >> There is an obstacle. There is no way to execute the script using PHP or >> similar sever side language and trigger GA as discussed earlier: >> https://issues.apache.org/jira/browse/INFRA-15182 >> >> How else can we tackle this? >> >> Denis >> >> On Thursday, September 7, 2017, Dmitriy Setrakyan <dsetrak...@apache.org> >> wrote: >> >>> I think it is safe to assume at this point that everyone is in general >>> agreement, since there are no active objections. >>> >>> I have filed a ticket for the 2.3 release. Let's try to make it happen: >>> https://issues.apache.org/jira/browse/IGNITE-6305 >>> >>> D. >>> >>> On Sat, Aug 26, 2017 at 3:06 PM, Dmitriy Setrakyan < >> dsetrak...@apache.org >>> <javascript:;>> >>> wrote: >>> >>>> >>>> >>>> On Sat, Aug 26, 2017 at 3:22 AM, Raúl Kripalani <raul....@evosent.com >>> <javascript:;>> >>>> wrote: >>>> >>>>> Yeah, I guess that's doable as well and requires less management >> effort >>>>> than my suggestion. We could use events [1] to store payload data >> (e.g. >>>>> IP, >>>>> version, etc.) >>>> >>>> >>>> Yes, we could use events or some other similar API provided by GA. >>>> >>>> >>>>> What the download page CGI developed in? PHP? >>>>> >>>> >>>> To be honest, no clue. I guess someone in the community can figure it >>> out: >>>> https://svn.apache.org/repos/asf/ignite/site/trunk/download.html >>>> >>>> >>>>> However, I'm not sure whether storing this data in a 3rd party >> (Google) >>> is >>>>> compliant with the ASF policy. I guess it's no biggie, but if there's >>>>> doubt >>>>> in the PMC, it's better to ask legal@. >>>> >>>> >>>> I am not sure there is anything to ask about. The whole Ignite website >> is >>>> GA enabled, and all we are doing is accessing a standard web page from >>> the >>>> Ignite web site. The information gathered from GA is available only to >>> the >>>> Ignite PMC. Frankly, I think legal@ will be very confused by this >>>> question. >>>> >>>> Even ASF website itself uses GA: https://www.apache.org/ >>>> foundation/policies/privacy.html >>>> >>>> >>>>> I think Cos said it's OK; maybe Roman can pitch in. >>>>> >>>> >>>> Sure, would be nice to hear from Roman as well. >>>> >>>> >>>>> Cheers. >>>>> >>>>> [1] >>>>> https://developers.google.com/analytics/devguides/collection >>>>> /analyticsjs/events >>>>> >>>>> On Sat, Aug 26, 2017 at 12:23 AM, Dmitriy Setrakyan < >>>>> dsetrak...@apache.org <javascript:;>> >>>>> wrote: >>>>> >>>>>> Raul, >>>>>> >>>>>> Could point about Javascript, it will not work because it executes >> in >>>>> the >>>>>> browser. This means we need a server-side script, like CGI we are >>> using >>>>> on >>>>>> our download page. >>>>>> >>>>>> How about this approach. We create something like ignite-version.cgi >>>>> script >>>>>> which will invoke a call to GA and then return the latest version. >>>>>> >>>>>> This should work, right? >>>>>> >>>>>> D. >>>>>> >>>>>> On Fri, Aug 25, 2017 at 2:42 PM, Raúl Kripalani < >> raul....@evosent.com >>> <javascript:;>> >>>>>> wrote: >>>>>> >>>>>>> Hey Dmitriy and all >>>>>>> >>>>>>> Also, since we have GA enabled for the website, we can track how >>> many >>>>>> times >>>>>>>> this page was accessed, which will be equal to the number of >>> starts. >>>>>> This >>>>>>>> way, the counter information is tracked and monitored by the >>> Ignite >>>>>> PMC. >>>>>>> >>>>>>> >>>>>>> Unfortunately this won't work because GA is loaded via Javascript >> on >>>>> the >>>>>>> browser, so Google will never receive the page hit. >>>>>>> >>>>>>> Given the constraints, the most viable solution is an HTTPS >> endpoint >>>>>>> running on ASF infra that Ignite invokes via a GET or POST >> request. >>>>> The >>>>>>> simplest thing is to write each request in a log file, along with >>> the >>>>>>> timestamp, the version reported by the client, maybe the IP (not >>> sure >>>>>> about >>>>>>> the ASF rules about this concerning privacy, I guess it's OK if >> you >>>>> make >>>>>> it >>>>>>> an opt-in) and a unique node identifier, i.e. a UUID the node >>> creates >>>>> on >>>>>>> first startup or something. >>>>>>> >>>>>>> This endpoint would need some basic DDoS protection and >> blacklisting >>>>> to >>>>>>> prevent data spoofing. >>>>>>> >>>>>>> If we'll be implementing this endpoint anyway, then there's no >> point >>>>>>> placing another file on Git or elsewhere for reporting the latest >>>>>> versions: >>>>>>> the endpoint itself can return them. >>>>>>> >>>>>>> WDYT? >>>>>>> >>>>>>> Cheers. >>>>>>> >>>>>>> On Fri, Aug 25, 2017 at 9:56 PM, Dmitriy Setrakyan < >>>>>> dsetrak...@apache.org <javascript:;>> >>>>>>> wrote: >>>>>>> >>>>>>>> Cos, Raul, >>>>>>>> >>>>>>>> Thanks for the feedback. I completely agree about Maven Central >>>>> being a >>>>>>> 3rd >>>>>>>> party repo (did not think about that initially). All your >>>>> suggestions >>>>>>> make >>>>>>>> sense, but I would like to keep it as simple as possible, and so >>> far >>>>>>>> everything suggested required GIT dependencies and extra work. >>>>>>>> >>>>>>>> How about Yakov's suggestion. We simply add a page to the Ignite >>>>>> website >>>>>>>> which will have only the latest version. Every time a node >> starts, >>>>> it >>>>>>>> receives the latest version from the page and suggests that >> users >>>>>> upgrade >>>>>>>> if needed. >>>>>>>> >>>>>>>> Also, since we have GA enabled for the website, we can track how >>>>> many >>>>>>> times >>>>>>>> this page was accessed, which will be equal to the number of >>> starts. >>>>>> This >>>>>>>> way, the counter information is tracked and monitored by the >>> Ignite >>>>>> PMC. >>>>>>>> >>>>>>>> This approach looks pretty innocent to me and everything is kept >>> and >>>>>>>> managed within Apache. >>>>>>>> >>>>>>>> Thoughts? >>>>>>>> >>>>>>>> D. >>>>>>>> >>>>>>>> >>>>>>>> On Fri, Aug 25, 2017 at 11:29 AM, Konstantin Boudnik < >>>>> c...@apache.org <javascript:;>> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> I agree with Raul. >>>>>>>>> - providing a ping-back address to a 3rd party might be frown >>>>> upon by >>>>>>>> some. >>>>>>>>> And might have a consequences like stats collection about >>> users' >>>>>>>>> infrastructure. >>>>>>>>> - checking an ASF git-repo is easy and won't download any >> binary >>>>>> data: >>>>>>>>> everything is clear text and could be easily monitored by >> any >>> of >>>>>> the >>>>>>>>> network >>>>>>>>> diagnostic tools, shall it be required. But it involves a >> bit >>> of >>>>>> the >>>>>>>>> release >>>>>>>>> discipline. >>>>>>>>> - the binary data download in the runtime is my main concern. >>>>> This is >>>>>>> the >>>>>>>>> vector of MMA. What if someone gains the control over the >>>>>> repository >>>>>>>> and >>>>>>>>> replaces the file with some malicious content. >>>>>>>>> >>>>>>>>> As for the particular mechanism: IIRC Ignite used to make a >> call >>>>> to >>>>>> an >>>>>>>>> external script to check upon the atest software version >>> available >>>>>> for >>>>>>>>> download. In the past, the endpoint was running on a 3rd party >>>>>> server, >>>>>>> I >>>>>>>>> believe the best way would be to put this script on ASF infra >>> and >>>>>> have >>>>>>>> the >>>>>>>>> "update checker" running in a completely controlled >> environment. >>>>>>>> Actually, >>>>>>>>> I >>>>>>>>> recall we had this very discussion during the Incubation; I >> can >>>>>>> probably >>>>>>>>> dig >>>>>>>>> out the corresponding thread. >>>>>>>>> >>>>>>>>> Thoughts? >>>>>>>>> Cok >>>>>>>>> >>>>>>>>> On Fri, Aug 25, 2017 at 10:41AM, Raul Kripalani wrote: >>>>>>>>>> Hey guys >>>>>>>>>> >>>>>>>>>> In my opinion, maven.org is still owned by a third party >>>>>> (Sonatype). >>>>>>>> We >>>>>>>>>> don't know what kind of data analysis or intelligence >>> extraction >>>>>> they >>>>>>>>> run. >>>>>>>>>> >>>>>>>>>> If Ignite servers all over the world were hitting maven.org >>>>>>>> periodically >>>>>>>>>> asking for an Ignite Maven artifact, it gives Sonatype a >> clear >>>>>>>> indication >>>>>>>>>> of who is running an Ignite server. >>>>>>>>>> >>>>>>>>>> They could reverse lookup the IP address and find out what >>>>>>> corporation >>>>>>>> it >>>>>>>>>> is. >>>>>>>>>> >>>>>>>>>> How about having Ignite check the ASF Git directly? >>>>>>>>>> >>>>>>>>>> We could use the Git ssh API, but that would require a new >>>>>>> dependency, >>>>>>>>>> which I advise against. >>>>>>>>>> >>>>>>>>>> Alternatively, we could have it scrape this HTML for new Git >>>>> tags: >>>>>>>>>> https://git-wip-us.apache.org/repos/asf?p=ignite.git >>>>>>>>>> >>>>>>>>>> Another option is to place a txt file in the root of the >>> master >>>>>>> branch >>>>>>>>> (e.g >>>>>>>>>> LATEST), containing a list of the latest GA versions for >> each >>>>> major >>>>>>>>> version >>>>>>>>>> line (1.x, 2.x). >>>>>>>>>> >>>>>>>>>> I would advocate this last option, but it requires somebody >>>>>>> remembering >>>>>>>>> to >>>>>>>>>> update the file with every release, unless we automate it >>> with a >>>>>>> Maven >>>>>>>>>> plugin. >>>>>>>>>> >>>>>>>>>> Hope that helps! >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 24 Aug 2017 19:37, "Denis Magda" <dma...@apache.org >>> <javascript:;>> wrote: >>>>>>>>>> >>>>>>>>>> I see nothing wrong with this approach. >>>>>>>>>> >>>>>>>>>> Cos, Roman, Raul, as Apache veterans, what do you think? Is >> it >>>>> good >>>>>>> to >>>>>>>>> go? >>>>>>>>>> >>>>>>>>>> — >>>>>>>>>> Denis >>>>>>>>>> >>>>>>>>>>> On Aug 23, 2017, at 11:17 PM, Dmitriy Setrakyan < >>>>>>>> dsetrak...@apache.org <javascript:;> >>>>>>>>>> >>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>> Is everyone OK with this approach? Should I file a ticket >> on >>>>> it? >>>>>>>>>>> >>>>>>>>>>> On Mon, Aug 21, 2017 at 2:07 PM, Dmitriy Setrakyan < >>>>>>>>> dsetrak...@apache.org <javascript:;>> >>>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>>> Igniters, >>>>>>>>>>>> >>>>>>>>>>>> There has been lots of talk of proposals about various >>> usage >>>>>>> metrics >>>>>>>>> for >>>>>>>>>>>> Ignite and nothing came of it. I would like to resurrect >>> the >>>>>> topic >>>>>>>> and >>>>>>>>>>>> propose something very simple and non-intrusive. >>>>>>>>>>>> >>>>>>>>>>>> 1. Update Checker >>>>>>>>>>>> The main purpose of the update checker is not to collect >>>>>> metrics, >>>>>>>> but >>>>>>>>> to >>>>>>>>>>>> notify users about a new version of Ignite by accessing >>>>>> maven.org >>>>>>>> and >>>>>>>>>>>> getting the version out of the metadata file: >>>>>>>>>>>> http://repo2.maven.org/maven2/ >>> org/apache/ignite/ignite-core/ >>>>>>>>>>>> maven-metadata.xml >>>>>>>>>>>> >>>>>>>>>>>> This way we do not send any information anywhere and, at >>> the >>>>>> same >>>>>>>>> time, >>>>>>>>>>>> urge our users to download and start using the latest >>>>> version of >>>>>>>>> Ignite. >>>>>>>>>>>> >>>>>>>>>>>> 2. Startup Counter >>>>>>>>>>>> This piece is optional, but we can also get an insight in >>> how >>>>>> many >>>>>>>>> times >>>>>>>>>> a >>>>>>>>>>>> certain Ignite release gets started. This is just a cool >>>>> metric >>>>>>> for >>>>>>>>> the >>>>>>>>>>>> community to gauge the project popularity. You can think >> of >>>>> it >>>>>> as >>>>>>>> of a >>>>>>>>>> page >>>>>>>>>>>> visit counter shown on many websites. We can even decide >> to >>>>>>> display >>>>>>>>> this >>>>>>>>>>>> counter on the Ignite website as well. >>>>>>>>>>>> >>>>>>>>>>>> To do this, we can simply add a JAR in maven for every >>>>> release, >>>>>>> e.g. >>>>>>>>>>>> ignite-start-counter.jar, which will contain only 1 byte. >>>>> Every >>>>>>> time >>>>>>>>> an >>>>>>>>>>>> Ignite node starts, it will download this JAR in the >>>>> background. >>>>>>>> Then >>>>>>>>> we >>>>>>>>>>>> will be able to view the number of the total downloads >> for >>>>> this >>>>>>> JAR >>>>>>>> in >>>>>>>>>>>> Maven Central, which is essentially the number of starts >> of >>>>>> Ignite >>>>>>>>> nodes. >>>>>>>>>>>> >>>>>>>>>>>> *Note that neither of the above suggestions require >> Ignite >>> to >>>>>> send >>>>>>>> or >>>>>>>>>>>> track any user information whatsoever.* >>>>>>>>>>>> >>>>>>>>>>>> Please reply suggesting weather you are OK with this >>>>> approach. >>>>>>>>>>>> >>>>>>>>>>>> D. >>>>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>>> >>> >>
