Hi, On Mon, Dec 17, 2018 at 3:46 PM <lawrence.krub...@gmail.com> wrote:
> Laurens Van Houtven, good ideas, but then I'd also have to write some code > to catch documents that got lost when a process died while trying to fetch > a document from S3. If I simply check every 15 minutes, and grab everything > that has not already been stored in the database, then I automatically > fetch documents that were dropped due to error. It seems simple. In my > current case, there is no penalty for fetching the same document twice, but > it is important that we avoid missing a document. > I'm not sure why that makes my suggestions not work: you create a CloudWatch Event that just fires every 15 minutes :-) Heck, you can write one that fires on S3 write, and then another that fires on cloudwatch errors for when that process dies :-) But I'd probably just use a CloudWatch event that fires every 15 minute sand run your code as an ECS Fargate job. lvh > > > On Monday, December 17, 2018 at 2:59:22 PM UTC-5, Laurens Van Houtven > wrote: >> >> Honestly I'd use CloudWatch Timed Events to kick off a Lambda or ECS >> Fargate job (which of course you can write in Clojure) assuming you're >> using AWS yourself anyway. If you don't care about batching maybe even just >> attach a Lambda to the write-to-S3 bucket itself instead of checking every >> 15m? >> >> If you want to do it in-process, my tool of choice is ztellman's >> manifold: >> https://github.com/ztellman/manifold/blob/d67a8c1b9f1268c094895d70dbbf146521f5774b/src/manifold/time.clj >> >> On Mon, Dec 17, 2018 at 1:54 PM <lawrence...@gmail.com> wrote: >> >>> I'm coming back to Clojure development after a year away. This is a fast >>> moving community and it is hard to keep up when one is not working on it >>> full time. I'm dusting off some code I wrote 2 years ago, and trying to >>> bring all the dependencies up to their current versions. >>> >>> I have a function that fetches files from an AWS S3 bucket, every 15 >>> minutes. I had previously used the at-at library for this: >>> >>> https://github.com/overtone/at-at >>> >>> But at-at has not been updated in 6 years, so I assume it is abandoned. >>> I have two questions about this: >>> >>> 1.) how else do Clojure programmers usually call recurring >>> functionality? >>> >>> 2.) I am ignorant about the JVM, so I'm afraid I have to ask, at a >>> fundamental level, how does at-at work? I know that if a function calls >>> itself recurringly, on the JVM, one eventually gets stackoverflow. So how >>> does at-at make its magic work? >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Clojure" group. >>> To post to this group, send email to clo...@googlegroups.com >>> Note that posts from new members are moderated - please be patient with >>> your first post. >>> To unsubscribe from this group, send email to >>> clojure+u...@googlegroups.com >>> For more options, visit this group at >>> http://groups.google.com/group/clojure?hl=en >>> --- >>> You received this message because you are subscribed to the Google >>> Groups "Clojure" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to clojure+u...@googlegroups.com. >>> For more options, visit https://groups.google.com/d/optout. >>> >> -- > You received this message because you are subscribed to the Google > Groups "Clojure" group. > To post to this group, send email to clojure@googlegroups.com > Note that posts from new members are moderated - please be patient with > your first post. > To unsubscribe from this group, send email to > clojure+unsubscr...@googlegroups.com > For more options, visit this group at > http://groups.google.com/group/clojure?hl=en > --- > You received this message because you are subscribed to the Google Groups > "Clojure" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to clojure+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
