Thanks for the interest in this APE Rithwik, and especially for the interest in moving it forward toward implementation. I have been a bit too preoccupied with other tasks to get a full patch ready :) I think all of your proposed alterations to move the enhancement forward in a more reasonable timeframe sound great. Indeed the API that handles the credentials is a bit of a sticky part, since there is no really canonical or accepted way to store credentials in AsterixDB today. I have some thoughts on that separately, so maybe in parallel I can move that piece forward, while these alterations unblock this enhancement. I am more than happy to help with review or whatever else I can with respect to getting everything merged into master for the next release.
Regarding Wail's point, that's very interesting. I think in the original proposal I had imagined the batch mode being kind of like how it works for UDFs today, where you can send an array of arguments rather than one at a time, for those APIs that might accept it. I had never considered something like sending n requests in parallel, asynchronously, and waiting for them to come back. That's a very neat idea, and it would fit pretty well with the way the whole UDF pipeline works today (i.e. the optimizer should guard us from those situations where it wouldn't work semantically) - Ian On Sat, Oct 31, 2026 at 11:31 PM Wail Alkowaileet <[email protected]> wrote: > > Hi, > > I just want to chime in as I was working on something similar: > > > 5. No batched mode yet; each row makes its own call. > > Not sure if there's a way around that unless the API accepts a batch. > However, we found that one request at a time can be too slow because each > request must establish a connection, handshake, ... etc. (latency). We use > asynchronous requests, where we perform multiple requests concurrently and > wait for them. This way you can amortize the latency cost. > > On Sat, Oct 31, 2026 at 11:04 AM Rithwik Koul via dev < > [email protected]> wrote: > > > Hi all, > > > > I'd like to work on APE 34 (external functions). I'm worried the full > > scope is too large for the next release, so I'd like to > > propose a smaller first version. > > > > Proposed changes for the first version: > > > > 1. No endpoint objects or endpoint admin API yet. AT takes the URL > > (template) directly, as APE 34 already allows, and any credentials go > > in the WITH clause, the way external collections take them today: > > > > CREATE EXTERNAL FUNCTION weather(city) > > AT 'https://api.weather.com/v1/now' > > WITH { "bearerToken": "abc123", "method": "GET" }; > > > > 2. Credential types: Basic, Bearer, OAuth client credentials, and an > > API key sent in a header of the API's choosing (e.g. x-api-key). > > Client certificates, "Local" auth and refresh intervals are left > > for later. > > > > 3. No URL whitelist or local/link-local address check in AsterixDB for > > now. A proxy, request and response size limits, default and maximum > > timeouts, and a per-NC rate limit are read from the config file. > > > > 4. The method moves from AS into WITH: "method" is "GET" or "POST", and > > for POST, "body" is "json", "form" or "formdata". This keeps the DDL > > free of new keywords. "headers" becomes a plain object. > > > > 5. No batched mode yet; each row makes its own call. > > > > 6. External functions cannot be called from feed transform functions > > in the first version. > > > > Would these changes be OK? > > > > Thanks, > > Rithwik > > > > > -- > > *Regards,* > Wail Alkowaileet
