Hello everyone, A couple of months ago I decided that I wanted to really understand how the Beam FnApi works and how it interacts with the Portable Runner. For me at least that usually means I need to write some code so I can see things happening in a debugger and to really prove to myself I understood what was going on I decided I couldn't use an existing SDK language to do it since there would be the temptation to read some code and convince myself that I actually understood what was going on.
One thing led to another and it turns out that to get a minimal FnApi integration going you end up writing a fair bit of an SDK. So I decided to take things to a point where I had an SDK that could execute a word count example via a portable runner backend. I've now reached that point and would like to submit my prototype SDK to the list for feedback. It's currently living in a branch on my fork here: https://github.com/byronellis/beam/tree/swift-sdk/sdks/swift At the moment it runs via the most recent XCode Beta using Swift 5.9 on Intel Macs, but should also work using beta builds of 5.9 for Linux running on Intel hardware. I haven't had a chance to try it on ARM hardware and make sure all of the endian checks are complete. The "IntegrationTests.swift" file contains a word count example that reads some local files (as well as a missing file to exercise DLQ functionality) and output counts through two separate group by operations to get it past the "map reduce" size of pipeline. I've tested it against the Python Portable Runner. Since my goal was to learn FnApi there is no Direct Runner at this time. I've shown it to a couple of folks already and incorporated some of that feedback already (for example pardo was originally called dofn when defining pipelines). In general I've tried to make the API as "Swift-y" as possible, hence the heavy reliance on closures and while there aren't yet composite PTransforms there's the beginnings of what would be needed for a SwiftUI-like declarative API for creating them. There are of course a ton of missing bits still to be implemented, like counters, metrics, windowing, state, timers, etc. Any and all feedback welcome and happy to submit a PR if folks are interested, though the "Swift Way" would be to have it in its own repo so that it can easily be used from the Swift Package Manager. Best, B
