I should also note: I was encoding strings using only the lower 8-bits assuming that JSON will UTF-8 encode anyway, but there is some experimentation to be done using 16bit representations etc, but I right now we need an order of magnitude perf increase and this won't get us there.
On Tue, Jan 15, 2013 at 10:28 AM, Michal Mocny <[email protected]> wrote: > Don, I had previously tried a byte array and found it quite a bit less > efficient. I'll add that implementation and benchmark it officially. > > I did benchmarks on iphone5 today for two initial implementations: > (1) Using base64 encoded javascript strings > (2) Using non base64 encoded, unicode javascript strings > > (1) YES BASE64 ENCODE > > 2013-01-15 10:16:02.233 MobileSpecTest[1930:907] Multi-tasking -> Device: > YES, App: YES > 2013-01-15 10:16:02.749 MobileSpecTest[1930:907] [LOG] Device = iOS 6.0.2 > 2013-01-15 10:16:38.104 MobileSpecTest[1930:907] [LOG] Started exec > benchmark with payload length: 1 > 2013-01-15 10:16:39.103 MobileSpecTest[1930:907] [LOG] Calls per second: > 274.72527472527474 > 2013-01-15 10:16:55.995 MobileSpecTest[1930:907] [LOG] Started exec > benchmark with payload length: 32000 > 2013-01-15 10:17:01.007 MobileSpecTest[1930:907] [LOG] Calls per second: > 18.909235668789808 > 2013-01-15 10:17:02.795 MobileSpecTest[1930:907] [LOG] Started exec > benchmark with payload length: 32000 > 2013-01-15 10:17:07.785 MobileSpecTest[1930:907] [LOG] Calls per second: > 18.59628074385123 > 2013-01-15 10:17:14.402 MobileSpecTest[1930:907] [LOG] Started exec > benchmark with payload length: 32 > 2013-01-15 10:17:19.402 MobileSpecTest[1930:907] [LOG] Calls per second: > 265.89364254298283 > 2013-01-15 10:17:20.651 MobileSpecTest[1930:907] [LOG] Started exec > benchmark with payload length: 32 > 2013-01-15 10:17:25.651 MobileSpecTest[1930:907] [LOG] Calls per second: > 263.69452219112355 > 2013-01-15 10:17:36.187 MobileSpecTest[1930:907] [LOG] Started exec > benchmark with payload length: 1024 > 2013-01-15 10:17:41.189 MobileSpecTest[1930:907] [LOG] Calls per second: > 185.8141858141858 > 2013-01-15 10:17:44.701 MobileSpecTest[1930:907] [LOG] Started exec > benchmark with payload length: 1024 > 2013-01-15 10:17:49.699 MobileSpecTest[1930:907] [LOG] Calls per second: > 186.9626074785043 > 2013-01-15 10:17:51.084 MobileSpecTest[1930:907] [LOG] Started exec > benchmark with payload length: 1024 > 2013-01-15 10:17:56.082 MobileSpecTest[1930:907] [LOG] Calls per second: > 184.8 > > (2) NO BASE64 ENCODE > > 2013-01-15 10:12:12.387 MobileSpecTest[1905:907] Multi-tasking -> Device: > YES, App: YES > 2013-01-15 10:12:12.892 MobileSpecTest[1905:907] [LOG] Device = iOS 6.0.2 > 2013-01-15 10:12:21.462 MobileSpecTest[1905:907] [LOG] Started exec > benchmark with payload length: 1 > 2013-01-15 10:12:22.458 MobileSpecTest[1905:907] [LOG] Calls per second: > 267 > 2013-01-15 10:12:32.212 MobileSpecTest[1905:907] [LOG] Started exec > benchmark with payload length: 32000 > 2013-01-15 10:12:37.204 MobileSpecTest[1905:907] [LOG] Calls per second: > 15.18784972022382 > 2013-01-15 10:12:38.898 MobileSpecTest[1905:907] [LOG] Started exec > benchmark with payload length: 32000 > 2013-01-15 10:12:43.897 MobileSpecTest[1905:907] [LOG] Calls per second: > 15.163607342378292 > 2013-01-15 10:12:51.802 MobileSpecTest[1905:907] [LOG] Started exec > benchmark with payload length: 32 > 2013-01-15 10:12:56.801 MobileSpecTest[1905:907] [LOG] Calls per second: > 261.8952419032387 > 2013-01-15 10:12:58.534 MobileSpecTest[1905:907] [LOG] Started exec > benchmark with payload length: 32 > 2013-01-15 10:13:03.533 MobileSpecTest[1905:907] [LOG] Calls per second: > 262.89484206317474 > 2013-01-15 10:14:19.153 MobileSpecTest[1905:907] [LOG] Started exec > benchmark with payload length: 1 > 2013-01-15 10:14:24.154 MobileSpecTest[1905:907] [LOG] Calls per second: > 269.43833699780134 > 2013-01-15 10:14:27.605 MobileSpecTest[1905:907] [LOG] Started exec > benchmark with payload length: 1 > 2013-01-15 10:14:32.604 MobileSpecTest[1905:907] [LOG] Calls per second: > 272.34553089382126 > 2013-01-15 10:14:42.134 MobileSpecTest[1905:907] [LOG] Started exec > benchmark with payload length: 1024 > 2013-01-15 10:14:47.132 MobileSpecTest[1905:907] [LOG] Calls per second: > 173.6 > 2013-01-15 10:14:49.217 MobileSpecTest[1905:907] [LOG] Started exec > benchmark with payload length: 1024 > 2013-01-15 10:14:54.216 MobileSpecTest[1905:907] [LOG] Calls per second: > 173.8 > 2013-01-15 10:14:55.518 MobileSpecTest[1905:907] [LOG] Started exec > benchmark with payload length: 1024 > 2013-01-15 10:15:00.520 MobileSpecTest[1905:907] [LOG] Calls per second: > 173.22677322677322 > > > A note about the data: I was using bytes from 0..255 evenly distributed, > and the JSON serialization of that looks > like: > "\u0000\u0001\u0002\u0003\u0004\u0005\u0006\u0007\b\t\n\u000b\f\r\u000e\u000f\u0010\u0011\u0012\u0013\u0014\u0015\u0016\u0017\u0018\u0019\u001a\u001b\u001c\u001d\u001e\u001f > !\"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\\]^_`abcdefghijklmnopqrstuvwxyz{|}~ > € ‚ƒ„…†‡ˆ‰Š‹Œ Ž ‘’“”•–—˜™š›œ žŸ > ¡¢£¤¥¦§¨©ª«¬®¯°±²³´µ¶·¸¹º»¼½¾¿ÀÁÂÃÄÅÆÇÈÉÊËÌÍÎÏÐÑÒÓÔÕÖ×ØÙÚÛÜÝÞßàáâãäåæçèéêëìíîïðñòóôõö÷øùúûüýþÿ" > > Which is a 400 character length string. Base64 encoding 256 characters is > a 342 character string. However, if the data were not evenly distributed, > and instead consisted mostly of readable characters (as is the case with > many protocols) then non-base64 encoded would be faster. > > I've left the base64 encoded version, assuming binary data will be random, > and also because we already support sending raw strings across the bridge, > so a plugin developer can do that if its more efficient. > > Either way, this is terribly slow, and I'll have to start getting creative. > > Finally, there is a bug where it seems that data larger than ~64k wont > transfer across the bridge at all. Andrew, is there a known limit to how > much payload exec can send across? > > -Michal > > > > On Wed, Jan 9, 2013 at 3:57 PM, Don Coleman <[email protected]> wrote: > >> For the phonegap-nfc plugin I send binary data between native and >> javascript as a byte array in JSON. >> >> static JSONArray byteArrayToJSON(byte[] bytes) { >> JSONArray json = new JSONArray(); >> for (int i = 0; i < bytes.length; i++) { >> json.put(bytes[i]); >> } >> return json; >> } >> >> >> On Wed, Jan 9, 2013 at 3:00 PM, Andrew Grieve <[email protected]> >> wrote: >> > >> > On Wed, Jan 9, 2013 at 10:15 AM, Michal Mocny <[email protected]> >> wrote: >> > >> > > On Tue, Jan 8, 2013 at 10:22 PM, Andrew Grieve <[email protected]> >> > > wrote: >> > > >> > > > Rather than "binary data", I think we should focus strictly on >> > > > "ArrayBuffer". I think the only other binary data concept is Blob, >> and >> > > Blob >> > > > is easy to create from an ArrayBuffer. >> > > > >> > > > As for how to have one of these show up in a JSON object, I think >> it will >> > > > be tough to make this work. Most platforms assume that you can >> paste the >> > > > JSON into an eval / JSON.parse, and it will come out correctly. That >> > > can't >> > > > be the case for ArrayBuffers. Maybe we should just not support >> this. The >> > > > use-case here is being able to pass multiple values to callbacks. >> For >> > > > ArrayBuffers, maybe we can passing the usual payload plus an >> optional >> > > > ArrayBuffer as an additional parameter. I don't think it would be >> > > necessary >> > > > to have multiple ArrayBuffers for a single callback... >> > > > >> > > >> > > To understand correct, does this mean that you propose a single >> ArrayBuffer >> > > instead of an arguments list, or, to add a 6th parameter to exec >> which is >> > > the one and only ArrayBuffer? >> > > >> > > I propose yet another option, maybe you can any number of >> ArrayBuffers in >> > > the arguments list, but not embedded in any subobject. Thus, >> semantically >> > > the arguments array stops being JSON serializable, and the individual >> > > arguments are either ArrayBuffers or json-serializable (this also >> leaves >> > > door open to add more types in the future). >> > > >> > >> > Hmm, I wasn't even thinking about the JS->native case actually. I was >> only >> > thinking about native->JS. >> > What you propose sounds good for JS->Native. >> > >> > >> > > >> > > >> > > > >> > > > >> > > > My ideas for transferring this over the bridge on iOS include: >> > > >> > > - an array of numbers >> > > > - a string of shorts encoded as chars with unicode chars encoded as >> > > \uXXXX >> > > > - a URL that the client can fetch via XHR and we can return the >> binary >> > > data >> > > > via CDVURLProtocol (advantage here is that XHR2 can request the >> payload >> > > as >> > > > an ArrayBuffer instead of having to construct the ArrayBuffer >> > > > after-the-fact >> > > > >> > > interesting idea! >> > > >> > > >> > > > >> > > > For Android, the string of short-encoded chars is probably the best >> bet >> > > > since the bridge natively supports transferring strings without >> having to >> > > > parse them as JS literals. >> > > > >> > > > >> > > > >> > > > On Tue, Jan 8, 2013 at 4:05 PM, Michal Mocny <[email protected]> >> wrote: >> > > > >> > > > > Created https://issues.apache.org/jira/browse/CB-2173 >> > > > > >> > > > > >> > > > > On Tue, Jan 8, 2013 at 3:52 PM, Michal Mocny <[email protected]> >> > > wrote: >> > > > > >> > > > > > Background: I've been working on implementing chrome.socket [1] >> for >> > > > > mobile >> > > > > > chrome apps [2] and porting circ (an irc chrome app) [3]. >> Teaser >> > > > video: >> > > > > > https://docs.google.com/open?id=0B0UdPHoQPXheTzhTZXZHUlpGWHM (this >> > > is >> > > > > > still in its infancy, and I certainly do plan to submit a >> version of >> > > > the >> > > > > > socket api that will work in cordova without any mobile chrome >> app >> > > > > magic). >> > > > > > >> > > > > > As part of doing that, I had to implement sending binary data >> across >> > > > the >> > > > > > exec bridge (on ios for now), and I think it may be a good idea >> to >> > > just >> > > > > add >> > > > > > that functionality to cordova core. My current implementation >> is >> > > > > certainly >> > > > > > not the best, so I will extend the exec bridge echo benchmarks >> to >> > > test >> > > > > > binary transfer speeds and we can improve over time. >> > > > > > >> > > > > > My first step would be to implement helpers in js/ios/android >> which >> > > > > plugin >> > > > > > devs can call manually to serialize/deserialize binary data >> into/from >> > > > > > whatever magical form is most efficient. >> > > > > > >> > > > > > Next, I would try to automate these helpers from within the >> exec call >> > > > > > (e.g. iterate arguments looking for ArrayBuffer/Blob/etc types >> before >> > > > > json >> > > > > > serialization). As part of doing this, I would need to add >> "hints" >> > > > about >> > > > > > the underlying argument types/some clever way to encode semantic >> > > > > > information about the arguments list, which we currently do not >> do. >> > > > > > >> > > > > > If anyone has any suggestions or objections please let me know! >> > > > > > >> > > > > > -Michal >> > > > > > >> > > > > > [1] http://developer.chrome.com/apps/socket.html >> > > > > > [2] >> > > > > > >> > > > > >> > > > >> > > >> https://github.com/MobileChromeApps/chrome-cordova/blob/master/api/chrome/socket.jsand >> > > > > > >> > > >> https://github.com/MobileChromeApps/chrome-cordova/tree/master/plugins >> > > > > > [3] https://github.com/flackr/circ >> > > > > > >> > > > > >> > > > >> > > >> > >
