I switched to a Map which looks like it may have solved the issue.
See this: https://jsperf.com/es6-map-vs-object-properties/
And, running Node.js --perf with the code from that jsperf yields the
following (on Node v8.9.1 which, according to process.versions, is V8
6.1.534.47):
*Map (15.7% in GC)*
[Summary]:
ticks total nonlib name
903 68.1% 68.4% JavaScript
395 29.8% 29.9% C++
208 15.7% 15.7% GC
5 0.4% Shared libraries
23 1.7% Unaccounted
[C++]:
ticks total nonlib name
91 6.9% 6.9% t v8::internal::Scavenger::ScavengeObject(v8::
internal::HeapObject**, v8::internal::HeapObject*)
38 2.9% 2.9% t v8::internal::OrderedHashTable<v8::internal::
OrderedHashMap, 2>::Rehash(v8::internal::Handle<v8::internal::OrderedHashMap
>, int)
36 2.7% 2.7% T v8::internal::Runtime_GenerateRandomNumbers(int,
v8::internal::Object**, v8::internal::Isolate*)
36 2.7% 2.7% T v8::internal::IncrementalMarking::Step(unsigned
long, v8::internal::IncrementalMarking::CompletionAction, v8::internal::
IncrementalMarking::ForceCompletionAction, v8::internal::StepOrigin)
30 2.3% 2.3% T v8::internal::Heap::Scavenge()
25 1.9% 1.9% T v8::internal::Heap::AllocateFixedArrayWithFiller(
int, v8::internal::PretenureFlag, v8::internal::Object*)
24 1.8% 1.8% t node::(anonymous namespace)::ContextifyScript::
New(v8::FunctionCallbackInfo<v8::Value> const&)
14 1.1% 1.1% t v8::internal::(anonymous namespace)::
GetSimpleHash(v8::internal::Object*)
*Object (26.8% in GC)*
[Summary]:
ticks total nonlib name
698 32.2% 32.2% JavaScript
1417 65.3% 65.4% C++
581 26.8% 26.8% GC
2 0.1% Shared libraries
53 2.4% Unaccounted
[C++]:
ticks total nonlib name
269 12.4% 12.4% T v8::internal::IncrementalMarking::Step(unsigned
long, v8::internal::IncrementalMarking::CompletionAction, v8::internal::
IncrementalMarking::ForceCompletionAction, v8::internal::StepOrigin)
148 6.8% 6.8% t v8::internal::HashTable<v8::internal::
NameDictionary, v8::internal::NameDictionaryShape>::Rehash(v8::internal::
NameDictionary*)
122 5.6% 5.6% T v8::internal::Heap::Scavenge()
107 4.9% 4.9% t v8::internal::Scavenger::ScavengeObject(v8::
internal::HeapObject**, v8::internal::HeapObject*)
103 4.7% 4.8% T v8::internal::HashTable<v8::internal::StringTable
, v8::internal::StringTableShape>::Rehash(v8::internal::StringTable*)
91 4.2% 4.2% T v8::internal::StringTable::LookupKey(v8::internal
::Isolate*, v8::internal::StringTableKey*)
82 3.8% 3.8% t v8::internal::LookupIterator::State v8::internal
::LookupIterator::LookupInRegularHolder<false>(v8::internal::Map*, v8::
internal::JSReceiver*)
48 2.2% 2.2% T v8::internal::Heap::AllocateFixedArrayWithFiller(
int, v8::internal::PretenureFlag, v8::internal::Object*)
41 1.9% 1.9% T v8::internal::String::SlowEquals(v8::internal::
String*)
If I plot out the actual duration of the add/update operation to either a
Map or Object, they're both almost equivalent as N increases, but the
memory usage and GC activity *around* that add/update for the Object is
much higher. In fact, depending on the length of the random string being
created, the Object often maxes out the heap and crashes Node.js. (I
experimented with increasing --max_old_space_size up to 10000) There's also
a point (random strings of length 5, with N of 8,000,000) where even if
your heap is large enough to hold all the strings, the adding/appending to
an Object just seizes up (and Map still works).
On Thursday, November 23, 2017 at 12:59:06 AM UTC-8, Camillo Bruni wrote:
>
> Hi Jonathan,
>
> this seems suspicious. Would it be possible for you to write a standalone
> repro?
> Without further information it is hard to asses what exactly might be
> going wrong here.
>
> Cheers,
> Camillo
>
> On Wednesday, November 22, 2017 at 1:29:50 AM UTC+1, Jonathan Otto wrote:
>>
>> I have a plain JavaScript object that looks like:
>>
>> {
>> sha256a: { id, fieldA, fieldB, fieldC },
>> sha256b: { id, fieldA, fieldB, fieldC },
>> ...
>> }
>>
>> and it seems like when I get over ~50,000 sha's then I start to have high
>> CPU usage (~3%) when adding a key or changing a value for an existing key -
>> note I am adding/changing keys once or twice per second. Naively I'm
>> wondering if the GC is traversing the entire object on each addition or
>> change and whether there's a cliff where that falls off.
>>
>> Is there a better way to manage a local store in JavaScript?
>>
>
--
--
v8-users mailing list
v8-users@googlegroups.com
http://groups.google.com/group/v8-users
---
You received this message because you are subscribed to the Google Groups
"v8-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to v8-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
