Hello,
I've been trying to track-down a pretty significant memory bloat issue with
ActiveRecord objects. Running some generic experiments e.g. an empty model
with 6 enum_fields, created_at, updated_at - and fetching 72000 of these
with ActiveRecord from a Postgres Database allocates approximately 120mb of
memory or (1747 bytes per object) - measured using Instruments.On the other
hand fetching the same 72000 objects into a list of hashes directly with
Ruby-PG is only about 10mb of memory.
One particular thing I found which was surprising is that enum is not any
more efficient than string. This appears to be because the Postgresql
Adapter for ActiveRecord uses TypeMapAllString on the RubyPG level and than
performs all of the TypeMapping in Ruby.
<https://github.com/rails/rails/blob/master/activerecord%2Flib%2Factive_record%2Fconnection_adapters%2Fpostgresql%2Fdatabase_statements.rb#L168>
<https://github.com/rails/rails/blob/master/activerecord%2Flib%2Factive_record%2Fconnection_adapters%2Fpostgresql%2Fdatabase_statements.rb#L168>This
seems to be grossly inefficient since Ruby-PG has support for both basic
and custom type-mapping on the C-level.Here is some notable output from the
memory_profiler gem - this is for *only* 729 objects.
1617 "0"
1458 /Users/user_name/.rvm/gems/ruby-2.2.2/gems/activerecord-4.2.3/lib/
active_record/connection_adapters/postgresql/database_statements.rb:169
159 /Users/user_name/.rvm/gems/ruby-2.2.2/gems/activerecord-4.2.3/lib/
active_record/connection_adapters/postgresql/oid/type_map_initializer.rb:
161459 "2"
1459 /Users/user_name/.rvm/gems/ruby-2.2.2/gems/activerecord-4.2.3/lib/
active_record/connection_adapters/postgresql/database_statements.rb:1691459
"1"
1459 /Users/user_name/.rvm/gems/ruby-2.2.2/gems/activerecord-4.2.3/lib/
active_record/connection_adapters/postgresql/database_statements.rb:169729
"_initialize_callbacks"
729 /Users/user_name/.rvm/gems/ruby-2.2.2/gems/activesupport-4.2.3/lib/
active_support/callbacks.rb:81729 "_find_callbacks"
729 /Users/user_name/.rvm/gems/ruby-2.2.2/gems/activesupport-4.2.3/lib/
active_support/callbacks.rb:81729 "initialize"
729 /Users/user_name/.rvm/gems/ruby-2.2.2/gems/activesupport-4.2.3/lib/
active_support/callbacks.rb:81729 "find"
729 /Users/user_name/.rvm/gems/ruby-2.2.2/gems/activesupport-4.2.3/lib/
active_support/callbacks.rb:81505 "\n"
262 /Users/user_name/.rvm/gems/ruby-2.2.2/gems/activerecord-4.2.3/lib/
active_record/relation.rb:17
87 /Users/user_name/.rvm/gems/ruby-2.2.2/gems/activerecord-4.2.3/lib/
active_record/relation/delegation.rb:15
42 /Users/user_name/.rvm/gems/ruby-2.2.2/gems/activerecord-4.2.3/lib/
active_record/relation/delegation.rb:16
34 /Users/user_name/.rvm/gems/ruby-2.2.2/gems/activerecord-4.2.3/lib/
active_record/connection_adapters/abstract/database_statements.rb:228
31 /Users/user_name/.rvm/gems/ruby-2.2.2/gems/activesupport-4.2.3/lib/
active_support/dependencies.rb:274
20 /Users/user_name/.rvm/gems/ruby-2.2.2/gems/activerecord-4.2.3/lib/
active_record/relation.rb:654
20 /Users/user_name/.rvm/gems/ruby-2.2.2/gems/activerecord-4.2.3/lib/
active_record/connection_adapters/postgresql/database_statements.rb:169
7 /Users/user_name/.rvm/gems/ruby-2.2.2/gems/activerecord-4.2.3/lib/
active_record/relation/delegation.rb:17
2 /Users/user_name/.rvm/rubies/ruby-2.2.2/lib/ruby/2.2.0/mutex_m.rb:41
It seems to me that potentially two improvements could be made here:
a) Freezing the callback strings. While "_initialize_callbacks" and
"_find_callbacks" are dynamically generated strings I think the variations
are quite few in number, using hash lookup for this could be a large cost
saver when fetching multiple objects.
b) Switching from TypeMapAllString in the ActiveRecord Postgres Adapter to
native C type-mapping in Ruby-PG.
I have experimented with a patch for Freezing the callback strings
(attached). It may not be very nice but perhaps still preferable to
dynamically generating the callback strings per object.
P.S. I have only about ~3 months of experience with Ruby, so I am quite apt
to have some serious misunderstandings, please take it easy on me :)
--
You received this message because you are subscribed to the Google Groups "Ruby
on Rails: Core" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at http://groups.google.com/group/rubyonrails-core.
For more options, visit https://groups.google.com/d/optout.
diff --git a/lib/active_support/callbacks.rb b/lib/active_support/callbacks.rb
index 1e98547..4006beb 100644
--- a/lib/active_support/callbacks.rb
+++ b/lib/active_support/callbacks.rb
@@ -77,8 +77,13 @@ module ActiveSupport
# run_callbacks :save do
# save
# end
+
+ # Store a frozen copy of a callback string so it doesn't need to be
+ # allocated and de-allocated for every call
+ @@_callbacks_store = {}
+
def run_callbacks(kind, &block)
- callbacks = send("_#{kind}_callbacks")
+ callbacks = send(callbacks_store(kind))
if callbacks.empty?
yield if block_given?
@@ -91,6 +96,10 @@ module ActiveSupport
private
+ def callbacks_store(kind)
+ @@_callbacks_store[kind] ? @@_callbacks_store[kind] : @@_callbacks_store[kind] = "_#{kind}_callbacks".freeze
+ end
+
# A hook invoked every time a before callback is halted.
# This can be overridden in AS::Callback implementors in order
# to provide better debugging/logging.
@@ -764,6 +773,7 @@ module ActiveSupport
#
# NOTE: +method_name+ passed to `define_model_callbacks` must not end with
# `!`, `?` or `=`.
+
def define_callbacks(*names)
options = names.extract_options!
