Skip to content

Implements on-demand ('lazy') loading of ActiveRecord columns to save memory & query time when they aren't required

License

Notifications You must be signed in to change notification settings

sorare/columns_on_demand

 
 

Repository files navigation

ColumnsOnDemand

Lazily loads large columns on demand.

By default, does this for all TEXT (:text) and BLOB (:binary) columns, but a list of specific columns to load on demand can be given.

This is useful to reduce the memory taken by Rails when loading a number of records that have large columns if those particular columns are actually not required most of the time. In this situation it can also greatly reduce the database query time because loading large BLOB/TEXT columns generally means seeking to other database pages since they are not stored wholly in the record's page itself.

Although this plugin is mainly used for BLOB and TEXT columns, it will actually work on all types - and is just as useful for large string fields etc.

Compatibility

Supports mysql, mysql2, postgresql, and sqlite3.

Currently tested against Rails 6.1.0.rc1, 6.0.3.4, 5.2.4.4, 5.1.7, and 5.0.7.2, with older gem versions compatible with earlier Rails versions.

Example

Example.all will exclude the file_data and processing_log columns from the SELECT query, and example.file_data and example.processing_log will load & cache that individual column value for the record instance:

  class Example
    columns_on_demand :file_data, :processing_log
  end

Scans the examples table columns and registers all TEXT (:text) and BLOB (:binary) columns for loading on demand:

  class Example
    columns_on_demand
  end

Thanks

  • Tim Connor (@tlconnor)
  • Tobias Matthies (@tobmatth)
  • Phil Ross (@philr)
  • Jens Schmidt (@w3dot0)

Copyright (c) 2008-2020 Will Bryant, Sekuda Ltd, released under the MIT license

About

Implements on-demand ('lazy') loading of ActiveRecord columns to save memory & query time when they aren't required

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Ruby 98.3%
  • Shell 1.7%