Using ActiveRecord Strict Loading to explicitly prevent N+1

Active Record strict loading is an awesome feature in Rails that can significantly improve your application’s performance by preventing N+1 queries. In the past days before Rails 6.1, We mainly used bullet gem to statically scan this. But now, It can be explicit in the code using strict_loading. Let’s see what strict loading is, how to use it, and see some examples.

What is N+1 issue 🔗

The N+1 issue is a common performance problem when using an Object-Relational Mapper (ORM) that follows Active Record pattern such as ActiveRecord in Ruby on Rails.

Consider a User model and a Post model where a user has many posts:

class User < ApplicationRecord
  has_many :posts
end

class Post < ApplicationRecord
  belongs_to :user
end

Now, suppose you want to display a list of users along with their posts:

users = User.all
users.each do |user|
  user.posts.each do |post|
    puts post.title
  end
end

Here it will first fetch the query for Users.all and then for each user it will generate one separate query for posts

SELECT "posts"."*" from posts where "posts"."user_id" = 1; -- AND SO ON

Finally due to this it will generate N queries for N users. Adding the query for User.all it will become N+1 queries. Hence we call it the N+1 query issue. This can have a huge impact on performance.

Solution for N+1 issue 🔗

Eager loading is a technique to solve the N+1 issue by loading all necessary data with fewer queries. In Rails, this can be done using includes, eager_load, or preload.

Using preload 🔗

The preload method tells ActiveRecord to load the associated records in as few queries as possible. It takes all use user_ids and loads all the posts matching those user_ids upfront.

users = User.preload(:posts).all
users.each do |user|
  user.posts.each do |post|
    puts post.title
  end
end

With preload, Rails will execute two queries:

1. SELECT * FROM users
2. SELECT * FROM posts WHERE user_id IN (1, 2, 3, ...)

loading all the required data upfront.

Using eager_load 🔗

The eager_load method forces Rails to use a single query with JOINs to load the associated records.

users = User.eager_load(:posts).all
users.each do |user|
  user.posts.each do |post|
    puts post.title
  end
end

This approach results in a single query using a LEFT OUTER JOIN:

SELECT users.*, posts.* FROM users
LEFT OUTER JOIN posts ON posts.user_id = users.id

Using includes 🔗

It’s the most commonly used method to handle the N+1 problem. It will detect the requirements and select one of the eager_load and preload internally. So it is considered as the best practice. It will use preload by default.

users = User.includes(:posts).all
users.each do |user|
  user.posts.each do |post|
    puts post.title
  end
end

What is Strict Loading? 🔗

In Rails, Strict loading is a mechanism that raises an error or logs a warning when you try to load an association that hasn’t been explicitly preloaded. This helps developers catch and fix N+1 query issues early, leading to more efficient database interactions.

It can be enabled globally, per model, per association, or on an individual query.

1. Enabling it to an association 🔗

class Author < ApplicationRecord
  has_many :books, strict_loading: true
end

2. Enabling it on a query 🔗

author = Author.first
author.strict_loading!

3. Enabling it on a model 🔗

class User < ApplicationRecord
  self.strict_loading_by_default = true
end

4. Enabling it globally 🔗

Set this in your application.rb

config.active_record.strict_loading_by_default = true

Modes 🔗

You can choose between two modes for strict loading:

1. Raise Exception (default): Raises a ActiveRecord::StrictLoadingViolationError when a lazy load occurs. this is the default
2. Log Warning: Logs a warning instead of raising an exception.

To enable logging mode globally, Use this in application.rb:

config.active_record.action_on_strict_loading_violation = :log

Checking Strict Loading Status 🔗

You can check if strict loading is enabled using the strict_loading? method:

user = User.first
user.strict_loading? # Returns true or false

Temporarily Disabling Strict Loading 🔗

You can temporarily disable strict loading for a block of code:

User.strict_loading do
  # Strict loading enabled here
end

User.strict_loading(false) do
  # Strict loading disabled here
end

Example 1 🔗

In this example I will show how to optionally enable this.

class Author < ApplicationRecord
  has_many :books
  has_many :published_books, -> { where(published: true) }, class_name: 'Book'

  def self.with_published_books
    includes(:published_books).strict_loading
  end
end

# Usage
authors = Author.with_published_books
authors.each do |author|
  author.published_books.each { |book| puts book.title }  # This works
  author.books.each { |book| puts book.title }  # This raises an exception
end

Example 2 🔗

This is an example of a N+1 issue that can cause huge money loss. In an e-commerce application, we might process orders like this

# Without strict loading
Order.where(status: 'pending').each do |order|
  total = order.line_items.sum(&:price)
  order.update(total: total, status: 'processed')
end

This innocent-looking code could lead to a severe performance issue with N+1 queries for line_items.

Now, with strict loading it will raise an error so we will be forced to add an includes:

# With strict loading
Order.includes(:line_items).strict_loading.where(status: 'pending').each do |order|
  total = order.line_items.sum(&:price)
  order.update(total: total, status: 'processed')
end

Strict loading would raise an error or log a warning if we forgot to include line_items, prompting us to fix the potential performance issue.

Conclusion 🔗

N+1 query issues are indeed very dangerous in any Rails applications. We are glad Rails has given us options like includes and strict_loading. strict_loading is a good secondary measure to includes. By raising errors or logging warnings when associations aren’t preloaded, it encourages developers to write more efficient queries from the start. Remember, while strict loading is a great tool, it’s not a silver bullet. It’s still important to understand your data model and query patterns to determine the most efficient way to load your data. Always consider your specific use case and test thoroughly to ensure optimal performance.