With the addition of RubyGems, we’ve solved the issue of direct dependencies. These are now specified with named versions. However, we still have a problem with our indirect dependencies (in other words: our dependencies’ dependencies).

We have no say in these. They have already been determined by the gem developer. But what happens if they’re incompatible?

In the cartoon above, both gem_pink and gem_green need to use gem_yellow.

# Gem Green is loaded first, and runs this code
gem "gem_yellow", "> 3.0"
# RubyGems finds gem_yellow v3.1 on the machine and activates it

# Gem Pink is loaded second, and runs this code
gem "gem_yellow", "< 2.0"
# RubyGems finds gem_yellow v1.6 on the machine and tries to activate it

When RubyGems comes across this second statement it will throw an “Activation Error”.

"Gem::LoadError (can't activate gem_yellow-1.6, 
already activated gem_yellow-3.1.)"

It’s a pretty reasonable complaint. A single Ruby process can’t load two different versions of the same gem¹.

So somehow, we need to agree on which version of Gem Yellow we’re going to use in the app.

Solving the problem

We can’t solve this by changing our indirect dependencies. They are already set in stone for each version of each gem. We can only do this by changing our direct dependencies. Sure, the latest version of Gem Green needs the latest version of Gem Yellow. But do we need the latest version of Gem Green? Can we downgrade and still make the app work?

Sorting this out used to be the job of the developer. If you added a new gem which upset the existing gems, it was your job to upgrade or downgrade gems until everything worked again.

There wasn’t a particularly scientific way of doing this - it was mostly trial and error.

But this was nuts! This is exactly the sort of problem that computers are really good at, and humans are pretty bad at.

Software to the rescue

And so it was that a new piece of software was developed to solve this problem - Bundler. Bundler is essentially a negotiator. She wants a solution that works for everyone.

Bundler required developers to create a new kind of file - a Gemfile - for each project. This set out two things:

• the dependencies that are needed to make the project work
• the range of versions of each gem that would be acceptable

Here’s an example Gemfile (along with the implied range of acceptable versions for each gem). This is a super small Gemfile with just two gems.

gem 'http', '~> 5.1' # works with: 5.1.0, 5.1.1, 5.2.0
gem 'phonelib', '~> 0.9' # works with: 0.9.0, 0.9.1, 0.9.2   

It uses the same RubyGems syntax that we’re now familiar with. But Bundler is actually doing something quite different with it.

RubyGems looks at each gem statement in isolation, and uses it to find a gem which satisfies the constraints on the existing machine.

Bundler doesn’t care about the gems that you have installed on your machine. Instead, she looks at all of the requirements of the project as a whole, and works out which set of versions will satisfy all these constraints at the same time.

Once she has a set of gems that work together, Bundler spits out a new file - a Gemfile.lock. This tells us the exact versions of each gem that should be used in the project.

Here’s the Gemfile.lock from our example Gemfile.

GEM
remote: https://rubygems.org/
specs:
  addressable (2.8.7)
    public_suffix (>= 2.0.2, < 7.0)
  base64 (0.2.0)
  capybara (3.40.0)
    addressable
    matrix
    mini_mime (>= 0.1.3)
    nokogiri (~> 1.11)
    rack (>= 1.6.0)
    rack-test (>= 0.6.3)
    regexp_parser (>= 1.5, < 3.0)
    xpath (~> 3.2)
  domain_name (0.6.20240107)
  ffi (1.17.0-arm64-darwin)
  ffi-compiler (1.3.2)
    ffi (>= 1.15.5)
    rake
  http (5.2.0)
    addressable (~> 2.8)
    base64 (~> 0.1)
    http-cookie (~> 1.0)
    http-form_data (~> 2.2)
    llhttp-ffi (~> 0.5.0)
  http-cookie (1.0.7)
    domain_name (~> 0.5)
  http-form_data (2.3.0)
  llhttp-ffi (0.5.0)
    ffi-compiler (~> 1.0)
    rake (~> 13.0)
  matrix (0.4.2)
  mini_mime (1.1.5)
  nokogiri (1.16.7-arm64-darwin)
    racc (~> 1.4)
  public_suffix (6.0.1)
  racc (1.8.1)
  rack (3.1.7)
  rack-test (2.1.0)
    rack (>= 1.3)
  rake (13.2.1)
  regexp_parser (2.9.2)
  xpath (3.2.0)
    nokogiri (~> 1.8)

You’ll see that it’s already pretty long and complicated, with just two gems.

If you graph out the relationships between the gems, as in the image below², you’ll see some of the potential issues.

So this isn’t a simple “tree” with a clear hierarchy, this is a crazy graph with links all over the place.

• our indirect dependencies also have dependencies
• sometimes our indirect dependencies will have depdendencies on our direct dependencies
• some dependencies have multiple “parents”

Bear in mind that in a typical Rails project, there will be hundreds of gems and connections between them. Not an easy task.

How does it work?

Unlike a hapless developer using trial and error and hoping for the best, Bundler has a systematic approach.

Bundler uses a ‘dependency resolution algorithm’ to find a set of direct and indirect dependencies that satisfy all the constraints³.

We can think of the algorithm as Bundler heading to shops with a shopping list. The requirements are her shopping list, and the basket represents the gems she believes will satisfy those requirements activate.

She goes though each item on the list in turn, looking for a gem version that satisfies the stated requirements. If she finds one that works, she puts it in her basket.

However, if she later finds a requirement that conflicts with one of the gems that she has already put in the basket, she “backtracks”. This means taking out of the basket every gem that she has put in since she added the conflicting gem, and starting over from that point. She then tries to find a new version of that original gem that satisfies both the original requirement, and the new conflicting requirement, before carrying on again.

It’s pretty laborious stuff⁴. Once she’s finished, she’ll download any new versions that are needed.

To ensure that the gems in your Gemfile.lock are the ones that are activated, you’ll need to add bundle exec before your Ruby command. If you forget to do this, RubyGems will just do it’s normal thing and use the latest versions of the gems installed on your system.

An unexpected advantage

Using a Gemfile.lock also has another advantage. Not only do we know that the dependencies will work together, it also means that everyone working on the project is using the exact same set of dependencies.

If we don’t commit the Gemfile.lock to our repository, but instead rely on each developer running bundle install, then it’s very possible that different machines would end up with slightly different versions of some gems⁵.

If we’re all running exactly the same code, we’ve just eliminated a whole class of bugs. 🎉

Hooray for Bundler

So Bundler is pretty great. Compared to the node and python ecosystems, I think the way manages Ruby dependencies is pretty good.

Of course, if you make an impossible request in your Gemfile, and specify two gems that have fundamentally incompatible dependencies - Bundler can’t help you. In this case Bundler will stop and let you know.

However, if there is a way to make your Gemfile requirements work, Bundler will find it.

We have already discussed:

• Part 1 - Loading Files In Ruby
• Part 2 - Loading Files in Rails (using Ruby’s in-built mechansims and the Autoloader)

But so far, we’ve not talked about gems. “Gem” is the Ruby community’s name for libraries - essentially other people’s code.

Using other people’s code instead of writing your own saves a lot of time, so it’s an important part of the story.

A trip back in time

Back in the 90s, Ruby was a cool programming language. Probably it’s what Thomas Anderson was using before he got kicked out of the Matrix.

Unfortunately, it was super janky to share Ruby code with other people. You could download code from other people’s websites. Then you chucked the code into a folder in the $LOAD_PATH[^setup-rb]. By default, most Ruby libraries were put into a single global folder called /site_ruby[^vendoring].

So, let’s say you needed a libary called calculator. You’d add this code to your project:

require "calculator"

Ruby goes hunting through all the directories in the $LOAD_PATH array to find a file named calculator.rb.

You’ve already downloaded calculator.rb, and put it in /site_ruby (and you’ve added /site_ruby to the $LOAD_PATH) - so you’re good to go 👍.

More versions, more problems

But now what if your colleague tells you about a new version of calculator that has just comes out? (It has a new “mutiply” feature that you’re excited about.)

So, now you have to:

• go to the calculator.rb website
• download the latest version of the code (hoping it’s the right one, there’s no standard way to check)
• delete all the files from the old calculator.rb (sorry, there’s no uninstall)
• copy the new code into site_ruby where you keep all your libraries

If you did all these steps, you were probably fine. But it was pretty tedious and manual. It was up to you to manage your $LOAD_PATH to ensure that there were no conflicts[^also-catastrophes].

RubyGems to the Rescue

So in 2003, RubyGems was released.

RubyGems makes things a lot easier:

• it hosts your code for you (at rubygems.org) 🥳
• it downloads the code and installs the code for you with a single terminal command 👍
• it lets you download specific versions each library 🥇
• it lets you specify which version of the library each piece of Ruby code needs 🎉
• it uninstalls each gem for you 😄

This is great. It removes a lot of the tedium. Instead of all the downloading and uninstalling, we can now just run

$: gem install calculator

and the code (and all its dependencies) get magically installed onto your machine for you to use.

Now if you want to use this gem, you can just add:

require "calculator"

Through a clever hack[^the-clever-hack], Ruby will pass over control to RubyGems and ask it to add a folder containing this gem to the $LOAD_PATH. Magic. ✨

Can we be more specific?

By default, RubyGems will download and use the latest version of a gem. If you want to download or use a specific version though, RubyGems lets you do that too.

To install a specific version:

$: gem install calculator -v 1.0

Then put this in your Ruby code to use that version:

gem 'calculator', '1.0'

By using the gem method in your Ruby code, you are “activating” the gem. This means that RubyGems is adding the folder containing the gem to the $LOAD_PATH. Then when Ruby requires the gem, it will find the right gem code.

You can even specify that you want a gem version that falls between two versions, and Ruby wil look within all your versions of that gem and return one that satisfies these constraints.

gem "calculator", ">= 1.1", "< 2"

Or you can use the pessimistic operator[^pessimistic-operator] ~> to allow upgrades but avoid breaking changes.

gem "calculator", "~> 1.1"

Rubygems will look for versions that satisfy these constraints among the versions that you have installed on your machine. So, if you start on a new project that specifies a version of calculator ~> 1.1, and you have v1.5 of calculator on your machine already, RubyGems will activate that version.

Putting it into practice

This was great for people who regularly ran more than one Ruby project on their computer. Now you could easily use v1.0 of a gem in one project and v2.0 in another.

Rails developed a syntax (which used RubyGems gem method under the hood) for specifying the gems that each project needed.

Rails::Initializer.run do |config|
  config.gem "nokogiri", :version => "1.4.2"
  config.gem "paperclip", :version => "2.3.3"
end

Often people used a rake gem:install command to install all of these gems with a single command.

However…

While RubyGems was pretty great, a pretty major problem still lurked. Solving this problem would require the final piece in our Ruby dependency management story.

Read the final part of our series here - Part 4 - Unpacking Bundler

# Gemfile

# Reduces boot times through caching; required in config/boot.rb
gem "bootsnap", require: false

But what does it do? And how does it work?

Bootsnap’s goal is to make Rails bootup as fast as possible (or ‘boot-snappily’ if you will).

The gem was created by Shopify, and solves a particular problem they had.

Shopify’s problem

Shopify have a lot of Ruby code. They also use Rails largely as a monolith, so much of this code sits within a single application - over 2.8 million lines. With all this application code, Rails takes a while to load.

If a unit tests uses Rails, the test runner needs to load the whole of Rails. So when a Shopify developer runs a single test, they need to load all that code. Before Bootsnap, this could take the Shopify monolith 25 seconds. This was a big problem that Shopify needed to solve¹.

How Bootsnap speeds things up

Bootsnap caches the results of the last time that your application loaded up². It does this in two ways:

• Caching the $LOAD_PATH
• Caching compiled Ruby code

Let’s look at each of these in turn.

Caching the LOAD_PATH

As you may remember from our previous article on loading files in Ruby, when Ruby comes across a require statement that it hasn’t seen before, it searches through the $LOAD_PATH to find that file. This may seem like a pretty speedy process. After all Ruby already has a list of places to look, how hard can it be?

Well, in some cases, there might be hundreds of entries in the $LOAD_PATH³. Ruby has to look through every one of these directories to find the matching file⁴.

Bootsnap reduces this work by scanning the files that are in each directory when the application boots, and keeping a record of which files are stored where. When it comes time to require a file, Bootsnap can tell Ruby exactly where it needs to look.

Much faster!

Caching compiled Ruby code

The other thing Bootsnap caches is compiled Ruby code. This might seem a bit strange if you’re used to thinking of Ruby as an ‘interpreted’ language.

How are intepreted languages different from compiled languages?

With an interpreted language, the code is read by the computer shortly before it is executed.

With compiled languages, the computer reads through the whole program and converts it into a different format before attempting to execute any of it. The new format is easier for the computer to understand (typically machine code).

For example, to run ‘C’ code, you’ll first need to compile it. This step generates machine code in an executable file. Then, to execute the code, you then need to run this generated file in a separate step.

But Ruby is also compiled

However, this distinction between compiled and interpreted languages is fuzzier than you might think.

Unlike C, Ruby doesn’t compile all your code ahead of time, but there is a compilation step involved in running Ruby code. When Ruby first loads a file, the code goes through a build process.

This turns the Ruby file into a nice neat little package that the Ruby engine can then run. Specifically, it turns it into bytecode.

What is bytecode?

Bytecode is a lower-level language, that can be interpreted by the Ruby Virtual Machine⁵.

You can do this compilation step yourself to see what your Ruby code looks like when it has been turned into bytecode.

# This method...
def hello
  puts "hello, world"
end

# When compiled by Ruby...
puts RubyVM::InstructionSequence.disasm(method(:hello))

## Turns into this bytecode...

== disasm: <RubyVM::InstructionSequence:hello@/tmp/method.rb>============
0000 trace            8                                               (1)
0002 trace            1                                               (2)
0004 putself
0005 putstring        "hello, world"
0007 send             :puts, 1, nil, 8, <ic:0>
0013 trace            16                                              (3)
0015 leave                                                            (2)

This process is deterministic. So a file compiled by the same version of ruby will always end up as the same bytecode.

(This RailsConf talk by Maple Ong gives a great description of how the Ruby Virtual Machine works.)

Given that this is an expensive process, it seems a little silly that we repeat the same task again and again, every time Rails boots up. Bootsnap has a different idea, why don’t we just cache the bytecode?

Bootsnap caches the bytecode in the tmp folder of your Rails application. Then, when Rails requires the file again, rather than having to re-compile the code into bytecode, it can just read the pre-compiled bytecode from disc. If the code changes, the cache will be busted, and the file will be re-compiled.

So how much faster does this make everything?

These two techniques together making rebooting Rails a lot faster. According to Shopify, using Bootsnap sped up their boot times from 25 seconds to 5 seconds. A pretty huge win for developer experience.

This is more noticeable with larger apps, but there’s a reasonable chance that this little gem is saving you a second or so every time you want to run a test or load up the Rails console.

So the next time you reload Rails, spare a thought for the friendly boot.

So we’ve gone through how Ruby loads files. But we’re still trying to work out how Rails does its magic trick of knowing about all your classes.

# app/controllers/invoice_controller.rb

class InvoiceController < ApplicationController # 🪄🌈❓❗❓
  # some business logic
end

I want to introduce you to two characters, let’s call them Classic and Zeitwerk. Here’s a picture of them:

As you can probably tell, they’re very different characters.

• Zeitwerk is organised, proactive and never makes mistakes. He’s a joy to work with (although some people think he’s a bit fussy).

• Classic is the complete opposite. He does everything last minute, makes a lot of mistakes and is always one step away from being fired.

In this post, we’re going to look at the way that Rails used to load files (Classic), then we’re going to look at the new way (Zeitwerk).

How autoloading used to work

Back in the day, Rails got Classic to do all the autoloading.

Classic’s approach for this was to read through all the code, and then when he hit a constant he hadn’t seen before, he quickly tried to find a file which contained that constant. For this, he relied pretty heavily on usingObject.const_missing.

I’m sorry, what’s missing?

const_missing is a cool ruby meta-programming trick that allows you to determine what happens when a constant is referenced that doesn’t exist. Normally, Ruby would raise a NameError if you refer to a non-existent constant.

But we can change that…

class Object
  def self.const_missing(c)
    puts "there is no such constant #{c}!"
  end
end

BANANA_NAME => "there is no such constant BANANA_NAME!"
APPLE_NAME => "there is no such constant APPLE_NAME!"
GRAPE_NAME => "there is no such constant GRAPE_NAME!"

Here, we’re re-opening the Object class and overriding const_missing. Now, instead of raising of raising an error, it prints a message to the user.

Where things really get interesting is where we use const_missing in combination with const_set. Object.const_set allows you to define (or redefine) any constant. So with these two methods working together, we can dynamically set a constant if we find that it doesn’t exist yet. 🤯

Imagine a customer walking into a shop and asking for something that the shop doesn’t have in stock. Then imagine the shop owner going into the back and quickly making the item on the spot. They bring it out and the customer buys it.

That’s basically what we’re doing here.

Here’s an example of this sort of thing:

class Object
  def self.const_missing(constant)
    # Turn `BANANA_NAME` into "Banana"
    fruit_name = constant.to_s.split('_').first.capitalize  
    new_value = "#{["Tim", "Ann", "Bob", "Ada"].sample} the #{fruit_name}" 
    
    # Sets the value of the constant (for any future references)
    const_set(constant, new_value)
    
    # Now we return the new value of `BANANA_NAME`
    new_value
  end
end

BANANA_NAME => "Tim the Banana"
APPLE_NAME => "Ann the Apple"
GRAPE_NAME => "Bob the Grape"

This is the trick that Rails (and Classic) relied on for many years.

So how did this work in Rails?

When Ruby hits ApplicationController, it tries to find this class using its Constant Lookup Hierarchy.

If Ruby hasn’t seen the constant before, then it will go all the way through the Constant Lookup Hierarchy, and find nothing. Ruby’s next step is to trigger Object.const_missing. This effectively hands the task of finding this class over to Rails, because Classic has overridden const_missing.

In Classic’s version of Object.const_missing , it would convert the class name to snake_case then try to load a file of that name.

Here’s a rough idea of how that worked:

class Object
  def self.const_missing(constant)
    # translates `ApplicationController` to `application_controller`
    underscored_version_of_constant_name = constant.to_s.underscore
    
    # here we `require` the file `application_controller`, thus loading 
    # the missing constant `ApplicationController`
    require constant.to_s.underscore 
    
    # now we fetch (and return) the - no longer missing! - `
    # ApplicationController` constant
    Object.const_get(constant) 
  end
end

What was wrong with how Classic did things?

This approach worked for many years, but it came with some problems.

I said earlier that Classic made a lot of mistakes. You can see a long list of these mistakes in the Rails Guide for version 5.2 (the last version that still exclusively used the Classic autoloader).

Here’s an example of one of them, imagine the code below:

# date.rb
class Date
  def initialize
    puts "Would you like to go on a date with me ❤️?"
  end
end

# supermarket/date.rb
module SuperMarket
  class Date
    def initialize
      puts "Would you like to buy one of these tasty dates? 🌴"
    end
  end
end

# supermarket/dried_fruits.rb
module Supermarket
  class DriedFruits
    def initialize
      @date = Date.new
    end
  end
end

So we have a couple of Date classes, which we definitely don’t want to mix up (🌴 != ❤️)

If we first call Supermarket::DriedFruits.new, before any references are made to either of the Date classes, everything works as expected:

> Supermarket::DriedFruits.new
Would you like to buy one of these tasty dates 🌴?
✅

Ruby doesn’t know about either of the Date classes, so we trigger const_missing. The Classic autoloader does its thing and loads the correct class.

But! If you first referenced one of the top-level (romantic) dates, then we get different behaviour.

> Date.new
Would you like to go on a date with me ❤️?
✅

> Supermarket::DriedFruits.new
Would you like to go on a date with me ❤️?
❌

This happens because Classic autoloading is only triggered if Ruby is unable to first find a constant of that name in it’s constant lookup hierarchy. In the above example, Ruby looks in the following places:

Supermarket::DriedFruit::Date # nope 
Supermarket::Date # nope - `supermarket/date.rb` has not been loaded yet!
::Date # aha! I found a Date class!

It already knows about a Date class, so we never get to const_missing and supermarket/date.rb doesn’t get loaded. 😞

Having the value of constants depend on the order that are referenced is some pretty crazy behaviour.

So the old autoloader was a bit of a hack. It also completely ignored the perfectly nice autoload functionality that already existed in Ruby (as we discussed in Part 1…). You’d think if you were going to implement an autoloader you would use autoload - right?

And, as we’ll see, that’s exactly how Zeitwerk likes to do things.

So how does zeitwerk do it?

Zeitwerk doesn’t wait to find a constant that is has not seen before. Zeitwerk is proactive.

Before looking at any of the application code, Zeitwek first scans all the autoload paths when the app loads.

It then creates a bunch of autoload statements, based on the files it has seen, and assumptions about what the classes in those files are likely to be called.

# Zeitwerk sees:
`application_controller.rb`
# Zeitwerk assumes this class exists:
=> ApplicationController
# So Zeitwerk autoloads:
autoload(:ApplicationController, 'application_controller.rb')

# Zeitwerk sees:
`supermarket/date.rb`
# Zeitwerk assumes this class exists:
=> Supermarketr::Date
# So Zeitwerk autoloads:
Supermarket.autoload(:Date, 'supermarket/date.rb')

So to go back to our earlier example, this is Ruby’s new search through the Constant Lookup Hierarchy.

Supermarket::DriedFruit::Date # nope 
Supermarket::Date # I have an autoload for that! Let's load the file!

By using Ruby’s in-built autoload functionality, rather than hacking around it, we bypass all the problems we saw with the Classic autoloader.

By creating these autoload statements when your app first boots, the required classes can then be loaded while Ruby is searching the Constant Lookup Hierarchy, rather than waiting for this search to finish

So, the new autoloader is much better than the old one, and everyone is happy about it. 🎉🌈

In 2021, the Rails wizard finally fired Classic (Rails 7 no longer supports the classic autoloader), and the long list of autoloading “Gotchas” was removed from the Rails guides.

One small problem

However, remember I mentioned before that Zeitwerk was a bit fussy? It’s not his fault really, it’s just the way he likes to work.

You see, while Classic found the class name, then guessed the file name:

# Classic
"ClassName".underscore => "class_name"

Zeitwerk finds the file name and then guesses the class name:

# Zeitwerk
"class_name".camelize => "ClassName"

But actually, it’s harder to guess class names from file names than the other way round, because of Rails conventions about file and class naming. Files are always follow the snake_case format, eg api.rb. However, the class name in this file could be either API or Api.

It’s not a big problem. Zeitwerk can handle either situation, you just need to let him know 🙂.

There are a few other things to be aware of if you’re upgrading a Rails app that currently uses the classic autoloader, which you can find here.

Still hungry for more autoloading?

If you want to keep learning about how Rails loads files, below is a great talk from Xavier Nora (who created zeitwerk) about how he did it.

(If you’re something of a Ruby magician already, and you’re just looking for the Rails bits, you can skip straight ahead to Part 2 - Loading Files in Rails)

Have you ever wondered: “How does Rails knows about all the classes in my app?”

Look at the code below:

# app/controllers/invoice_controller.rb

class InvoiceController < ApplicationController
  # some business logic
end

How does your code know about ApplicationController? You never explicitly told Rails that ApplicationController existed, but it didn’t blink when it saw it for the first time.

Rails just seems to have a sixth sense for these sorts of things.

This will be extra confusing if you come from a language like Python or Javascript. In those languages, you generally need to add import statements to the top of every file specifying where to find code that you want to use (which can get pretty tedious).

How come Rails can get away without import statements? And is this a good thing or a bad thing? As we’ll discover, Ruby and Rails both have a few tricks that help us get around the tediousness of manually importing files

How we load files in Ruby

If you’ve written simple Ruby programs you’ll know that we do need to use the require method to load code from other files.

Here’s a nice Book class I wrote:

# book.rb

class Book
  attr_reader :title

  def initialize(title)
    @title = title
  end
end

And here’s a Bookshelf class (which obviously needs to know about Books):

# bookshelf.rb

class BookShelf
  attr_reader :books

  def initialize
    @books = [Book.new('Catch 22')]
  end
end

p BookShelf.new.books.map(&:title)

The two files are sitting next to each other in the same directory:

$: tree
.
├── bookshelf.rb
└── book.rb

Now, if we try and run $: ruby bookshelf.rb, we’ll get a NameError.

bookshelf.rb:7:in `initialize': uninitialized constant BookShelf::Book (NameError)

    @books = [Book.new('Catch 22')]

Ruby has never heard of the Book class. 😞

But! If we add require './book' to the top of bookshelf.rb, the code runs successfully.

$: ruby bookshelf.rb
["Catch 22"]

So how does this work?

What does require do?

To understand require, we need to understand the $LOAD_PATH. The $LOAD_PATH is a Ruby global variable (notice the $) which references an array of strings. Each of the strings is the file path to a directory.

Here’s my $LOAD_PATH:

irb(main):003:0> puts $LOAD_PATH
/opt/homebrew/Cellar/rbenv/1.2.0/rbenv.d/exec/gem-rehash
/Users/olly/.rbenv/versions/2.7.2/lib/ruby/site_ruby/2.7.0
/Users/olly/.rbenv/versions/2.7.2/lib/ruby/site_ruby/2.7.0/arm64-darwin20
/Users/olly/.rbenv/versions/2.7.2/lib/ruby/site_ruby
/Users/olly/.rbenv/versions/2.7.2/lib/ruby/vendor_ruby/2.7.0
/Users/olly/.rbenv/versions/2.7.2/lib/ruby/vendor_ruby/2.7.0/arm64-darwin20
/Users/olly/.rbenv/versions/2.7.2/lib/ruby/vendor_ruby
/Users/olly/.rbenv/versions/2.7.2/lib/ruby/2.7.0
/Users/olly/.rbenv/versions/2.7.2/lib/ruby/2.7.0/arm64-darwin20
=> nil

When we call the require method below, Ruby will look in each of the directories listed in the $LOAD_PATH to try and find a file called byebug.rb in them.

irb(main):002:0> require 'byebug'
=> true

We can tell that it found one, because it returns true. If Ruby can’t find a byebug.rb file within those directories, it will throw a LoadError.

We can check that there definitely is a file called byebug.rb, if we run the following command from the console:

$: gem which byebug
/Users/olly/.rbenv/versions/2.7.2/lib/ruby/gems/2.7.0/gems/byebug-11.1.3/lib/byebug.rb

There it is! Ruby wasn’t lying.

Finding your own code

This is all nice and simple for gems like byebug. These files are automatically added to directories in the $LOAD_PATH. But what if we want to require some of our own code?

If you look at the directories listed in the $LOAD_PATH, you won’t see the directory of the current file¹. This is a shame if you want to make an app with a bunch of related files and put them all in the same directory. Our Ruby files won’t be able to talk to each other.

There are a few ways to get around this, but first we need to understand the difference between:

• the working directory - the directory where we initiated the Ruby process (by running ruby ruby_project/bookshelf.rb) and
• the directory of the current file - where the Ruby file is located

Option 1 - require_relative

One option is to use require_relative.

# bookshelf.rb

require_relative 'book'

require_relative completely ignores the $LOAD_PATH and instead just searches for files in the current file’s directory (no matter what your working directory is).

Option 2 - plain, old require (with a '.')

Alternatively, we can use require with the ./ the syntax that we tried above.

# bookshelf.rb

require './book'

Adding ./ ( or ../) to the start of the argument we pass to require tells Ruby to search for the file in your working directory (where you started the ruby process). It also means Ruby completely ignores the $LOAD_PATH again.

Not really sure why you would want this behaviour (require_relative seems much simpler), but it’s there².

Couldn’t we just amend the $LOAD_PATH?`

Alternatively, rather than bypassing the $LOAD_PATH as in the options above you can edit the $LOAD_PATH to add the current file’s directory to it (it’s just a mutable array after all).

Option 3 - edit the $LOAD_PATH

Here is a common pattern for this:

absolute_path_for_directory_of_current_file = File.expand_path('..', __FILE__)
$LOAD_PATH.unshift(absolute_path_for_directory_of_current_file)

Let’s unpack this a little bit.

__FILE__ is a relative path to the current file (eg bookshelf.rb) from the current working directory. So, the value of __FILE__ is going to be different depending on where you’re running your ruby process.

File.expand_path is a clever method that takes relative filenames and turns them into absolute filenames. So File.expand_path('..', __FILE__) will give us a an absolute path to the directory of the current file³. 🎉 Because it’s an absolute path, it will always be the same regardless of where on your machine you start your Ruby process.

Then we use Array#unshift to shove this absolute filename at the beginning of the $LOAD_PATH. Now, when you add require 'book' to the top of bookshelf.rb, Ruby will find our file when it goes hunting through the $LOAD_PATH.

Which approach should I use?

While require_relative works for small projects, amending the $LOAD_PATH is probably preferable for larger projects. With a lot of files spread across a number of directories, you probably don’t want the complexity of working out the relative file path for each of them.

What about load?

You may also have come across load. load does something very similar to require, but with a couple of differences.

require will only load a file once, no matter how many times you ask Ruby to reload it. (Ruby stores a list of the files it has already required in the $LOADED_FEATURES global variable. If a file is already in that list require will not reload it again.)

load isn’t as fussy. It will keep loading and reloading files as often as you ask it to (and doesn’t even bother consulting $LOADED_FEATURES).

So require tends to be used for loading libraries and modules (that you probably only want to load once), whereas load is used to load things that might change frequently (like configuration classes).

What about autoload?

OK, we’re nearly there. But to understand how Rails does it’s file loading magic, we need to understand one more aspect of file loading in Ruby: the autoload method.

autoload does an interesting thing that we haven’t seen yet: lazy loading.

When Ruby encounters the autoload keyword, it doesn’t load the file immediately. Instead it saves a reference to the two arguments it is passed: a filename and a class name. When it encounters that class for the first time, it will then load the file (and will assume that it will find the class defined within it).

Going back to our BookShelf example, we can pass autoload a symbol of the class we want to load (in this case :Book). Then if it encounters a class of that name, it will require the file that we passed in as the second argument (./book.rb).

# book.rb

class Book
  attr_reader :title

  puts("Loading the Book class")

  def initialize(title)
    @title = title
  end
end

# bookshelf.rb
autoload :Book, './book.rb'

class BookShelf
  attr_reader :books

  def initialize
    puts("Initializing a BookShelf")
    @books = [Book.new('Catch 22')]
  end
end

p BookShelf.new
=> "Initializing a BookShelf"
=> "Loading a Book class" # <- we don't load the Book class until Ruby encounters the Book constant for the first time

This allows lazy loading, which I guess is good if you’re worried about startup speed for your program. It means you don’t have to load a bunch of files before running any code.

(As an aside, autoload actually requires files rather than loading them. In other words, it only loads each file once. So it should be called autorequire - the world is imperfect in so many ways.)

We made it

Ok, there we are. Hopefully you now understand the different methods that Ruby uses to load files. But we’re really only just getting started.

In Part 2 of this post we’re going to look at how Rails uses these Ruby tricks (and a few others) to magically find classes that it has never even heard of before.

You can find Part 2 here