Examining The Internals Of The Rails Request/Response Cycle

??? words · ??? min read

Come with me on a journey through the internals of Rails, as we trace a request from the web server to the controller action method, and follow the response back again. This will give you a glimpse of how Rails works under the hood, hopefully exposing some of the “magic” of the framework. We will also get to see some of the design/architectural decisions at the core of Rails.

This analysis was conducted against a newly generated Rails 5.0.0.1 app.

Prerequisite: Rack

If you already understand Rack, you can skip this section.

Because Rails is built on top of Rack, this article isn’t going to make much sense unless you know Rack. I highly suggest reading the official specification (it’s fairly short) but I will give you a quick crash course here.

Rack apps are objects that handle web requests, and return responses. This is a simple Rack app class:

class ComplimentApp
  def call(env)
    [200, {'Content-Type' => 'text/plain'}, ["I like your shoes"]]
  end
end

There is only one method that a rack app must implement: the #call method. This method receives the web request as an argument and returns an array as the response.

The request env should not be confused with the ENV constant. The request env represents a HTTP request. The ENV constant is a global built in to Ruby, and contains all the environment variables for the current process.

The web request is represented as a hash. This hash is referred to as the “request environment,” usually shortened to just “env.” It contains all the information about the request, like the HTTP method, the server’s hostname, the URL path, and so on.

The response is represented as a three-element array. The first element is the HTTP status code. The second element is a hash of HTTP headers. The third element is the body of the response, which can be any object that responds to #each, yielding only strings. In the example above, the response body is an array with a string in it, which is valid.

This is how Rails, and most other Ruby web frameworks, interact with web servers. When the web server receives a HTTP request, it converts that request into an env hash, and calls your rack app. The rack app returns a response, which the web server sends back to the browser.

1. The Rack Entry Point

Rack apps define their entry point in the config.ru file, and Rails is no different. Looking inside the default config.ru file we find this:

require_relative 'config/environment'
run Rails.application

The Rails.application call returns an instance of your application class, defined in config/application.rb. I named my app “Endoscopy”, so my object is an instance of Endoscopy::Application.

As with all rack apps, this application object must respond to the #call method. This is implemented in the Rails::Engine class, which your application class inherits from, and it looks like this:

def call(env)
  req = build_request env
  app.call req.env
end

The build_request call merges a bunch of values into the env. These values are accessible to all the middleware and controllers. Here is a list of keys that are added:

  • action_dispatch.parameter_filter
  • action_dispatch.redirect_filter
  • action_dispatch.secret_token
  • action_dispatch.secret_key_base
  • action_dispatch.show_exceptions
  • action_dispatch.show_detailed_exceptions
  • action_dispatch.logger
  • action_dispatch.backtrace_cleaner
  • action_dispatch.key_generator
  • action_dispatch.http_auth_salt
  • action_dispatch.signed_cookie_salt
  • action_dispatch.encrypted_cookie_salt
  • action_dispatch.encrypted_signed_cookie_salt
  • action_dispatch.cookies_serializer
  • action_dispatch.cookies_digest
  • action_dispatch.routes
  • ROUTES_70198900278360_SCRIPT_NAME
  • ORIGINAL_FULLPATH
  • ORIGINAL_SCRIPT_NAME

The application object is a kind of rack middleware itself. It adds things to the env hash, and then passes it along to the next rack app. In this case, the next app is an instance of Rack::Sendfile, which indicates that the request is entering the middleware stack.

2. Middleware

Rack middleware are app objects that call other app objects. This allows middleware to do two things:

  • Modify the env before it is passed to the next app
  • Modify the response returned from the next app

Rails provides a bunch of middleware that are enabled by default. Here is an ordered list of all the middleware in the development environment:

  1. Rack::Sendfile – Makes responses from files on disk
  2. ActionDispatch::Static – Responds to requests for static files
  3. ActionDispatch::Executor – Undocumented ¯\_(ツ)_/¯
  4. ActiveSupport::Cache::Strategy::LocalCache::Middleware – Response caching
  5. Rack::Runtime – Response time measurement
  6. Rack::MethodOverride – Overrides the HTTP request method based on the _method param
  7. ActionDispatch::RequestId – Gives each request a unique id
  8. Sprockets::Rails::QuietAssets – Silences logging on requests for sprockets assets
  9. Rails::Rack::Logger – Request logging
  10. ActionDispatch::ShowExceptions – Makes responses for unhandled exceptions
  11. WebConsole::Middleware – Interactive console for running code on the server
  12. ActionDispatch::DebugExceptions – Logging and debug info pages for unhandled exceptions
  13. ActionDispatch::RemoteIp – Determines the IP address of the client
  14. ActionDispatch::Callbacks – Runs callbacks before/after/around each request
  15. ActiveRecord::Migration::CheckPending – Raises an exception if there are pending migrations
  16. ActionDispatch::Cookies – Cookie serialization and encryption
  17. Rack::Session::Abstract::Persisted – Session management
  18. Rack::Head – Removes response body from HEAD requests
  19. Rack::ConditionalGet – HTTP caching
  20. Rack::ETag – HTTP caching

Once the request has passed through all 20 middleware objects, it then enters the router.

3. Routing

All of your app’s routes are stored inside an instance of ActionDispatch::Routing::RouteSet at runtime. This object also doubles as a rack app that dispatches the requests to the correct controller and action method. This is the app that gets called after all the middleware.

The first thing that happens is that the request env gets converted into an ActionDispatch::Request object. Whereas env hashes are generic representations of a web request, these request objects contain functionality that is specific to Rails.

The request object is then used to lookup the correct route to dispatch to, which includes the corresponding controller class and action method.

Next, an empty response object is created, which is an instance of ActionDispatch::Response.

Lastly, the controller is invoked via the #dispatch class method, like so:

controller_class.dispatch(action, request, response)

Routing is implemented in a way that is similar to Rack middleware, but not quite the same. The router does not dispatch to other rack apps, it dispatches to controller classes. Controller classes are not rack apps – they return Rack-compatible responses, but are called via a different API.

4. The Controller

The entry point for the controller is the #dispatch class method, which is implemented in ActionController::Metal. This is the implementation:

def self.dispatch(name, req, res)
  if middleware_stack.any?
    middleware_stack.build(name) { |env| new.dispatch(name, req, res) }.call req.env
  else
    new.dispatch(name, req, res)
  end
end

In addition to the middleware stack that we saw earlier, each controller class can have its own stack of middleware. In the new Rails 5 app that I’m examining, however, controller classes have no middleware by default.

Middleware or not, the controller class creates a new instance of itself, and forwards the arguments to the #dispatch instance method. This means each request is handled by a new, clean controller object.

Here is the implementation of the #dispatch instance method, from ActionController::Metal:

def dispatch(name, request, response)
  set_request!(request)
  set_response!(response)
  process(name)
  request.commit_flash
  to_a
end

The first thing the controller object does is store the request and response objects. This makes sense, as these two can be accessed from any method within the controller.

Then comes the process(name) call, which eventually calls the correct action method – code that you have written yourself. Before that, however, the process call makes its way through several places in the controller’s very deep inheritance hierarchy. How deep is “very deep”? Have a look for yourself:

ExampleController.ancestors.size #=> 68
ExampleController.ancestors #=>
# [ExampleController, #<Module:0x007fe42c6632b8>, ApplicationController,
# #<Module:0x007fe42b6acd10>, #<Module:0x007fe42b855018>,
# #<Module:0x007fe42b855040>, ActionController::Base, Turbolinks::Redirection,
# Turbolinks::Controller, ActiveRecord::Railties::ControllerRuntime,
# ActionDispatch::Routing::RouteSet::MountedHelpers,
# ActionController::ParamsWrapper, ActionController::Instrumentation,
# ActionController::Rescue,
# ActionController::HttpAuthentication::Token::ControllerMethods,
# ActionController::HttpAuthentication::Digest::ControllerMethods,
# ActionController::HttpAuthentication::Basic::ControllerMethods,
# ActionController::DataStreaming, ActionController::Streaming,
# ActionController::ForceSSL, ActionController::RequestForgeryProtection,
# AbstractController::Callbacks, ActiveSupport::Callbacks,
# ActionController::FormBuilder, ActionController::Flash,
# ActionController::Cookies, ActionController::StrongParameters,
# ActiveSupport::Rescuable, ActionController::ImplicitRender,
# ActionController::BasicImplicitRender, ActionController::MimeResponds,
# AbstractController::Caching, AbstractController::Caching::ConfigMethods,
# AbstractController::Caching::Fragments, ActionController::Caching,
# ActionController::EtagWithTemplateDigest, ActionController::ConditionalGet,
# ActionController::Head, ActionController::Renderers::All,
# ActionController::Renderers, ActionController::Rendering, ActionView::Layouts,
# ActionView::Rendering, ActionController::Redirecting,
# ActiveSupport::Benchmarkable, AbstractController::Logger,
# ActionController::UrlFor, AbstractController::UrlFor,
# ActionDispatch::Routing::UrlFor, ActionDispatch::Routing::PolymorphicRoutes,
# ActionController::Helpers, AbstractController::Helpers,
# AbstractController::AssetPaths, AbstractController::Translation,
# AbstractController::Rendering, ActionView::ViewPaths,
# #<Module:0x007fe42b67f478>, ActionController::Metal, AbstractController::Base,
# ActiveSupport::Configurable, ActiveSupport::ToJsonWithActiveSupportEncoder,
# Object, ActiveSupport::Dependencies::Loadable, PP::ObjectMixin,
# JSON::Ext::Generator::GeneratorMethods::Object, ActiveSupport::Tryable, Kernel,
# BasicObject]

Below is the call stack between the #dispatch instance method and the action method. Keep in mind that these are all methods called on a single controller object.

  1. ActionView::Rendering#process – Sets/restores global I18n config
  2. AbstractController::Base#process – Resets @_action_name and @_response_body, and raises an exception if the action is not found.
  3. ActiveRecord::Railties::ControllerRuntime#process_action – Resets runtime measurements for ActiveRecord query logging
  4. ActionController::ParamsWrapper#process_action – Wraps params inside a hash
  5. ActionController::Instrumentation#process_action – Per-request instrumentation
  6. ActionController::Rescue#process_action – Calls rescue_from blocks when exceptions are raised.
  7. AbstractController::Callbacks#process_action – Runs callbacks before/after/around the action method
  8. ActionController::Rendering#process_action – Sets self.formats
  9. AbstractController::Base#process_action – Does nothing except call #send_action
  10. ActionController::BasicImplicitRender#send_action – Renders the implicit view, if nothing was explicitly rendered in the action method

After all of that, the action method is finally called. On to rendering!

5. Rendering

This is the controller implementation:

class ExampleController < ApplicationController
  def index
    render plain: 'Hello, world!'
  end
end

I’m going to skip over the details of view rendering – that could be an article in itself.

No matter what you render, the end result is stored in response.body. It also sets the Content-Type HTTP header on the response object, to text/plain in this case.

That’s it for the controller. Now we unwind the stack, and see what happens on the way out.

6. Leaving The Controller

After control has left the action method, the call stack unwinds back through the controller class heirarchy. In order:

  1. after_action callbacks are run.
  2. Instrumentation finishes.
  3. Global I18n config is restored to its original value.
  4. Flash messages are stored within the session.

Lastly, the controller provides a return value by calling response.to_a. This converts the response object into a Rack-compatible response array, which then flows back through the router and all the middleware.

7. Leaving Routing

Routing simply passes the controller’s return value back to the middleware, unless the response headers include X-Cascade: pass. When a contoller returns this header, it is basically saying “nah, that request isn’t for me, so try find someone else to handle it.” The router then attempts to find another matching route, and re-dispatch the request. This seems to be a little-known, and rarely used feature.

8. Leaving Middleware

After leaving routing, the Rack response array is returned through all the middleware. During this phase, middleware have the opportunity to modify the status code, headers, and response body. All the middleware are explained earlier in this article, but the notable response modifications are:

  1. The ETag header is set, based on the response body
  2. The response body may be removed entirely, based on HTTP caching headers
  3. The session is “committed,” meaning that is serialized and stored in a cookie
  4. Cookies are serialised, and added to the response headers
  5. Information about the request is logged out

After all the middleware, the Rack-compatible response is returned from your app to the web server. There, it is serialized into a HTTP response string, and sent back to the client.

Conclusion

The Rails request/response cycle is heavily based upon Rack. The root application object is a rack app, and a bunch of Rails functionality is implemented as Rack middleware.

Routing dispatches requests to a class method on the controller. The controller class then instantiates itself, and dispatches the request to the newly created object.

While controllers are not Rack apps, they are quite similar. Action methods do not return a response directly – they mutate a response object, which is later converted into a Rack-compatible response array, and then returned. There is potentially a Rack middleware stack specific to each controller class.

Controllers inherit from 67 ancestor classes/mixins. This probably contributes to the impression that Rails is full of “magic.”

Got questions? Comments? Milk?

Shoot an email to [email protected] or hit me up on Twitter (@tom_dalling).

← Previously: The Pure Function As An Object (PFAAO) Pattern

Next up: Raise On Developer Mistake →

Join The Pigeonhole

Don't miss the next post! Subscribe to Ruby Pigeon mailing list and get the next post sent straight to your inbox.