Rubyisms in Rails

Next Page

Book Cover
Rubyisms in Rails
By Jacob Harris
...............................................
Publisher: Addison Wesley Professional
Pub Date: July 28, 2006
Print ISBN-10: 0-32-147407-4
Print ISBN-13: 978-0-321-47407-0
Pages: 54

Table of Contents
Overview

Rubyisms is an examination of how the style of Ruby informs the design of Rails. In particular, it looks at a few specific examples of how Rails' internal code is implemented in Ruby to instruct about Ruby's design principles. The main goal is simply aesthetic appreciation. But, if you are a beginning programmer in Rails who is stymied in your understanding of Rubyor an intermediate Rails developer still writing code that looks like Ruby-tinged PHP or Javathis Shortcut will hopefully impart enlightenment and inspiration about the Ruby way of programming. It also reveals how the revolutionary design of the Rails framework can only be built upon the beauty of Ruby.

Next Page


Table of Contents

Previous Page
Next Page

Book Cover
Rubyisms in Rails
By Jacob Harris
...............................................
Publisher: Addison Wesley Professional
Pub Date: July 28, 2006
Print ISBN-10: 0-32-147407-4
Print ISBN-13: 978-0-321-47407-0
Pages: 54

Table of Contents

Copyright
 | The Philosophy of Ruby
 | Objects, Classes, and Modules
 | Duck Typing
 | Symbols
 | Blocks
 | Metaprogramming
 | Domain-Specific Languages
 | Conclusion
 | Acknowledgments

Previous Page


Copyright

Previous Page
Next Page


Previous Page
Next Page

This is not a programming guide. Yes, there is code, but this digital shortcut is not about coding. Rather, this is really a short about design, about philosophy, and about appreciating beauty. This short looks at the way in which the grace of Ruby informs the design of Rails.

This is a surprisingly underappreciated concept. Of course, many people have already written about what they find beautiful in Railsaspects of style we will explore furtherbut what the newcomer to Rails might miss is that Rails's design directly reflects Ruby's underlying beauty. Although the name Ruby on Rails suggests that the Rails model could be developed on top of other languages (Perl on Rails? Visual Basic on Rails? Cobol on Rails?), such development is impossible without losing some things in the process. Sure, any Rails port or clone can share certain outward similarities (in the way a cheap lemon and a well-designed automobile both have steering wheels and engines and doors and seats), but Rails's success is as much a measure of Ruby's expressive power as it is of Rails's conventions. Sadly, however, this point is usually lost on people coming to Rails from other more limited languages (PHP, Perl, C++, Java, and so on); but then again, it is never really a connection that is stressed.

This digital shortcut is not a guide for learning Rails; you can already find many excellent books on that topic. However, all of them are designed to quickly drive their readers toward proficiency in Rails. They just do not have the luxury of exploring the Ruby foundation of Rails in any depth. Instead, they usually have to treat distinctive Ruby featuresthings such as symbols, blocks, and metaprogrammingin passing (if at all), like quaint towns and scenic overlooks glimpsed briefly from a car hurtling forward on an interstate highway. The problem here is that the beauty of Ruby gets short shrift. Therefore, many people who have come to coding in Ruby from Rails do not really get Ruby as much as they would like. That is where this digital shortcut comes in.

We're taking the scenic route instead. I do not cover a hitlist of topics either quickly or in depth. I do not teach you Ruby or Rails in 10 days or your money back. I assume you already know how Rails works. By extension, I also assume you have some experience with Ruby but do not understand it entirely; that is, your Ruby coding resembles more Ruby-inflected PHP or Ruby-tinged Java, and you want to become more articulate in the Ruby language. This is not an in-depth guide to the Ruby language, however. Instead, I share with you a few examples of code from within the Rails source code (and perhaps a few other places) that I think are beautiful, some of which also conveniently cause confusion and bewilderment to newcomers. The main goal is aesthetic appreciation and philosophical alignment, nothing more. However, I hope this approach affects you in subtler waysimproving your understanding of why Ruby works and how you can become a better programmer in it.


The Philosophy of Ruby

If you have written any application in Rails, you have (of course) coded in Ruby. You probably have a good sense of Ruby's syntax by now and a working knowledge of some of the core library. However, you might have missed some things along the way. You have seen the whats and the hows of Ruby but probably not the whys. Specifically, you might not have been exposed to the philosophy behind why Ruby does things the way it does. So, let's start by exploring where Ruby came from and what it tries to accomplish.

For me, the purpose of life is, at least partly, to have joy. Programmers often feel joy when they can concentrate on the creative side of programming, so Ruby is designed to make programmers happy.

Yukihiro Matsumoto

Like Perl, Ruby is what Larry Wall termed a "postmodern programming language." This means that Ruby's creator, Yukihiro Matsumoto, created Ruby's syntax and features by deconstructing and recombining the functionality he admired the most in other existing languages. Perl was chief among his influences (hence Ruby's gemlike name), but useful features and functionality were also integrated from such languages as disparate as Ada, C++, CLU, Lisp, Java, Python, PHP, and Smalltalk. Despite all these wide-ranging influences, Ruby has a tightly consistent design in its syntax and libraries, reflecting the conscious effort of its creator to keep to guiding principles. In an introduction to the language from 2000, Matsumoto articulated three guiding principles that Ruby strives for:

  1. Conciseness Writing code in Ruby should involve the minimum amount of commands necessary. Code should be terse but also understandable.

  2. Consistency Ruby coding should follow common conventions that make coding intuitive and unambiguous.

  3. Flexibility There is no one right way. You should be able to pick the best approach for your needs and be able to even modify the base classes if necessary.

By keeping to these principles, Ruby assimilates the best features from many different languages while maintaining a clean, simple, and harmonious style. Understanding these principles is the key to understanding how to code effectively in Ruby. I call examples of this style Rubyisms, and this digital shortcut takes a closer look at selected Rubyisms to illustrate how Ruby works.

Rubyisms can be subtle. When you start learning Rails, it is easy to miss some of the first examples of Rubyisms you see. For instance, you might be working on a sample program (perhaps a blogging engine) and want to highlight titles posted less than 30 minutes ago. In Rails, you could express this idea simply with the following code:

@highlight_title = true if (posting_time < 30.minutes.ago)

Let's look at that again, thinking about the three principles described earlier. For starters, it certainly is concise. Suppose, however, that you had to do this instead by instantiating a TimeDelta object and then calling methods against it; that might get to you the same result, but the route is far less direct. It is also consistent, in that it is exactly clear to the writer and the reader what this code is doing, with no ambiguity or side effects to worry about. But what about flexibility? Well, this is not the "official" Ruby mechanism for doing date comparisons. Another person using Ruby might conceivably decide to code this time comparison as follows:

def mindiff(now, later)
  (now  later).abs / 60 # times are in seconds, abs is absolute value
end

if mindiff(Time.now, posted_time) < 30
  @highlight_title = true
end

And he is certainly free to if he feels more comfortable with that way. This principle of flexibility is directly inherited from Perl, where it has long been expressed with this phrase: There's more than one way to do it. The beauty of this approach is that you can program in your most comfortable style. If you want to write Java-like Ruby, you can; no Ruby police will make judgments on you and declare huffily that you should implement all datetime comparisons using the official DateTimeDiff class defined in one of the auxiliary libraries. However, just because there are more ways than one to do it does not mean that all ways are equally beautiful. You can write Java-like Ruby, but you'll do much better in the long run if you don't. Arguably, the first time comparison example is clearly better than the second (because it is more concise and understandable). There are more ways than one to do it, but some ways are better than others, and those ways are Rubyisms.

Ruby inherited the Perl philosophy of having more than one way to do the same thing. I inherited that philosophy from Larry Wall, who is my hero actually. I want to make Ruby users free. I want to give them the freedom to choose. People are different. People choose different criteria. But if there is a better way among many alternatives, I want to encourage that way by making it comfortable. So that's what I've tried to do.

Yukihiro Matsumoto

One other concept often used to characterize Ruby is the principle of least surprise. This is commonly (but mistakenly) understood to mean that good Ruby code works exactly the way we expect it to, displaying both conciseness and consistency. To demonstrate, here are some more examples of Rubyisms:

return if no_harm?

puts "No Comments" unless @comments.any?

def quality
  case score
      when (67..100) then :good
      when (34..66) then :okay
      when (0..33) then :bad
      else :unknown
  end
end

def category_links(article)
  "Posted in " + article.categories.collect { |c|
        link_to c.name,
     { :controller=>"articles", :action => "category",
       :id => c.permalink },
     :rel => "tag"
  }.join(", ")
end

Even if you had barely any understanding of Ruby, you should be able to look at the first three examples and grasp immediately what they are doing. Like that other high-level dynamic interpreted language Python, Ruby is sometimes lovingly described as "pseudo-code that runs." The fourth example (taken from a blogging engine named Typo) is admittedly a little more involved, using several array methods and a Rails helper method. The link_to method of Rails creates a URL to an action/controller pair. If you also know that collect in this example returns a new array of values returned by link_to (that is, HTML href tags) and that join is used to create a delimited list of array values, it is apparent this code is used to do what the method name category_links hints at: It creates an output string, such as "Posted in Ruby, Rails, Howtos," with hyperlinks in it, that is placed at the foot of every blog post. Not necessarily readable by the complete outsider, this code still makes complete sense to someone who understands Ruby. And that is the correct meaning of least surprise. Ideally, code should be intuitive to noncoders, but it most definitely should always be nonsurprising to people familiar with the Ruby language.


Objects, Classes, and Modules

Like many modern programming languages, Ruby is object oriented. However, Ruby could be considered to be a purer object-oriented language than many. To see what I mean, let's look closer at that expression 30.minutes.ago from the previous example. You might have naturally assumed that it is a bit of syntactic sugar, something that the Ruby interpreter intercepts and translates into something real like Time.now 30*60 upon execution. Indeed, that is how it would have to work in languages such as C++, Java, and PHP, where integers, floats, and sometimes strings are treated as primitives below the OOP framework. In Ruby, however, everything is an objecteven integersas the following example demonstrates. This and other examples are executed within irb (the Interactive RuBy interpreter), a program included within the Ruby distribution that you can use to run these examples:

irb> 30.class
=> Fixnum
irb> Fixnum.superclass
=> Integer
irb> Integer.superclass
=> Numeric
irb> Numeric.superclass
=> Object
irb> Object.superclass
=> nil

As you can see, we can call methods on the integer directly and traverse up its class hierarchy to the Object class, which is the root class for all other classes in Ruby. This process should be familiar to Java programmers, although we shall soon see this does not have the same importance as it does in Java. Looking further, we can see that because integers are objects in Ruby, they can have methods associated with them:

irb> (30.public_methods - Object.public_methods).sort
["%", "&", "*", "**", "+", "+@", "-", "-@", "/", "<<", ">>", "[]", "^", "abs", "between?",
"ceil", "chr", "coerce", "div", "divmod", "downto", "floor", "id2name", "integer?",
"modulo", "next", "nonzero?", "prec", "prec_f", "prec_i", "quo", "remainder", "round",
"singleton_method_added", "size", "step", "succ", "times", "to_f", "to_i", "to_int",
"to_sym", "truncate", "upto", "zero?", "|", "~"]

This example lists all the public methods for integers (with methods inherited from the Object superclass removed for the sake of clarity). This ability to query objects and see their classes and methods is called reflection, and it is part of Ruby's powerful metaprogramming capabilities that we examine later. But let's return to the fact that even lowly integers are objects. That means you no longer have to worry about wrapper classes and other awkward interfaces like in other languages and that every variable in the system contains general methods inherited from object. In addition, it means we do not need to rely on syntactic sugar. When you call 30.minutes.ago, it is executed using methods defined in the Integer class against that object "30", and it really just runs like any other method does, with no syntactic sugar involved.

Except that it doesn't. Try running this code in irb now. You are instead greeted with the following output from the interpreter:

irb> 30.minutes.ago
NoMethodError: undefined method 'minutes' for 30: Fixnum from (irb):1

Something is wrong here. Indeed, look at the list of methods for Fixnum. No minutes method is listed. And yet, this code works in Rails; the method is clearly there. Let's dig a little deeper and look at what Rails's console reports as methods for the Fixnum class. We can do this by running our method-dumping command within Rails's console debugger (found at script/console within any Rails project):

[View full width]
~rails/myproject> script/console
>> (30.public_methods - Object.public_methods).sort
=> ["%", "&", "*", "**", "+", "+@", "-", "-@", "/", "<<", ">>", "[]", "^", "abs", "ago",
"between?", "byte", "bytes", "ceil", "chr", "coerce", "day", "days", "denominator", "div",
"divmod", "downto", "even?", "exabyte", "exabytes", "floor", "fortnight", "fortnights",
"from_now", "gcd", "gcd2", "gcdlcm", "gigabyte", "gigabytes", "hour", "hours", "id2name",
"integer?", "kilobyte", "kilobytes", "lcm", "megabyte", "megabytes", "minute", "minutes",
"modulo", "month", "months", "multiple_of?", "next", "nonzero?", "numerator", "odd?",
"ordinalize", "petabyte", "petabytes", "power!", "prec", "prec_f", "prec_i", "quo", "rdiv",
"remainder", "round", "rpower", "second", "seconds", "since", "singleton_method_added",
   "size",
"step", "succ", "terabyte", "terabytes", "times", "to_bn", "to_f", "to_i", "to_int", "to_r",
"to_sym", "truncate", "until", "upto", "week", "weeks", "year", "years", "zero?", "|", "~"]

Wow! Where did all those extra methods come from? And how did they get within Fixnum? To understand is to see how seriously Ruby takes the principle of flexibility; it allows users to override and extend the functionality of its base classes if needed. And that is what Rails does here. Listing 1 is a simplified excerpt of code within Rails that extends the Numeric superclass of Fixnum.

Listing 1. Example of How Rails Extends Ruby's Core Fixnum Class

class Numeric
  def seconds
    self
  end
  alias :second :seconds
  def minutes
    self * 60
  end
  alias :minute :minutes

  def hours
    self * 60.minutes
  end
  alias :hour :hours

  # Reads best without arguments: 10.minutes.ago
  def ago(time = ::Time.now)
    time - self
  end

  # Reads best with argument: 10.minutes.until(time)
  alias :until :ago

  # Reads best with argument: 10.minutes.since(time)
  def since(time = ::Time.now)
    time + self
  end

  # Reads best without arguments: 10.minutes.from_now
  alias :from_now :since
end

Allowing users to extend base classes is a remarkable example of Ruby's flexibility. Consider how Rails would have to add helper methods under a more traditional OOP language. In languages such as Java, base classes are precompiled and extended only through inheritance. You can only define a subclass and add methods to it. Under this approach, the Rails source would have to define classes such as RailsNumeric, RailsString, RailsTime, RailsArray, and so on that inherit from Ruby's core classes. Rails code would be built using such methods, and we would have to be careful to convert to these new classes when we wanted to use the additional methods. For instance, our simple time-comparison example would turn into this:

RailsFixnum temp = RailsFixnum.new(30)
@highlight_title = true if (posting_time < temp.minutes.ago)

Although this might be acceptable if you are a Java programmer and have no other choice, writing Ruby is supposed to be creative and immediate; it is just not supposed to look like that. And this is where Ruby's principle of flexibility really shines. There is no particular reason why even the core classes of the library need to remain absolute and inviolable. This is what Matsumoto meant by the "freedom to choose": Any user is allowed to tweak and extend Ruby's core libraries for his or her own ends. It is democracy in action. Not only does this make for happier hackers and simpler code, it improves the language by letting end users discover new ways of doing things not anticipated by Ruby's creators; indeed, some of Rails's extensions to the Enumerable module will be incorporated into a future Ruby release for everybody. And it makes the other two principles attainable; by allowing flexibility, Ruby enables code to be concise and consistent.

Before moving on, we should cover one other mechanism for extending functionality in Ruby: the mixin. Like Java, Ruby supports only single inheritance for classes. However, Ruby also allows users to define modules that contain methods, constants, classes, and even other modules. This is normally used for namespacing (it is the technique that eliminates confusion between the classes ActiveRecord::Base and ActionController::Base), but it is also the basis for mixing in code to classes. Mixins enable you to easily add functionality to multiple places without repeating yourself, which reduces cut-and-paste coding. Listing 2 illustrates how this technique conceptually works within Ruby (the actual source code within Ruby looks more complicated). As discussed later, both the Array and Hash classes in Ruby support a wide range of similar functions for iterating over their elements. Instead of writing the same code in multiple places, Ruby defines the functions only once and includes the methods into Hash and Array via mixins. (The include directive in the listing is doing the mixin.) In this particular case, the module Enumerable adds 28 additional functions that work by calling the each method within the class they are included within. This demonstrates an interesting caveat when using mixins: Be sure to check whether methods in the mixin are expecting to call certain attributes or methods within your classes. Each module may have varied requirements, and some may be completely self-contained, but it is up the user to figure that out. As discussed in the next section, Ruby does not precompile or prebind any methods or attributes called within methods, so it has no way of automatically warning you if they are not found. Fortunately, the documentation usually indicates this sort of dependency.

Listing 2. An Example of Module Mixins in Ruby

module Enumerable
  def inject

    ...
  end

  def partition

    ...
  end

  ...
end

class Array
include Enumerable
end

class Hash
include Enumerable
end


Duck Typing

So, what happens if you include Enumerable within a class of yours and forget to create an each method? If you have come to Ruby from the Java world, you might see a natural affinity between Ruby's class hierarchy and Java's (both have Object at the root). And you might also conclude that Ruby's modules act in some way similar to Java's interfaces. And so, you might expect that including the Enumerable module would check to see whether the each method is defined and throw an exception if it is not there. In Java, after all, interfaces force implementing classes to comply with their public list of methods; in essence, that acts as a contract with which the class must comply. You might reasonably expect Ruby to behave similarly. But let's explore with an example.

For the sake of illustration, Listing 3 defines a class for bird watching that contains an internal hash of birds with counts of how many times they have been spotted. You might also notice this class declaration includes Enumerable but has not implemented the each function. Uh oh.

Listing 3. A Simple Bird-Watching Class

class BirdWatcher
  include Enumerable

  def initialize
    @birds = {}
  end

  def saw_bird(bird)
    if @birds[bird].nil?
      @birds[bird] = 1
    else
      @birds[bird] += 1
    end
  end
end

Yet, nothing happens when we interpret that class declaration in irb. It is even possible to create and use the object without any errors being thrown. In fact, an exception is thrown only when you attempt to call one of the methods from Enumerable that uses each:

irb> b = BirdWatcher.new
irb> b.saw_bird("Robin")
irb> b.saw_bird("Sparrow")
... # 23 sparrows later
irb> b.saw_bird("Sparrow")
irb> common_birds = b.select {|bird , count| count > 10 }
NoMethodError: undefined method 'each' for #<BirdWatcher:0x356950 @birds={"Sparrow"=>24,
"Robin"=>1}>
        from (irb):11:in 'select'
        from (irb):11
        from :0

In this example, we use the Enumerable#select method to obtain an array of common birds out of our overall sighting statistics. Don't worry about that weird syntax after select in there (we get to that later). Instead, notice that the error is thrown only for the first time we use the select method, when it in turn tries to call each. Interestingly enough, this is easily fixed within irb if we define an each function after the fact:

irb> class BirdWatcher
irb> def each
irb> @birds.each {|bird, count| yield bird, count }
irb> end
irb> end
=> nil
irb> common_birds = b.select {|name, count| count > 3 }
=> [["Sparrow", 4]]

See whether you can understand what we are doing here in the context of the previous section. (Again, do not worry about the syntax of that each statement yet.) We are just extending an existing class with irb in the same way that Rails adds new methods to Numeric. Let's pose a different question then. How does Ruby figure out what methods an object can call? Why doesn't Ruby get upset that we have added some methods when the underlying method they call is undefined? What type system does Ruby use anyway?

In a strongly typed language such as Java, type is paramount. An object's type is its class, and only variables of the same class can use its methods. Variables whose type is a superclass or interface can also use the object, but they can only call methods defined at or above their level, an information-hiding principle generally known as abstraction. Thanks to C++ and Java, this approach has largely become canon. But why do we need types in those languages? In the early days of OOP, types were needed so that the compiler could determine exactly which methods would be invoked where; this in turn allows the compiler to optimize method calls because it has all the information it needs at compilation time. In most modern statically typed object-oriented languages, however, method resolution is performed at runtime anyway. Instead of being an optimization technique, static type checking is a security mechanism screening callers: A caller can use an object only if it matches the type he is expecting. Otherwise, the program does not compile. This assurance is known as type safety, and the goal of it has been to build a web of types and expectations to ensure that nothing unexpected occurs in the program. This sounds great, except for all the ways in which it no longer applies.

As the saying goes, the road to hell is paved with good intentions. (That is one back road we won't be traveling down in this shortcut.) The goal of type safety is to eliminate unexpected behavior (that is, bugs) through the contracts of types, but this approach has several shortcomings. The first problem is the massive amount of bureaucracy required for static typing. For instance, consider an example of Java code we might need to get a datetime out of a generic collection of events generated by a system logger:

Datetime dt = (Datetime) ((Event) eventQueue.front).getDatetime();

Do you think that dt variable might possibly be a datetime there? Now repeat for every line of your program: It is a lot like writing a novel by focusing solely on the punctuation. Still, if type safety works well, this might seem like a fair cost. Unfortunately, however, this example does not even have that safety. If you look again, it is doing a dynamic runtime cast of an item retrieved from the top of eventQueue (which, in this example, is just a quickly named generic queue whose values are stored as type Object). Because this is a dynamic cast, the type coercion and checking happens at runtime as opposed to compilation time, thus bypassing compilation's type safety mechanics entirely. Of course, the smart Java programmer might say that I could mitigate this issue by creating an EventQueue object that encapsulates the generic queue with properly typed input and output functions. And that programmer is right, too, except that then the programmer is providing extra logic to correct a failing of the type safety net. In addition, many programmers would just leave it at the cast anyway. In any event, this problem gets worse when you read data from generic sources, be they distributed architectures such as DCOM/CORBA, database layers such as ADO/ODBC, or even certain web services. In those cases, you will likely have to do dynamic casting from a generic object or variant type. Meaning, the notion of all-encompassing type safety fails at the very place you want it most: protecting the integrity of your data.

True type safety is an illusion. Without type safety, however, are your programs destined to a buggy doom? Not really. Type safety is definitely good for eliminating a basic class of bugs, but the problem is that program errors are often more subtle than confusing an integer for a string. Type safety is like a broad contract; it can guarantee general conditions (that a field is a string, or an object is of a particular transaction type), but not specific uses. Unfortunately, correct programs often require more specific constraintsfor example, it is not enough that a customer ID is typed as an integer; it must be an integer in a particular range that validates against internal checksumsthat go beyond what can be expressed with mere types. In other words, static types can never replace full-fledged testing, but too many programmers act like it does. Compiling is not testing; testing is testing. Bugs happen in both statically typed and dynamically typed programs, and the best defense is unit testing or some other more-rigorous testing approach. Because it is not compiled and has no static types, Ruby might seem to be more dangerous to develop in; but in some sense, it can actually be safer to program in: Because developers have no illusions that type checking is a serious defense against bugs, they will not confuse compilation with correctness.

Ruby takes a more enlightened approach to type: Ruby just does not care. Of course, in Ruby, objects do have classes, and you can also easily inspect any object's class if that is important to you. If you program in the Ruby style, however, you never need to care. Classes are still useful in Ruby as a way of defining methods for objects, but they are not important as a typing framework. Ruby does not check that a variable has the right class type for a call; instead, the only important thing Ruby cares about is this: Does this object have this method I'm calling at this instant? Methods are all that matters; the class is not used at all. This approach is commonly referred to as duck typing, after the aphorism "If it talks like a duck and walks like a duck, it is a duck." The class of the object has no effect, and you can call a method without needing to cast the object appropriately. The result is cleaner, simpler, and more-understandable code. For instance, we could rewrite our sample Java datetime snippet in Ruby as follows:

dt = eventStack.front.getDatetime

Without all the type coercion, that code certainly becomes a lot simpler. When executing that line, Ruby merely checks that the eventStack variable contains a front method and the value popped contains a getdatetime method to run in turn. I am repeating this over and overclasses and types are irrelevant to how Ruby handles method calls, they only exist to define object methodsbecause I know at least a few of you were still trying to figure out how Ruby determines and validates the classes are correct in that call. The eventStack variable could have the class EventStack or the class RandomStuffHappening or the class Giraffe; as long as it has a method named front, Ruby is happy. Don't feel bad if you still have that preoccupation with classes and types; old habits die hard, and the importance of types has been drummed into your head for years by every computer science textbook. If you really want to program in the Ruby way, however, you need to get over that. Forget about figuring out object types. And when you get used to duck typing, you will find it hard to go back.


Symbols

When you start programming in Ruby on Rails, you see a lot of funny little constructions that look somewhat like strings but start with colons rather than quotation marks:

[View full width]
link_to "log in", :controller => "accounts", :action => "login" options = { :type => 'text
  /xml',
:disposition => 'inline' }
Category.find(:all, :order => :position, :limit => 1000)

You might wonder what that funny little colon is doing in the front of :controller or :action. "Those are symbols, and they are very important," the typical Rails tutorial replies, "but we have to move on." Okay, but what are symbols? A symbol in Ruby is basically an immutable string. This is a useful abstraction for representing enumerated values in terms of performance and memory consumption. Because symbols are immutable, multiple references can refer to a single spot of storage in memory. It does not matter whether there are one or one million variables with the value :action in your code; they all use a single shared location in memory. In addition, comparing whether two symbol references are the same is trivial, because you need merely check that they point to the same location in memory. Symbols are certainly efficient.

However, this does not really explain the prevalence of symbols in Ruby on Rails, where you run into symbols everywhere! In the previous examples, you can see symbols used to name arguments for functions or as keys for hash tables (that is, as names) or in some cases where the code is using them as values, too (for example, :all and :position). This might seem a little strange. Why not just use strings? Does Rails really need to wring out that minor bit of optimization from symbols? Hardly. Instead, Rails's extensive use of symbols reflects a rather common but underacknowledged Rubyism: Symbols denote specialness. More explicitly, the use of a symbol indicates that the name or value is not an arbitrary string but one of a limited set of expected values. This is also known as an enumeration in other languages, but Ruby does not require you to define the list of allowed values beforehand. Ruby allows you to define new symbols ad hoc when you need them. In this case, the use of symbols is a clue to the user that link_to is expecting only a limited number of named arguments that match the symbols.

Astute readers might already know that Ruby also allows users to define constants. These identifiers starting with capital letters are used to substitute values in for a name. So, if we are calculating the circumference of a circle, we can express that formula as diameter * Math::PI, and the PI is replaced with 3.14159... at execution time. Constants are indeed useful, and you might wonder why Ruby needs both constants and symbols. Couldn't we just use for symbols constants that we don't care about the value of? This is, after all, the approach commonly used in other languages for enumerations. Yes, but why should we here? The whole point of constants is that they represent something else (for example, Math::PI represents a numeric value), whereas symbols represent nothing but themselves. Furthermore, doing enumerations with constants is a pain: They need to be defined before actual use; they need to be scoped within a class or module; and it is all too easy to accidentally sow confusion with them. To see what I mean, consider how we might implement the named arguments to link_to without symbols:

module ActionController
  Controller = 1
  Action = 2
  ...
  Style = 2
End

link_to "log in", Action::Controller => "accounts",
                ActionController::Action => "login"

This code could work, but it is both a lot more verbose and a lot less flexible. If we want to add a new argument to link_to (perhaps a Style argument that takes a string of CSS), we must add code in two separate places: an implementation within the link_to method and a separate declaration of the Style constant in the module. In addition, you might have noticed my declaration for the Style constant introduced a subtle bug; I accidentally added the constant with the same value as Action. This means if we ever pass in the optional Style parameter to link_to, it will accidentally be interpreted as another Action. Because symbols only represent themselves, there is no way for code to confuse :action and :style. And I can add support for style without having to predeclare it in the ActionController module somewhere. Simple to use and well suited for their purpose, symbols are one of the unique features of Ruby.


Blocks

I have described this digital shortcut as a drive down local roads, but now it is time to take to the highways. Highways are fascinating concepts. Although they seem like an inevitable part of the driving infrastructure, highways are actually quite new in the history of roads. As recently as the early 1900s, a drive across the country was essentially the same as driving across town, navigating a dense network of local roads that would get you there eventually but not efficiently. Routes were not clearly demarcated, and road maps as we know them did not exist. Navigation was a challenge when directions comprised only instructions such as "take right at third fork in the road" or "turn left at the willow tree." Of course, getting lost was inevitable. And accidents and mishaps were all too common on neglected backcountry roads. This might seem a cartoonish depiction of long-distance driving, but that is largely a reflection on just how fast, safe, predictable, and even boring intercity travel has become due to highways. And that is because the highway changed the very nature of long-distance trips.

Highways are built not from asphalt and cement, but from limitations. Instead of connecting to every road, a highway runs largely isolated from local roads, with only a scant selection of exits where new traffic can enter and leave. Highways never actually intersect directly with other roads except by exits and ramps. As a result, local traffic sticks to local roads, and interstate traffic stays on the highways, meaning long-haul truckers do not bumble into suburban cul-de-sacs. Most significant of all, highways simplify navigation for the driver. Instead of drivers having to pick a complex route over local roads, the highway limits their route. To enter a highway is to largely surrender navigational choices: There are no turns or forks in the road to choose; backtracking and U turns are forbidden; your only option is to drive forward until you reach your exit. You can drive faster because you have less to decide, and you can concentrate on driving because you have less to consider. So, what does this have to do with Ruby? Let's talk about iterators. As long as there have been arrays in computer languages, there has been code that traverses their elements. For the longest time, however, iterating through the elements of an array was a lot like driving cross-country on local roads. Your code (the car, for those slow on metaphors) has to figure out the route. If you are lucky, your code might just need to initialize a pointer to the first element of the array and drive forward to each element by incrementing that pointer's address, ending before you overshoot the bounds of the array. (Otherwise, you might get fencepost errors, which is a bit like turning at the wrong intersection.) In more-complicated operations such as sorting or binary searching, your travels might involve all sorts of turns and reversals. Better hope you have a good map! Although this is simple, it has never been satisfying. This approach practically breeds bugs: It is easy for callers to make a wrong turn and bumble into memory locations they are not meant to visitand it also forces callers to worry too much about navigation while driving. There had to be a better way.

When the Gang of Four published their groundbreaking book Design Patterns, they included something in that book called the Iterator pattern. Simply put, an iterator is like a highway for a block of code. The iterator acts by starting the code at the first element (the entrance ramp if you will) and then moving the code forward through each element until it finishes with the last (the exit ramp). The result is much cleaner code. Callers who need to iterate over the element no longer need to worry about keeping track of where they are in the array and how to get to the next element; the iterator handles that. All the calling code needs to do is apply whatever logic it needs against each array element. This process is clearly better, but iterators still have sporadic support in some modern languages. Worse still, many new developers have a weak understanding of how iterators work even in languages that provide them as an option.

As a thoroughly modern language, Ruby uses iterators extensively, but in a way slightly different from existing iterator implementations. In Java, if you want to iterate over a collection, you call a method to get an object that implements the Iterator interface, which abstracts away the internals of the target object. However, you still have to write logic in your loop to drive through the elements of the iterator. The abstraction helps generalize your movement through the array somewhat (you can call a general method such as hasNext(), as opposed to figuring out position < array.size()), but you still have to navigatewhich adds a little bit of extra complexity to your code and still makes it possible for you to take a wrong turn somewhere (that is, fencepost errors). Consider this example of code you might write to display a list of comments for a blog posting:

Iterator it = comments.iterator();
While (it.hasNext()) {
  Comment comment = (Comment) item.next();

  System.out.println("%s\n", c.text());
}

Ruby's iterators are more comprehensive. Not only do they abstract away the internals of the object, they do the driving through the array for you. Here is the equivalent of the preceding code in Ruby:

for c in @comments do
  puts c.text
end

Notice that the logic to print each comment does not need to include code to increment to the next position in the array. All you need to provide is a block of code to be applied at each position, and Ruby does the lower-level work of iterating through the object for you. However, even this example code is a bit too simplistic. In Ruby, the preceding command is an example of syntactic sugar provided as a bridge for PHP and Perl programmers. Ruby actually executes that iteration as follows:

@comments.each { |c| puts c.text }

This syntax might seem a little more obscure to express, but it is a lot more powerful to use. To understand how this works, you need to understand blocks. And understanding blocks is the single most important step to understanding Ruby.

Block is a generic computer science term for a chunk of logic, but in Ruby it has a specific meaning. In Ruby, a block is essentially a nameless method: a section of executable code with zero or more input arguments. In Ruby's syntax, blocks are demarcated either by {} or the keywords do and end. (The usual convention is to use {} for single-line blocks and the keywords for multiline blocks.) The input parameters into a block are surrounded by vertical pipes, as the following examples show:

{ puts "Hooray!" }

{ |x| x > 3 }

do |full, fname|
  if (FileTest.file? full) and (File.stat(full).mtime.to_i < old)
    File.unlink full
  end
end

If the block is the yin of Ruby iteration, yield is the yang. This keyword in Ruby indicates where within the method you want the block to be applied. After the yield statement, you can specify zero or more variables, which are passed in as parameters to the block. This is a bit of an unusual influence. Ruby actually borrowed this feature from CLU, an educational programming language developed at MIT in the late 1970s (and used there by your author in a software engineering course as late as the mid-1990s), which pioneered some early concepts of object-oriented programming but failed to have a major influence on the development of C++ and Java. Blocks are a wonderful feature (as you will see), but their obscure history can make them a bit bewildering to newcomers. The following examples show how yield and blocks work together:

def threepeat
  yield
  yield
  yield
end

threepeat { puts "Hooray!" }

def each
  @birds.each { |bird, count| yield bird, count }
end

You might have noticed the second example of yield looks strangely familiar. It is the definition we created for each earlier when we needed to get the Enumerable mixin to work. By convention, each is the standard name for forward iterators in Ruby, although you must not think that each is defined differently than other methods; it is possible to create iterators with other names, and it is also possible to define a noniterator method named each if you want (although that would probably confuse your users). You might notice that this implementation of BirdWatcher#each is just calling the each method of the underlying hash and yielding the results upward. Blocks and yields can be chained like this to create higher-level iterators out of lower-level ones (although our example is rather simple). For proof of how effective this is, let's look more closely at the Enumerable module.

As you saw before, the Enumerable module is a selection of 28 methods that can be mixed into any object that defines an iterator named each. As Listing 4 illustrates, each method in Enumerable could be described as a higher-level iterator, in that they abstract away using the each method within a block-driven call. As an example of how this works, suppose you need to determine whether an array of bird observations contains an endangered species. (In that case, you stop iterating and summon the ornithologists.) In Java, you construct this using a basic Iterator and breaking when you match your condition, as follows:

Iterator it = birdlist.iterator();
while (it.hasNext()) {
  Bird bird = (Bird) it.next();
  if (bird.endangered()) {
    call_scientists();
    break;
  }
}

This should seem straightforward, and you could probably write detection iteration examples like this in your sleep. But why do you need to? All of them are the same. If not found, go to the next element; otherwise, do something and break. What Ruby allows you to do is think at a higher level:

call_scientists if birdlist.any? {|bird| bird.endangered? }

Internally, the any? function works by running over the object's each method with the block and breaking and returning true if the block's logic evaluates to true for an element, and false otherwise. That is the genius of Ruby's block-driven iteration approach. By allowing you to layer iterators above iterators, Ruby enables you to write your logic at the highest level where it makes sense without worrying about wrangling iteration at the lowest level.

Listing 4. Some Examples of Enumerable's Higher-Level Iterators

Selection
common = bird.select {|bird, count| count > 5 }
rare = bird.reject { |bird, count| count > 5 }

Accumulation (summing up values)
sightings = birds.inject {|total, bird| total += b.count}

Detection
indie = jukebox.any? {|x| x.artist.hip? }

Searching
Winner = contestants.find {|p| p.has_answer? }

Reordering
titles = jukebox.sort_by {|x| x.title }

Partitioning
quick, dead = people.partition {|p| p.can_outrun? :bear }

Recombination
blind_dates = restaurants.zip(men, women)

Ruby's block/yield approach is much richer than mere iteration, however. The block approach allows the caller to perform arbitrary calculations against an object, but the target object still maintains control over what the caller can do. This approach also proves useful where the target object has to allow access while maintaining data integrity and properly managing resources. For instance, here is how Ruby uses blocks for reading from files:

File.open(path, 'rb') do |file|
  until file.eof
    puts file.gets
  end
end

In this example, the caller can pass a block (with a file parameter) into the call to File.open. There is no need to call File.close; when this block exits, the File object automatically closes the file. The File object can thus ensure that files are never left in an inconsistent state; even in cases where exceptions are thrown, the file is safely closed and handles released. If you can recall any code where you forgot to close a file on an error return and leaked handles as a result, the advantage of this approach should be obvious. Similarly, you can also use blocks for other limited resources that need to be used safely: locks, database connections, and server sockets, among others.

Blocks also prove useful when it is easier to tackle a problem as nested parts rather than as a complicated whole. The Builder library used in Rails's .rxml files to create XML is a prime example of this approach. XML has proven to be a popular format for data exchange with its human-readable, hierarchical structure; but XML contains many subtle gotchas that can stump developers who just need to output a file. The Builder class for Rails solves this issue by abstracting the process of building XML files as a series of nested blocks, with elements posing as methods and attributes as parameters:

xml.instruct! :xml, :version => "1.0", :encoding => "UTF-8"
xml.feed "xml:lang" => "en-US", "xmlns" => 'http://www.w3.org/2005/Atom ' do
  xml.title @feed_title
  @items.each do |item|
    render :partial => "atom10_item_article", :locals => {:item => item, :xml => xml }
  end
end

This example (simplified from code in Typo) builds a syndication file in the Atom feed format. The XML that results from this call might look like this:

<?xml version="1.0" encoding="UTF-8"?>
<feed xml:lang="en-US" xmlns=http://www.w3.org/2005/Atom>
  <title>Odds &amp; Ends</title>
  <item>...</item>
  <item>...</item>
</feed>

In this example, Builder mirrors the hierarchical structure of the result XML with a hierarchical nesting of blocks. (In this example, we also call a partial for items, which renders each item in the feed.) Blocks are prime examples of the concise power of Rubyisms in action.


Metaprogramming

Now it is going to get really interesting. Let's leave the highways and return to the back roads to see places far off the beaten track: We are going to explore how Rails is built from metaprogramming. Metaprogramming is an umbrella concept that describes a variety of techniques for manipulating the very building blocks of the language. What we did with blocks and strings and objects was programming. Metaprogramming manipulates classes and methods instead and comprises a few related concepts:

  1. Reflection

  2. Message sending

  3. Method missing

These techniques are like wizardry: You can use them to do great things, but you should not attempt them unless you have some mastery. Naturally, all of them are also used to great effect in Rails, and delving into example code provides some real demonstrations of Ruby's power. Let's explore them in more depth.

Reflection

You have already seen examples of reflection while exploring Ruby's duck typing mechanisms. When we looked at the methods defined for Fixnums, we used one of the class Object's built-in mechanisms for inspecting the contents of any object. (Because they are defined in Object, these methods are inherited by every class in Ruby.) The following listing shows all of Ruby's reflection methods in bold:

[View full width]
irb> Object.public_methods.sort
=> ["<", "<=", "<=>", "==", "===", "=~", ">", ">=", "__id__", "__send__", "allocate",
"ancestors", "autoload", "autoload?", "class", "class_eval", "class_variables", "clone",
"const_defined?", "const_get", "const_missing", "const_set", "constants", "display", "dup",
"eql?", "equal?", "extend", "freeze", "frozen?", "hash", "id", "include?", "included_modules",
"inspect", "instance_eval", "instance_method", "instance_methods", "instance_of?",
"instance_variable_get", "instance_variable_set", "instance_variables", "is_a?", "kind_of?",
"method", "method_defined?", "methods", "module_eval", "name", "new", "nil?", "object_id",
"private_class_method", "private_instance_methods", "private_method_defined?",
  "private_methods",
"protected_instance_methods", "protected_method_defined?", "protected_methods",
"public_class_method", "public_instance_methods", "public_method_defined?", "public_methods",
"respond_to?", "send", "singleton_methods", "superclass", "taint", "tainted?", "to_a", "to_s",
"type", "untaint"]

Most of these methods are similar to our usage of public_methods earlier, returning different information about the object we are inspecting. We also used Ruby's reflection techniques when we extended classes dynamically with new methods, and these capabilities are popular anywhere you might need to dynamically examine or alter the structure of a class. For instance, the Unit Testing library in Rails uses reflection to automatically discover and execute all the tests defined within a suite.

One particularly useful method among these is the respond_to? method. This method allows you to conditionally apply duck typing against any Ruby object. This call is a useful way to determine whether a method or attribute is defined, and can be used to avoid a dreaded NoMethodError when calling a method against a completely unknown object:

irb> i = 27
irb> str = "foo, bar, baz"
irb> str.respond_to? :split
=> true
irb> i.respond_to? :split
=> false
irb> str.split if str.respond_to? :split
=> ["foo,", "bar,", "baz"]

class Song
  attr_reader :title
  def initialize(title)
    @title = title
  end
end
irb> s = Song.new("Untitled")
=> #<Song:0x34fe70 @title="Untitled">
irb> s.respond_to? :title
=> true

This method respond_to? is also useful for conditional object invocation. When you have an array filled with objects of different types, you often need to figure out what the object is before you can use it. In Java's static typing, you might use the following code to send a string str to an arbitrary collection of strings, files, and sockets. (For the sake of example, assume everything has been pre-opened and each type had different methods.)

Iterator it = outputs.iterator();
while (it.hasNext()) {
  Object item = it.next();
  Class klass = item.getClass();
  if (klass.getName() == "String") {
    ((String) item).append( str);
  } else if (klass.getName() == "Socket") {
    ((Socket) item).write(str);
  } else if (klass.getName() == "File") {
    ((File) item).write(str);
  }
}

Because Java has static type checking, we need to cast the Object variable to each possible class before we can use it. What if you also want to add a NamedPipe class (with a write method, too) to your list of outputs? In that case, you must remember to explicitly check for the class again, even if append works in the same way as the other append methods. As you have already seen, Ruby's duck typing philosophy cares more about whether an object supports particular methods than what its class is. We could thus create a similar mechanism in Ruby using duck typing to cover several possible method names, as follows:

outputs.each do |out|
  case
  when out.respond_to? :write then out.write(str)
  when out.respond_to? :append then out.append(str)
  when out.respond_to? :post then out.post(str)
  end
end

Of course, you could still use the reflection methods class, type, kind_of?, and is_a? if you need to see an object's class, but Ruby's duck typing does not care. It is just as easy to define a generic string sender that works even for classes you do not know.

Message Sending

You might have noticed that the list of possible method names for sending a string to an output did not actually include a method named send. That is because send is reserved for an important method in Ruby, used to dynamically invoke object methods. As you have seen already, in Ruby an object's methods are not fixed at any compilation time but can be dynamically extended or modified at any point. For now, we have called the methods directly by name, but it is possible instead to invoke generically any object method by using a string or symbol variable to specify the target method.

To illustrate a use of send, consider another common programming technique: the dispatch table. Sometimes you might find yourself needing to create a method that dynamically invokes other methods based on the types of input. For example, assume we have an object with a generic search method that invokes one of several helper methods depending on the type of search:

def search(type, terms)
  case type
    when "title" then title_search(terms)
    when "author" then author_search(terms)
    when "subject" then subject_search(terms)
    else raise NoMethodError
  end
end

Generally, defining dispatch tables in cases such as this is bad because it creates an additional place we need to add code if we define another method: If we add publisher_search, we also need to remember to add code to search. In statically typed languages such as Java, this is the only way to do dispatching, because you need to explicitly declare the method calls if you want the program to compile. Under Ruby's dynamic duck typing model, however, we can radically simplify this code:

def search(type, terms)
  send("#{type}_search", terms)
end

In this case, we can dynamically construct the name of the target method we want to call and then send the search terms onward to it. When we add our publisher_search, it is automatically supported by this dispatch function. If the method is not defined, the object returns NoMethodError. But why is it called send and not invoke or execute? Philosophically, Ruby's object-oriented model is inspired by Smalltalk; in that language, programs do not call methods but instead send messages to an object. This might seem like splitting hairs, but a significant semantic difference exists. In languages such as Java or C++, programs can only call methods, because it is impossible for objects to dynamically modify their responses to calls while the program is running. (They can only execute the methods linked during compilation.) In a language such as Ruby, however, it is possible for objects to do quite a few things with that message. Normally, most objects just invoke a method, but an object could also raise an error, delegate the message to another object, or even allow a message for which it has no methods defined.

For completeness, Ruby also includes Method objects for those who prefer an object-based mechanism for dynamic execution. The Object#method call returns a reference to a specific method on an object:

split = "foo, bar, baz".method(:split)
split.call

This might prove useful if you want to defer execution of a method or store a workflow as a sequence of actions on multiple objects. You might also think this would be the mechanism for parameterized operations such as quick sorts, but that case is covered quite adeptly by blocks. In addition, the Method object is not just a reference to a generic class method, but a method within a particular object. (For example, the value of self in the Method is the object we called method against.) Rails also has module methods add_method, attr_reader, attr_writer, and attr to also add methods dynamically. Finally, for the really serious wizards, it is possible to execute strings of Ruby code within a Ruby program using the eval command (or one of its specific types, such as module_eval or class_eval). This is a powerful technique for when you need to dynamically build class definitions or module code within a Ruby program, but because it can create some serious performance and security issues, be sure to use it only if nothing else will do.

Method Missing

In Ruby's dynamic dispatching, it is even possible to execute a method that does not exist. Huh? This may seem like a Zen koan, but it is an actual technique in Ruby. How is it possible to invoke a method if it is not defined? Meet method_missing, another method defined in Object. In Ruby, method_missing is the catchall method invoked if you attempt to call a method that is not there. By default, it is simply defined as follows:

class Object
  def method_missing(method_id, *arguments)
    # code that constructs the error string elided
    raise NoMethodError, "undefined method..."
  end
end

This method is the source of those NoMethodError exceptions we saw when we tried calling 30.minutes.ago way back when. Notice, however, that this error is not being thrown by the Ruby interpreter but by a method within the Object classa method that, like any other method, can be overridden in a subclass. Let's see how this works; unlike our experiments with Rails extending Fixnum earlier, you probably do not want to redefine Object#method_missing. What if you define method_missing in a subclass to not raise an error but do something useful instead? You could then have an object that could run any arbitrary request thrown at it! Suddenly, one mysterious feature of Rails makes a whole lot of sense.

When you first began learning Rails, you might have noticed that it provided a comprehensive selection of methods for searching your back-end model. Supposing you have a User model, you might expect any of these methods to work:

User.find(4)
User.find_all
User.find_by_name("Jake")
User.find_all_by_country("Switzerland")
User.find_by_first_name_and_last_name("Jacob" , "Harris")

In addition, if you update your database structure, you could use new methods against that schema without needing to explicitly rebuild your model. Strangely, however, the actual definition of your model in Ruby seems to lack any of these methods. You can even search in vain through all the files of your Rails application and fail to find them anywhere. How can this work? Most Rails books can only answer "magic," but you now know magic is really just another name for method_missing. Indeed, Listing 5 shows a slightly simplified version of that magic deep within ActiveRecord::Base that makes it all possible.

Listing 5. ActiveRecord's Use of method_missing

def method_missing(method_id, *arguments)
  if match = /find_(all_by|by)_([_a-zA-Z]\w*)/.match(method_id.to_s)
    finder = determine_finder(match)

    attribute_names = extract_attribute_names_from_match(match)
    super unless all_attributes_exists?(attribute_names)

    conditions = construct_conditions_from_arguments(attribute_names, arguments)
    options = { :conditions => conditions }
    set_readonly_option!(options)
    send(finder, options)
  else
    super
  end
end

def determine_finder(match)
  match.captures.first == 'all_by' ? :find_every : :find_initial
end

def find_initial(options)
  # Code to execute a query against the DB with LIMIT 1
end
def find_every(options)
  # Code to execute a query against the DB
end

Let's briefly walk through what this code is doing. When you make a call such as find_all_by_first_name("Jacob"), this method is invoked because no such method is defined in your model. The first line uses a regular expression to check that you called a find method. If you did not, it sends the request to its superclass, which reaches Object's definition of method_missing and results in a NoMethodError. The regular expression is then used to determine how many results are to be returned from your find. (The determine_finder method returns a symbol to indicate which helper function executes the find.) The next step extracts conditions for the find from the method name. (In this case, there is only one condition, first_name.) If any of the conditions specified are not fields in the database table, the method again raises an error by calling Object's method_missing. The conditions are then used to construct a condition string such as "first_name = 'Jacob'", and are placed in a hash of arguments for searching that is then set to readonly as a precaution. The final step is to call send using the finder value (which is either :find_every or :find_initial), which executes the appropriate search against the back-end database and returns the results.

Wow. Notice how several Rubyisms we have covered already work in harmony within this method. Symbols are used to indicate methods. send is used to dynamically invoke a choice of finder methods. Reflection against fields in the databases determines what conditions are valid. And finally, Ruby's concise and consistent code makes the implementation of method_missing completely understandable and maintainable. In turn, this technique adds flexibility and power to Rails, enabling users to more concisely and clearly define their controllers in their Rails applications. Therefore, you might see why I think it will be impossible to create true Rails clones on top of other languages; Rails and Ruby are perfectly intertwined.


Domain-Specific Languages

After cruising through the back roads of Ruby, it is time to get back on the main highway and head back into the heartland of Rails again. It is time to look at the one other aspect of Rails's success: how Rails writes complicated code using only simple declarations. Let's examine our final Rubyism of this digital shortcut: domain-specific languages.

I have thus far neglected to touch upon one other important aspect of highways: signage. That would be a serious oversight, because highway signs are one of the most effectively designed examples of communication on the planet. They provide necessary and unambiguous information to motorists who have only seconds to read them. Consequently, they succeed only through simplicity. All highway signs are tediously similar in style. They are made from the same types of reflective material, use similar phrasing (Exit 12 1 mile), and have only a rigorously regulated selection of decorative navigational arrows and facility symbols allowed. Essentially, highway signs are a language constructed from a limited vocabulary of symbols and words. This is easy for motorists to understand because the same language is used everywhere. (An exit is an exit is an exit.) It is also easy for sign creators, who have a consistent way of laying out signs. And it is good for navigation, because a leg of any route can be expressed in the same language. ("Take Route 1 21 miles to Exit 7" is clearer to follow than "Go down the Old Boston Post road until you see intersection with a gas station; take a right to Pawcatuck.") In computer programming terms, highway signs are an example of a domain-specific language (DSL).

A DSL is a limited computer language subset targeted to a particular use, as opposed to a general-purpose programming language. DSLs provide several important advantages as a programming technique. For starters, because a DSL is targeted toward a specific problem domain, it can be understood even by a user who does not know the broader language. In addition, this specific focus necessitates simpler abstractions and models, which makes it easier to achieve conciseness and consistency. Even for an expert coder, a DSL makes sense because it allows you to code solutions as higher-level concepts rather than more-verbose lower-level implementations. This results in code that is easier to design and maintain. As an example, consider validations. It is far simpler to write

validates_presence_of :name, :email

than it is to write code such as this hypothetical alternative:

def validate_name_present
  record_error(:blank, "name") if @name.blank?
ends

def validate_email_present
  record_error(:blank, "email") if @email.blank?
end

def before_save
  validate_name_present
  validate_email_present
end

And what if we want to change our validation for e-mail later? It is easier to modify that one line than look for an explicit method declaration. And that is the point of DSLs in a nutshell: They enable you to express complicated operations as much simpler expressions. In essence, they trick you into coding without even realizing it!

Martin Fowler, who coined the term for DSLs, distinguishes between two types of DSLs. External DSLs are those cases where the DSL is a file format that can be parsed into a configuration or operations in the general language. The first file you modify in any Rails coding is an example of this type. Consider the database.yml file:

development:
  adapter: mysql
  database: myapp_dev
  host: localhost
  username: root
  password: changeme

External DSLs require additional parsing logic to work, which make them harder to create, but that is the limit of what you can do in statically typed languages. However, Ruby's flexibility also supports a second type of DSL. An internal DSL is a sublanguage specified within the higher-level language itself. Internal DSLs are general-purpose code disguised as a simplified language. Rails uses this approach heavily; indeed, our example of 30.minutes.ago might be considered a DSL because it allows the user to represent and manipulate date intervals using a limited sublanguage of minutes, hours, and days. In fact, Rails itself could essentially be described as a DSL for web applications. Or rather, it is a collection of various separate DSLs focused on specific tasks, including the following:

  • Associations

  • Validations

  • Routing

  • Rendering

  • Builder for XML

  • acts_as_* plug-ins

Let's take a closer look at associations. Associations are an excellent example of how Rails defines DSLs internally. In Rails, you can specify the relationship between two tables in your database using the has_one, has_many, and belongs_to declarations, as follows:

class Review < ActiveRecord::Base
  belongs_to :user
end

That belongs_to is one powerful statement. By specifying the relationship of your model to another table, Rails automatically adds all these additional methods to your Review class:

  • user, which returns a reference to the user a review is associated with

  • user=, which can be used to reassign the association to user

  • build_user, which creates a user and links this Review to it

  • create_article, which works like build and automatically saves the association, too

  • Automatic code to update foreign keys in the table when the object is saved

Most useful of these, of course, is the user method, because that enables you to traverse your SQL database as if it were built from object-oriented relationships. There is a lot of magic that is happening behind the scenes when you describe your associations. You just need to make the declaration, and it adds the methods and builds the SQL you need without you needing to think at that levelwhich is precisely the point of DSLs. By limiting you to a specific sublanguage, they give you the precise high-level way of specifying what you want to do (in this case, describe relationships between tables) without having to code the lower-level implementations.

To see this in action, let's look at what Rails actually does when you declare an association. Listing 6 is a slightly simplified excerpt of the code within Rails that sets up a belongs_to association to another model. It might still surprise you at this point, but belongs_to is not a keyword or special construct, but just a normal method within ActiveRecord::Base. DSLs are prime examples of how useful Ruby's metaprogramming can be. In this case, this simple declaration of one line automatically creates methods and adds them to the class. If you need to remove the relationship later, fixing the DSL automatically removes all the associations for the new settings. The result is not only clean but maintainable code.

Listing 6. How Rails Adds an Association

def belongs_to(association_id, options = {})
  reflection = create_belongs_to_reflection(association_id, options)

  if reflection.options[:polymorphic]
    # Code elided for brevity
  else
    association_accessor_methods(reflection, BelongsToAssociation)
    association_constructor_method(:build, reflection, BelongsToAssociation)
    association_constructor_method(:create, reflection, BelongsToAssociation)

    module_eval do
       before_save <<-EOF
         association = instance_variable_get("@#{reflection.name}")
         if !association.nil?
            if association.new_record?
               association.save(true)
            end

            if association.updated?
               self["#{reflection.primary_key_name}"] = association.id
             end
         end
        EOF
     end
   end
end

The first line of this listing sets up a reflection object, which is used to store essential information about the table (for example, its name and primary key). This object is then passed into methods that automatically add methods to the classthe mark of metaprogramming in action. We will delve into how that works further, but first notice how the code to add these methods is listed succinctly as accessor methods (user, user=) and constructor methods (create_user, build_user). This example also illustrates another metaprogramming technique when the method calls module_eval, which evaluates a string in the current context. (In this case, the context is Review.) To make this more comprehensible, the actual code being evaluated at this point with our example is this:

association = instance_variable_get("@user")
if !association.nil?
  if association.new record?
    association.save(true)
  end
  if association.updated?
    self["user_id"] = association.id
  end
end

This code is then saved into the before_save function, which is called whenever the method Review#save saves the model to the database. The addition of this method makes sure the foreign key association between this object and its owner is correctly stored in the database whenever the object is saved if we change the value of that association.

But how are the methods such as user and create_user added to the model? Let's look at how belongs_to adds methods to your model. Listing 7 displays the code for association_accessor_ methods, the method that sets up the additional methods to access and modify the parent model (user and user= specifically). This code uses another aspect of Ruby's metaprogramming toolkitdefine_methodwhich can be used to create a method in the calling class using block syntax to represent the method's body.

Listing 7. Defining Association Methods

def association_accessor_methods(reflection, association_proxy_class)
  define_method(reflection.name) do |*params|
    force_reload = params.first unless params.empty?
    association = instance_variable_get("@#{reflection.name}")

    if association.nil? || force_reload
      association = association_proxy_class.new(self, reflection)
      retval = association.reload
      unless retval.nil?
        instance_variable_set("@#{reflection.name}", association)
      else
        instance_variable_set("@#{reflection.name}", nil)
         return nil
      end
    end
    association
end

define_method("#{reflection.name}=") do |new_value|
  association = instance_variable_get("@#{reflection.name}")
  if association.nil?
    association = association_proxy_class.new(self, reflection)
  end
  association.replace(new_value)
    unless new_value.nil?
      instance_variable_set("@#{reflection.name}", association)
    else
       instance_variable_set("@#{reflection.name}", nil)
       return nil
    end

    association
  end
end

I am not going to diagram in detail what all this code is doing, but you might want to see whether you can figure out what the created methods look like. The answer for the first case is as follows:

def user(*params)
  force_reload = params.first unless params.empty?
  association = instance_variable_get("@user")
  if association.nil? || force_reload
    association = BelongsToAssociation.new(self, reflection)
    retval = association.reload
    unless retval.nil?
      instance_variable_set("@user", association);
    else
      instance_variable_set("@user", nil);
    end
  end
  association
end

Notice that the reflection object passed to define_method is used within the method and becomes part of the method's scope at creation. This is a technique known as a closure. This is truly fascinating stuff, and we've looked at only one of the ways Rails builds DSLs. There are many other interesting examples of DSLs to be found, but the important thing to grasp here is how Rails uses metaprogramming to construct its DSLs (and how the flexibility of this metaprogramming allows Rails to provide a DSL for specifying database associations in a way that is concise and consistent [the three principles yet again]). DSL is a powerful technique for containing complexity when building powerful systems.


Conclusion

And with that, our journey through Ruby and Rails has come to an end. I hope this has been educational and inspirational in your understanding of Ruby, but all I can really wish is that you now have a sense of Ruby's beauty. You can learn Ruby's syntax and structure, you can learn its classes and modules, you can even learn its metaprogramming; but without understanding its beauty and philosophy, you might as well learn nothing at all. I want you to have something in the end.

Which, funnily enough, brings me back to the beginning. This digital shortcut originally began life as a presentation to the New York City Ruby and Rails Users' Group. While hanging out at meetings and on IRC, I noticed that newcomers to Rails often made the same simple mistakes. For instance, instead of using one of Enumerable's many useful methods, they would construct a for loop and do all the extra work themselves. They were coding Ruby, but thinking of it like Java. It is not that people did not understand syntax or did not know about Ruby's classes, but rather they just could not conceive of there being any other way of doing iteration beyond the ways already learned from Java and PHP. It became clear to me that learning Ruby without understanding its philosophy was like studying French without hearing it being spoken by fluent speakersan interesting exercise in grammar but ultimately as dry and lifeless as Latin or ancient Greek.

To my chagrin, I realized I did not really know Ruby myself, because I did not know its principles and how those guided its design. Although I knew how to write small Ruby programs, I had no knowledge of the way fluent speakers use it. I needed a good example to learn from; luckily for me, I did not really know Rails either. So I began reading through Rails source and finding there was no better teacher about how Ruby really can be fluently spoken. And so, my originally dry presentation began life again as a rich exploration into Ruby's style and how Rails relies on it.

Now I hope it continues for you. A trip like this is easy enough to start (just run gem unpack rails to save Rails's gem source to a directory or retrieve the latest version of Rails from the subversion repository); all it takes is a question. So, start asking yourself some questions: How do validations work? Where did they add polymorphism to Rails's associations? How does the controller really know where to find the view? See where these questions take you. I think you will love the journey.


Acknowledgments

This digital shortcut originated as a talk to the New York City Ruby Group and would not be the same without the ample help from my colleagues there. I would like to thank David Black, Trotter Cashion, Jesse Chan-Norris, Sebastian Delmont, and Ryan Raaum for feedback on the original presentation. Thanks are also owed to Obie Fernandez, Matt Pellletier, and Debra Williams-Cauley for allowing me to turn my little slideshow into text, and Francis Hwang for ably correcting remaining errors in my initial drafts. The Ruby community is renowned for being friendly and supportive, and the help I've received attests that is no lie.

Most importantly of all, I would like to thank my wife, Jessica Kessler, for encouraging me along the way and inspiring me to constantly strive for bigger things. She may not have corrected my proofs, but her supportive influence is writ large on every page.