loopkid: Category Ruby

ruby 1.9 utf-8 mostly works

Stefan — Mon, 07 Jul 2008 21:40:00 +0200

I was curious to see how far the implementation of utf-8 in Ruby 1.9 has developed. First we assign a pure Ascii string and check it’s encoding.

>> a = "Restaurant"
=> "Restaurant"
>> a.encoding
=> #<Encoding:US-ASCII>

Okay, nothing spectacular so far. Next we take a Unicode string.

>> b = "Café"
=> "Café"
>> b.encoding
=> #<Encoding:UTF-8>

Looking great, now let’s work with that string.

>> b.each_byte {|byte| puts byte}
67
97
102
195
169
=> "Café"

Here we see the utf-8 encoded é as two bytes.

>> b.each_char {|char| puts char}
C
a
f
é
=> "Café"

Works as expected.

>> b.size
=> 4

In Ruby 1.8 this would have returned 5.

>> b.reverse
=> "éfaC"

In Ruby 1.8 this would have generated a broken character at the start.

>> b.chop
=> "Caf"

In Ruby 1.8 this would have generated a broken character at the end.

>> b.upcase
=> "CAFé"

This is where work is still needed! In Ruby 1.8 there is Nikolai Weibull’s Ruby Character Encodings Library that does the job.

>> require 'encoding/character/utf-8'
=> true
>> b = "Café"
=> "Caf\303\251"
>> b.length
=> 5
>> b = u"Café"
=> u"Caf\303\251"
>> b.length
=> 4
>> b = +"Café" 
=> u"Caf\303\251"
>> b.length 
=> 4
>> puts b.upcase
CAFÉ

The library however is not compatible with Ruby 1.9. Another outstanding issue is the sorting of arrays containing Unicode strings.

>> a = %w[ä a b c]
=> ["ä", "a", "b", "c"]
>> a.sort
=> ["a", "b", "c", "ä"]

The result would be okay, if I wanted swedish sorting order, but what if I wanted german sorting order? This needs to be addressed. There are libraries for this in Java, so Ruby shouldn’t stand behind!

predefined character classes in grep

Stefan — Sun, 06 Jul 2008 15:54:00 +0200

Many regex implementations have “macros” for various character classes. In Perl, for example, \d matches any digit ([0-9]) and \w matches any “word character” ([a-zA-Z0-9_]). Grep uses a slightly different notation for the same thing: [:digit:] for digits and [:alnum:] for alphanumeric characters. (BSD)

Finally, certain named classes of characters are predefined within bracket expressions, as follows. Their names are self explanatory, and they are [:alnum:], [:alpha:], [:cntrl:], [:digit:], [:graph:], [:lower:], [:print:], [:punct:], [:space:], [:upper:], and [:xdigit:]. For example, [[:alnum:]] means [0-9A-Za-z]. (grep man page)

Was unter Ruby

line = "length 1450"
puts line if line =~ /\d{4}/

heißt wird also unter der bash mit grep zu

export line="length 1450"
echo $line | egrep [[:digit:]]{4}

was zwar die regulären Ausdrücke unnötig aufbläht, aber immerhin die gleiche Funktionalität zur Verfügung stellt.

conciseness vs. readability

Stefan — Tue, 01 Jul 2008 03:21:00 +0200

To print the contents of all files passed as arguments or in case of no arguments print the contents of stdin you could write

puts *ARGF

but isn’t

if ARGV.empty?
  puts $stdin.readlines
else
  ARGV.each do |filename|
    puts File.readlines(filename)
  end
end

much more readable?

read file with one line of code

Stefan — Tue, 01 Jul 2008 01:58:00 +0200

There are probably at least a have a dozen ways to print the contents of a file in Ruby, here I will present four ways of doing it.

The first method is rather verbose. Assign a file object and iterate on the file object with the each_line method and then close the file.

f = open('file.txt')
f.each_line do |line|
  puts line
end
f.close

The second method still assigns a file object, but then uses the more compact readlines instance method and closes the file.

f = open('file.txt')
puts f.readlines
f.close

The third method uses block form to access the file and automatically close it.

open('file.txt') { |f| puts f.readlines }

The fourth method calls the class method readlines to wrap it all up in one statement.

puts File.readlines('file.txt')

the beauty of ruby 1.9

Stefan — Sat, 28 Jun 2008 12:11:00 +0200

Let’s say we have a string and three arrays

string = "672"
java = []
ruby18 = []
ruby19 = []

Now you want to convert the string to an array of integers. If you were writing Ruby in javastyle you would probably write something like

i=0
digits = string.split(//)
while i < digits.size do
  java << digits[i].to_i
  i+=1
end

but since we’re we’re doing Ruby in rubystyle it looks more like

string.split(//).each { |digit| ruby18 << digit.to_i }

which looks much nicer, but still not quite right, but with Ruby 1.9’s chars interator it gets even better.

string.chars { |digit| ruby19 << digit.to_i }

Goes down like butter, doesn’t it? Isn’t that how it always should have been? And the good thing is, there is lots of this good stuff in Ruby 1.9. Maybe in Ruby 2.0 there will even be a to_i method in the array class, but now I’m getting carried away.

ordered hashes in ruby 1.8

Stefan — Thu, 19 Jun 2008 23:51:00 +0200

Ruby 1.8 doesn’t offer builtin ordered hashes, but there are several libraries out there which extend or replace Ruby’s Hash class. Among them are Ruby/RBTree, Ara T. Howard’s orderedhash and Ruby Facets. All three of these offer a hashlike syntax, letting you do things like

ordered_hash = Library['a', 1, 'b' ,2]

resulting in

{'a'=>1, 'b'=>2}

Sadly all three of these libraries have poor documentation. The one with most bearable in my eyes is Ruby Facet’s Dictionary, which also has a RubyGem available, so let’s have a look at it.

$ sudo gem install facets
Successfully installed facets-2.4.1
1 gem installed
Installing ri documentation for facets-2.4.1...
Installing RDoc documentation for facets-2.4.1...

Ruby Facets is a rather large library, so you’ll want to require just the Dictionary class.

>> require 'rubygems'
>> require 'facets/dictionary'

Let’s make an ordered hash with five elements and print it out.

>> dictionary = Dictionary[*(1..10)]
>> dictionary.each {|k,v| puts "#{k}=>#{v}"}
1=>2
3=>4
5=>6
7=>8
9=>10

Works fine. Now for the bonus round. Dictionary can also be used as a sorted hash by calling the order_by or more conveniently the order_by_key method. Once called this method ensures that all previously and subsequently inserted keys will be in sorting order.

>> array = Array.new(10){rand(100)}
>> dictionary = Dictionary[*array]
=> {1=>49, 48=>97, 17=>66, 6=>70, 52=>77}
>> dictionary.order_by_key
>> dictionary.each {|k,v| puts "#{k}=>#{v}"}
1=>49
6=>70
17=>66
48=>97
52=>77

ordered hashes in ruby 1.9

Stefan — Thu, 19 Jun 2008 00:51:00 +0200

If you create a hash in Ruby 1.8 and then browse it with the each-iterator you get the keys delivered in abitrary order.

>> hash = {:a=>1, :b=>2, :c=>3,}
>> hash.each {|key,value| puts "#{key}=>#{value}"}
c=>3
b=>2
a=>1

This might be okay for some cases, but in other cases you might want the insertation order preserved. Well, good news in Ruby 1.9 this works.

>> hash = {:a=>1, :b=>2, :c=>3,}
>> hash.each {|key,value| puts "#{key}=>#{value}"}
a=>1
b=>2
c=>3

slow iterators in ruby 1.9 on mac

Stefan — Tue, 17 Jun 2008 22:45:00 +0200

If you ever wondered why your Ruby programs are not faster but much slower with Ruby 1.9 you might be using each or times iterators on large collections.

10000000.times {}

In fact if you do a benchmark,

require 'benchmark'

n = 10000000

Benchmark.bm do |x|
  x.report { n.times {} }
end

you get stunning results.

$ ruby bench.rb
   user     system      total        real
   0.720000   0.000000   0.720000 (  0.721872)
$ ruby1.9 bench.rb 
   user     system      total        real
   16.670000   9.110000  25.780000 ( 25.923977)

When increasing the number of iterations by powers of ten it looks like the computational complexity of the times iterator is O(n) in Ruby 1.9. According to Antonio Cangiano this behaviour is specific to Mac OS X and does not appear on Linux.

nested hashes with default value

Stefan — Tue, 17 Jun 2008 04:02:00 +0200

To instantiate arbitrarily nested hashes in Ruby you can write

h = Hash.new{|hash,key| hash[key] = Hash.new(&h.default_proc)}

This let’s you do

h[1] = 2 # => 2
h[2][1] = 3 # => 3
h[3][1][1] = 4 # => 4
h.inspect # => {1=>2, 2=>{1=>3}, 3=>{1=>{1=>4}}}

That’s nice, but maybe you want your innermost hash to have a default value to access the keys via +=-operator without having to check if the key has been initialized before? You might be tempted to write the following.

h = Hash.new(Hash.new(0))

Looks nice. Now let’s assign a value and check if it’s stored correctly.

h[1][1] = 2 # => 2
h[2][2] = 3 # => 3

Looks fine too. Now let’s inspect the hash.

h.inspect # => {}

Whoops! Where have my keys gone? Let’s examine the inner hashes.

h[1].object_id # => 1751410
h[2].object_id # => 1751410

Okay, so obviously instead of instantiating new hashes for each key the same hash was used. Let’s examine the default value.

h.default.object_id # => 1751410

Ah, so the keys were stored in the default value hash. Let’s check and see.

h.default # => {1=>2, 2=>3}

So instead of generating a nested structure for each key all keys are stored flatly in the innermost hash. That is not what we wanted. We can however talk the hash into doing the intented by using a default procedure instead of a default value, which is of course documentated in the lovely core API documentation.

h = Hash.new {|h,k| h[k] = Hash.new {|h,k| h[k] = 0}}

While working fine in Ruby 1.9, this causes problems in in Ruby 1.8 because block arguments are not local there.

h = Hash.new {|h,k| h[k] = Hash.new {|h,k| h[k] = 0}} # => {}
h[1][1] = 1 # => 1
h.inspect # {1=>{1=>1}}
h[2][1] # => 0
h.inspect # => {1=>0}

To get it working in Ruby 1.8 the variables in each block have to be uniquely named.

h = Hash.new {|h,k| h[k] = Hash.new {|h1,k1| h1[k1] = 0}} # => {}
h[1][1] = 1 # => 1
h.inspect # {1=>{1=>1}}
h[2][1] # => 0
h.inspect # {1=>{1=>1}, 2=>{1=>0}}

Now we can fill the twodimensional hash with data and see the result.

h = Hash.new {|h,k| h[k] = Hash.new {|h1,k1| h1[k1] = 0}} # => {}
h[1][1] += 1 # => 1
h[1][2] += 2 # => 2
h[2][3] += 3 # => 3
h.inspect # => "{1=>{1=>1, 2=>2}, 2=>{3=>3}}"
h[1][3] # => 0
h.inspect # => {1=>{1=>3, 2=>4, 3=>0}, 2=>{1=>0, 3=>6}}

Et voilà!

installing ruby 1.9 on leopard

Stefan — Tue, 17 Jun 2008 02:38:00 +0200

Thanks to Caspar Florian Ebeling installing Ruby 1.9 on Mac OS 10.5 is now a breeze. Just install MacPorts if you haven’t already and then issue a

sudo port install ruby19

Now you can launch Ruby 1.9 with ruby1.9 and irb with irb1.9.

For the more adventurous souls among you here’s how to compile it by hand. To compile Ruby 1.9 you need Readline 5.2, therefore it will be installed if it isn’t already there. Commands with a trailing backslash are multiline statements. If you prefer to compile Readline 5.2 on Leopard by hand, you should have a look at this patch.

sudo port install readline
curl -O ftp://ftp.ruby-lang.org/pub/ruby/1.9/ruby-1.9.0-0.tar.gz
tar xvfz ruby-1.9.0-0.tar.gz
cd ruby-1.9.0-0
./configure --prefix=`echo ~`/ruby19 --program-suffix=1.9 \
--with-readline-dir=/opt/local
make
make install

Now add ~/ruby19/bin to your PATH-variable (for example in ~/.bash_profile or ~/.bashrc)

export PATH=~/ruby19/bin:$PATH

To activate the changes just launch a new terminal window. Now you can launch Ruby 1.9 with ruby1.9 and irb with irb1.9.