Nicolas Charpentier’s blog

Posted on Friday, 19th December 2008 by charpi

In my previous post, I planned to have a closer look at the ruby’s
world. As the erlang world misses a real compilation tool, I’ve decided
to make a attempt to use rake.

Usually, Erlangers use make to build their
application. It’s a powerful tool but its syntax sucks a bit. With
this complex syntax it’s hard to keep makefiles clean and with time
they become very obscure.
There are a lot of other popular construction tools:

• Ant : But I don’t like to edit XML files
• Scons: Written in python. Some erlang tasks exist but I never had
  time to look at it.
• Rake: The make utility written in ruby. Rails integrates rake
  tasks to automate all the development process.

I tried rake to replace makefiles on my personal projects looking at
those posts as example Building erlang with rake and Erlang - Make, Rake and Emake.

Usually, my erlang projects are organized according to OTP principles.
All applications are under a lib directory, so I decided to put the
Rakefile just under the $TOP_DIR

$TOP_DIR/
        Rakefile
        lib/
                App_1/
                        ebin/
                        src/
                        include/
                        priv/
                App_2/
                        ebin/
                        src/
                        include/
                        priv/

Here are the basic rake rule and task to build erlang beam files

ERL_TOP="Path to your erlang installation"
ERLC_FLAGS = "+warn_unused_vars +warn_unused_import"

ERL_BEAM = FileList['lib/*/src/*.erl'].pathmap("%{src,ebin}X.beam")

ERL_DIRECTORIES = FileList.new('lib/*/src').pathmap("%{src,ebin}X")

ERL_DIRECTORIES.each do |d|
  directory d
  CLEAN.include d
end

rule ".beam" =>  ["%{ebin,src}X.erl"] do |t|
  output = t.name.pathmap("%d")
  sh "#{ERL_TOP}/bin/erlc -Ilib #{ERLC_FLAGS} -o #{output} #{t.source}"
end

desc "Compile Erlang sources"
multitask :erlang_modules => ERL_DIRECTORIES + ERL_BEAM

But an erlang application isn’t only beam files, we got several other
files to produce: .app, .rel, .boot and .script for example.

I made a Rakefile quite quite complete for erlang applications.

# Copyright (c) 2008 Nicolas Charpentier.
# Licence BSD

require 'rake'
require 'rake/clean'

ERL_TOP="Path_to_your_erlang_installation"
ERLC_FLAGS = "+warn_unused_vars +warn_unused_import"

ERL_BEAM = FileList['lib/*/src/*.erl'].pathmap("%{src,ebin}X.beam")

src_to_ebin = "%{src,ebin}X"

ERL_DIRECTORIES = FileList.new('lib/*/src').pathmap(src_to_ebin)

ERL_APPLICATIONS = FileList.new('lib/*/src/*.app.src')\
                            .pathmap(src_to_ebin)

ERL_RELEASE_FILES=FileList.new()
release_files = FileList.new('lib/*/src/*.rel.src')\
                        .pathmap(src_to_ebin)
release_files.each do |d|
  config_file = d.pathmap("%{ebin,src}d/../vsn.config")
  script = "scripts/make_version_info"
  vsn = `#{ERL_TOP}/bin/escript #{script} #{config_file} release_name`
  ERL_RELEASE_FILES.add d.pathmap("%X-#{vsn}.rel")
end

ERL_BOOT_FILES = ERL_RELEASE_FILES.pathmap("%{src,ebin}X.boot")
ERL_RELEASE_ARCHIVES = ERL_RELEASE_FILES.pathmap("distribs/%f")\
                                        .ext(".tar.gz")

ERL_RELEASE_ARCHIVES.each do |d|
  CLEAN.include d
end

directory "distribs"

CLEAN.include "targets"

ERL_DIRECTORIES.each do |d|
  directory d
  CLEAN.include d
end

rule ".beam" =>  ["%{ebin,src}X.erl"] do |t|
  output = t.name.pathmap("%d")
  sh "#{ERL_TOP}/bin/erlc -Ilib #{ERLC_FLAGS} -o #{output} #{t.source}"
end

rule ".app" => ["%{ebin,src}X.app.src",
                "%{ebin,src}d/../vsn.config"] do |t|
  appscript = '\'$$vsn=shift; $$mods=""; while(@ARGV){ $$_=shift;'\
  's/^([A-Z].*)$$/\'\'\'$$1\'\'\'/; if ($$mods) {$$mods.=", "} ;'\
  ' $$mods .= $$_; } while(<>) { s/%VSN%/$$vsn/; s/%MODULES%/$$mods/;'\
  'print; }\''
  script = "scripts/make_version_info"
  configuration = t.name.pathmap("%d/../vsn.config")
  vsn = `#{ERL_TOP}/bin/escript #{script} #{configuration} vsn`
  modules = FileList.new(t.name.pathmap("%d/*.beam"))\
                    .pathmap("%f").ext("")
  sh "perl -e #{appscript} #{vsn} #{modules} < #{t.source} > #{t.name}"
end

rule ".rel" => [proc {|a| a.split('-')[0..-2].join('-')\
                  .pathmap("%{ebin,src}X.rel.src")}] do |t|
  script = "scripts/make_version_info"
  configuration = t.name.pathmap("%d/../vsn.config")
  vsn = `#{ERL_TOP}/bin/escript #{script} #{configuration} release_name`
  output = t.name.pathmap("%X.rel")
  sh "#{ERL_TOP}/bin/escript scripts/make_release_file "\
     "#{t.source} #{output} #{vsn} #{ERL_DIRECTORIES}"
 end

rule ".boot" => [".rel"] do |t|
  output = t.name.pathmap("%d")
  source = t.source.ext("")
  script = "scripts/make_script"
  sh "#{ERL_TOP}/bin/escript #{script} distribs #{source} #{output}"
end

rule ".tar.gz" => [proc {|a|
                     FileList.new(a.ext("").ext("")\
                                  .pathmap("lib/*/ebin/%f.rel"))},
                   "distribs"] do |t|
  source = t.source.ext("")
  script = "scripts/make_release"
  sh "#{ERL_TOP}/bin/escript #{script} #{source} distribs without "\
  "#{ERL_TOP} #{ERL_DIRECTORIES}"
end

desc "Compile Erlang sources"
multitask :erlang_modules => ERL_DIRECTORIES + ERL_BEAM

desc "Build application resource file"
task :erlang_applications => [:erlang_modules] + ERL_APPLICATIONS

desc "Build erlang boot files"
task :erlang_release_files => [:erlang_applications] +
  ERL_RELEASE_FILES  + ERL_BOOT_FILES

desc "Build release tarball"
task :erlang_releases => [:erlang_release_files] + ERL_RELEASE_ARCHIVES

desc "Build release tarball with erts"
task :erlang_target_systems, :n, :needs=> [:erlang_release_files] +
  ERL_RELEASE_ARCHIVES do |t, args|
  source = FileList.new("lib/*/ebin/#{args.n}*.rel").ext("")
  mkdir "targets" rescue has_errors = true
  script = "scripts/make_release"
  sh "#{ERL_TOP}/bin/escript #{script} #{source} targets with "
  "#{ERL_TOP} #{ERL_DIRECTORIES}"
end

CLEAN.include "lib/*/doc/*.html"
CLEAN.include "lib/*/doc/*.css"
CLEAN.include "lib/*/doc/*.png"
CLEAN.include "lib/*/doc/edoc-info"

desc "Buid Application documentation"
task :edoc, :name, :needs => [:erlang_applications] do |t,args|
  script = "scripts/make_doc"
  sh "#{ERL_TOP}/bin/escript #{script} #{args.name} #{ERL_DIRECTORIES}"
end

desc "Buid all application documentation"
task :edocs => [:erlang_applications] do |t,args|
  ERL_APPLICATIONS.each do |application|
    name = application.pathmap("%f").ext("")
    script = "scripts/make_doc"
    sh "#{ERL_TOP}/bin/escript #{script} #{name} #{ERL_DIRECTORIES}"
  end
end

desc "Compile all project"
task :compile => [:erlang_modules, :erlang_applications]

task :default => [:erlang_releases]

To be usable you must add to your source tree some files:

$TOP_DIR/
        Rakefile
        scripts/
                make_doc
                make_release
                make_release_file
                make_script
                make_version_info
        lib/
                App_1/
                        vsn.config
                        ebin/
                        src/
                        include/
                        priv/
                App_2/
                        vsn.config
                        ebin/
                        src/
                        include/
                        priv/

All files in the scripts directory are escript files. You’ll find them
in sample_rake (14)
The vsn.config file is an erlang config file containing version
information.

{vsn,"0.1"}. %% Application version
{release_name, "Name of the release"}. %% Only needed for Apps
                                       %% containing a release

It’s still a work in progress but it’s usable. This Rakefile misses
several things:

• Dependency between .erl and .hrl
• Task and rule to launch unit tests (eunit or extremeforge)
• Driver and port compilation
• A better packaging of the application (start scripts, ….)

Of course, I’ll be glad to receive your comments and improvements.

Tags: erlang, ruby
Posted in Uncategorized | Comments (1)

Posted on Friday, 12th December 2008 by charpi

After some years of Object Oriented Programming and agility, I decided
to switch to erlang because this language is
amazingly powerful and productive.
For my point of view, it is the first choice when you want to build
robust, scalable and fault tolerant systems specially for all the
back-end architecture.
Those last days, I remembered an advice (told by one of the pragmatic
I think): A good programmer have to learn one new language/technology
per year.
It’s time for me to see something new, but what ?
I think that all ruby stuff is a good candidate for this. Ruby is
light language, helped by a lot of good tools/libraries
(rails, merb, rake).

A lot of ruby fans got a Mac … maybe that I’ll find one under my
Christmas tree.

Posted in Uncategorized | Comments (0)

Posted on Sunday, 30th November 2008 by charpi

Your company wants to make money as fast as possible. To make money,
it needs strong and competitive teams. Your team is competitive,
someone (probably some manager who don’t know you) asks it to perform
more tasks and - normally - gives you complementary means called in
another world people or resources.

Well, now that you have more men, how can you give to your boss what
he wants: more productivity ? How can you keep the competitiveness of
your team with all those “newbies” on your project ?

In the ancient age, your ancesters had a solution to integrate new
employes in teams. The first thing that the employe had to do - of
course after having filled out all administrative stuff to have access to
the car park, an e-mail account and other important things like that -
was to convert him-self to the project religion. All precepts of
this religion was wroten in holy papers that everybody called
documentation. After, and only after that the newbie known entirely
the documentation, the project chief (leader isn’t the right term)
assigns him a technical feature to test his capabilities. The feature
has to be not too risky otherwise the entire project might been
imperiled.

In the ancient age, employes spent weeks, sometimes monthes before to
give value to his boss. Due to moderns economics constraints with situation isn’t suitable anymore.

Try to think about an agile world where all companies will be
eXtreme companies. To produces money, they know that the most
important thing in the process is MEN. People make you
win or loose money. To be sure to win money, you must ensure to build
sort of team spirit where everyone is
optimized for the process. In the new world, the question
became how to optimize the way my teams work together ?

In an agile team, documentation is almost only oral so the first
step of the traditional integration will be very short. As a
replacement, we can imagine a special meeting where the whole
team could present the project (features, design) to the
newbie. This kind of meetings are great, but not essential to make
people productive for the day after.

Some agile teams use PairProgramming. Integrating the
newbie in a pair rotation, he’ll be able to talk to each other member
of the team, to share things with them. This point is the answer to
the human integration, but we was talking about money. His
productivity will increase as fast as pairs switch. At each task, he
will discover a new part of the application and his technical skills
will be improved. With a better knowledge of the system, he will be
confident enough to be active in the team life (estimation, design
discussion).

Tags: agile, extreme programming, pair programming
Posted in Uncategorized | Comments (0)

Posted on Saturday, 29th November 2008 by charpi

Looking for some emacs hacks, I re-discovered Sacha Chua’s blog.
This blog contains some nice emacs hacks (like how to blog
from emacs) but also a lot of posts about GenY and social networking.
I let you learn more about her and her job by reading her blog but
one of her talk (Networking 2.0, blogging your way out of a job and into a career)
lets me realize that I’m not blogging enough.
More precisely that I don’t pay enough attention to my social
network. Even if I know that it is important, I didn’t made the
effort to expand it and give it a real value.
In consequence my first decision is to really use this blog to share my thoughts and
my skills. The first expected velocity is at least one article/post per week.
I hope the GTD method will help me to succeed to keep the rythm.

Tags: social networking
Posted in Uncategorized | Comments (2)

Posted on Sunday, 19th October 2008 by charpi

I worked on erl_mock in order to publish my spike around mocks in
erlang. When I was almost ready to make the package, something interesting happened: my tests failed.

More precisely, they failed when run by the ‘future’ version of extremeforge.
After the first surprise, I find the reason. Extremeforge use 2 nodes,
the first one will find, order tests then send all code on the second
node in order to be run. Like this, extremeforge keeps the first node
totally safe if for some reasons a test module (or applicative module)
gets wrong.

As my library intends to load, unload dynamically modules the test
node is lost because it can’t find the beam files.

Conclusion: Don’t use mock,

Anyway I’ll publish my library soon

Tags: erlang, extremeforge, mock object
Posted in Uncategorized | Comments (0)