Nicolas Charpentier’s blog

Posted on Sunday, 18th January 2009 by charpi

I’ve updated erl_rake (265) with some enhancements:
- Refactoring of some rules that used external scripts (perl or escript)
- Header dependencies are handled
- ASN1 files are built

[edit] The package works with ruby 1.8.7

Tags: erlang, rake
Posted in Uncategorized | Comments (1)

Posted on Tuesday, 6th January 2009 by charpi

I finally uploaded the presentation that we (J.Couvreur and I) did during the Valtech-Days 2007 on SlideShare:

Posted in Uncategorized | Comments (0)

Posted on Friday, 19th December 2008 by charpi

In my previous post, I planned to have a closer look at the ruby’s
world. As the erlang world misses a real compilation tool, I’ve decided
to make a attempt to use rake.

Usually, Erlangers use make to build their
application. It’s a powerful tool but its syntax sucks a bit. With
this complex syntax it’s hard to keep makefiles clean and with time
they become very obscure.
There are a lot of other popular construction tools:

• Ant : But I don’t like to edit XML files
• Scons: Written in python. Some erlang tasks exist but I never had
  time to look at it.
• Rake: The make utility written in ruby. Rails integrates rake
  tasks to automate all the development process.

I tried rake to replace makefiles on my personal projects looking at
those posts as example Building erlang with rake and Erlang - Make, Rake and Emake.

Usually, my erlang projects are organized according to OTP principles.
All applications are under a lib directory, so I decided to put the
Rakefile just under the $TOP_DIR

$TOP_DIR/
        Rakefile
        lib/
                App_1/
                        ebin/
                        src/
                        include/
                        priv/
                App_2/
                        ebin/
                        src/
                        include/
                        priv/

Here are the basic rake rule and task to build erlang beam files

ERL_TOP="Path to your erlang installation"
ERLC_FLAGS = "+warn_unused_vars +warn_unused_import"

ERL_BEAM = FileList['lib/*/src/*.erl'].pathmap("%{src,ebin}X.beam")

ERL_DIRECTORIES = FileList.new('lib/*/src').pathmap("%{src,ebin}X")

ERL_DIRECTORIES.each do |d|
  directory d
  CLEAN.include d
end

rule ".beam" =>  ["%{ebin,src}X.erl"] do |t|
  output = t.name.pathmap("%d")
  sh "#{ERL_TOP}/bin/erlc -Ilib #{ERLC_FLAGS} -o #{output} #{t.source}"
end

desc "Compile Erlang sources"
multitask :erlang_modules => ERL_DIRECTORIES + ERL_BEAM

But an erlang application isn’t only beam files, we got several other
files to produce: .app, .rel, .boot and .script for example.

I made a Rakefile quite quite complete for erlang applications.

# Copyright (c) 2008 Nicolas Charpentier.
# Licence BSD

require 'rake'
require 'rake/clean'

ERL_TOP="Path_to_your_erlang_installation"
ERLC_FLAGS = "+warn_unused_vars +warn_unused_import"

ERL_BEAM = FileList['lib/*/src/*.erl'].pathmap("%{src,ebin}X.beam")

src_to_ebin = "%{src,ebin}X"

ERL_DIRECTORIES = FileList.new('lib/*/src').pathmap(src_to_ebin)

ERL_APPLICATIONS = FileList.new('lib/*/src/*.app.src')\
                            .pathmap(src_to_ebin)

ERL_RELEASE_FILES=FileList.new()
release_files = FileList.new('lib/*/src/*.rel.src')\
                        .pathmap(src_to_ebin)
release_files.each do |d|
  config_file = d.pathmap("%{ebin,src}d/../vsn.config")
  script = "scripts/make_version_info"
  vsn = `#{ERL_TOP}/bin/escript #{script} #{config_file} release_name`
  ERL_RELEASE_FILES.add d.pathmap("%X-#{vsn}.rel")
end

ERL_BOOT_FILES = ERL_RELEASE_FILES.pathmap("%{src,ebin}X.boot")
ERL_RELEASE_ARCHIVES = ERL_RELEASE_FILES.pathmap("distribs/%f")\
                                        .ext(".tar.gz")

ERL_RELEASE_ARCHIVES.each do |d|
  CLEAN.include d
end

directory "distribs"

CLEAN.include "targets"

ERL_DIRECTORIES.each do |d|
  directory d
  CLEAN.include d
end

rule ".beam" =>  ["%{ebin,src}X.erl"] do |t|
  output = t.name.pathmap("%d")
  sh "#{ERL_TOP}/bin/erlc -Ilib #{ERLC_FLAGS} -o #{output} #{t.source}"
end

rule ".app" => ["%{ebin,src}X.app.src",
                "%{ebin,src}d/../vsn.config"] do |t|
  appscript = '\'$$vsn=shift; $$mods=""; while(@ARGV){ $$_=shift;'\
  's/^([A-Z].*)$$/\'\'\'$$1\'\'\'/; if ($$mods) {$$mods.=", "} ;'\
  ' $$mods .= $$_; } while(<>) { s/%VSN%/$$vsn/; s/%MODULES%/$$mods/;'\
  'print; }\''
  script = "scripts/make_version_info"
  configuration = t.name.pathmap("%d/../vsn.config")
  vsn = `#{ERL_TOP}/bin/escript #{script} #{configuration} vsn`
  modules = FileList.new(t.name.pathmap("%d/*.beam"))\
                    .pathmap("%f").ext("")
  sh "perl -e #{appscript} #{vsn} #{modules} < #{t.source} > #{t.name}"
end

rule ".rel" => [proc {|a| a.split('-')[0..-2].join('-')\
                  .pathmap("%{ebin,src}X.rel.src")}] do |t|
  script = "scripts/make_version_info"
  configuration = t.name.pathmap("%d/../vsn.config")
  vsn = `#{ERL_TOP}/bin/escript #{script} #{configuration} release_name`
  output = t.name.pathmap("%X.rel")
  sh "#{ERL_TOP}/bin/escript scripts/make_release_file "\
     "#{t.source} #{output} #{vsn} #{ERL_DIRECTORIES}"
 end

rule ".boot" => [".rel"] do |t|
  output = t.name.pathmap("%d")
  source = t.source.ext("")
  script = "scripts/make_script"
  sh "#{ERL_TOP}/bin/escript #{script} distribs #{source} #{output}"
end

rule ".tar.gz" => [proc {|a|
                     FileList.new(a.ext("").ext("")\
                                  .pathmap("lib/*/ebin/%f.rel"))},
                   "distribs"] do |t|
  source = t.source.ext("")
  script = "scripts/make_release"
  sh "#{ERL_TOP}/bin/escript #{script} #{source} distribs without "\
  "#{ERL_TOP} #{ERL_DIRECTORIES}"
end

desc "Compile Erlang sources"
multitask :erlang_modules => ERL_DIRECTORIES + ERL_BEAM

desc "Build application resource file"
task :erlang_applications => [:erlang_modules] + ERL_APPLICATIONS

desc "Build erlang boot files"
task :erlang_release_files => [:erlang_applications] +
  ERL_RELEASE_FILES  + ERL_BOOT_FILES

desc "Build release tarball"
task :erlang_releases => [:erlang_release_files] + ERL_RELEASE_ARCHIVES

desc "Build release tarball with erts"
task :erlang_target_systems, :n, :needs=> [:erlang_release_files] +
  ERL_RELEASE_ARCHIVES do |t, args|
  source = FileList.new("lib/*/ebin/#{args.n}*.rel").ext("")
  mkdir "targets" rescue has_errors = true
  script = "scripts/make_release"
  sh "#{ERL_TOP}/bin/escript #{script} #{source} targets with "
  "#{ERL_TOP} #{ERL_DIRECTORIES}"
end

CLEAN.include "lib/*/doc/*.html"
CLEAN.include "lib/*/doc/*.css"
CLEAN.include "lib/*/doc/*.png"
CLEAN.include "lib/*/doc/edoc-info"

desc "Buid Application documentation"
task :edoc, :name, :needs => [:erlang_applications] do |t,args|
  script = "scripts/make_doc"
  sh "#{ERL_TOP}/bin/escript #{script} #{args.name} #{ERL_DIRECTORIES}"
end

desc "Buid all application documentation"
task :edocs => [:erlang_applications] do |t,args|
  ERL_APPLICATIONS.each do |application|
    name = application.pathmap("%f").ext("")
    script = "scripts/make_doc"
    sh "#{ERL_TOP}/bin/escript #{script} #{name} #{ERL_DIRECTORIES}"
  end
end

desc "Compile all project"
task :compile => [:erlang_modules, :erlang_applications]

task :default => [:erlang_releases]

To be usable you must add to your source tree some files:

$TOP_DIR/
        Rakefile
        scripts/
                make_doc
                make_release
                make_release_file
                make_script
                make_version_info
        lib/
                App_1/
                        vsn.config
                        ebin/
                        src/
                        include/
                        priv/
                App_2/
                        vsn.config
                        ebin/
                        src/
                        include/
                        priv/

All files in the scripts directory are escript files. You’ll find them
in erl_rake (265)
The vsn.config file is an erlang config file containing version
information.

{vsn,"0.1"}. %% Application version
{release_name, "Name of the release"}. %% Only needed for Apps
                                       %% containing a release

It’s still a work in progress but it’s usable. This Rakefile misses
several things:

• Dependency between .erl and .hrl
• Task and rule to launch unit tests (eunit or extremeforge)
• Driver and port compilation
• A better packaging of the application (start scripts, ….)

Of course, I’ll be glad to receive your comments and improvements.

Tags: erlang, ruby
Posted in Uncategorized | Comments (2)

Posted on Friday, 12th December 2008 by charpi

After some years of Object Oriented Programming and agility, I decided
to switch to erlang because this language is
amazingly powerful and productive.
For my point of view, it is the first choice when you want to build
robust, scalable and fault tolerant systems specially for all the
back-end architecture.
Those last days, I remembered an advice (told by one of the pragmatic
I think): A good programmer have to learn one new language/technology
per year.
It’s time for me to see something new, but what ?
I think that all ruby stuff is a good candidate for this. Ruby is
light language, helped by a lot of good tools/libraries
(rails, merb, rake).

A lot of ruby fans got a Mac … maybe that I’ll find one under my
Christmas tree.

Posted in Uncategorized | Comments (0)

Posted on Sunday, 30th November 2008 by charpi

Your company wants to make money as fast as possible. To make money,
it needs strong and competitive teams. Your team is competitive,
someone (probably some manager who don’t know you) asks it to perform
more tasks and - normally - gives you complementary means called in
another world people or resources.

Well, now that you have more men, how can you give to your boss what
he wants: more productivity ? How can you keep the competitiveness of
your team with all those “newbies” on your project ?

In the ancient age, your ancesters had a solution to integrate new
employes in teams. The first thing that the employe had to do - of
course after having filled out all administrative stuff to have access to
the car park, an e-mail account and other important things like that -
was to convert him-self to the project religion. All precepts of
this religion was wroten in holy papers that everybody called
documentation. After, and only after that the newbie known entirely
the documentation, the project chief (leader isn’t the right term)
assigns him a technical feature to test his capabilities. The feature
has to be not too risky otherwise the entire project might been
imperiled.

In the ancient age, employes spent weeks, sometimes monthes before to
give value to his boss. Due to moderns economics constraints with situation isn’t suitable anymore.

Try to think about an agile world where all companies will be
eXtreme companies. To produces money, they know that the most
important thing in the process is MEN. People make you
win or loose money. To be sure to win money, you must ensure to build
sort of team spirit where everyone is
optimized for the process. In the new world, the question
became how to optimize the way my teams work together ?

In an agile team, documentation is almost only oral so the first
step of the traditional integration will be very short. As a
replacement, we can imagine a special meeting where the whole
team could present the project (features, design) to the
newbie. This kind of meetings are great, but not essential to make
people productive for the day after.

Some agile teams use PairProgramming. Integrating the
newbie in a pair rotation, he’ll be able to talk to each other member
of the team, to share things with them. This point is the answer to
the human integration, but we was talking about money. His
productivity will increase as fast as pairs switch. At each task, he
will discover a new part of the application and his technical skills
will be improved. With a better knowledge of the system, he will be
confident enough to be active in the team life (estimation, design
discussion).

Tags: agile, extreme programming, pair programming
Posted in Uncategorized | Comments (1)