Discontinuing my musings here.

Dont worry, I am now blogging from
http://blog.joshsoftware.com

It has a wealth of information there, including all these posts here!

Ciao.

Scottish Ruby Conference 2010 roundup-1

Following some inspirational talks with some great folks at Scottish Ruby Conference, I am going to apply the pomodoro technique for this blog post – 25 minutes of writing on this post!

These are just some of the things you get to learn at this no-nonsense conference, which is full of geeks, innovators and innovative geeks. This time, the renamed conference (earlier called Scotland on Rails) had a very good mix of talks on Ruby, Rails, good practices and even philosophy. I could get into details of each talk but they have all been recorded and will be out soon – it would be so much better to listen in on them instead of reading about them here !

What I intend to do here  is to put forth some views of the talks, the styles, the key take-home points and what I learnt.

Jim Wreich started with an excellent keynote and also give an awesome theatrical performance in the closing key-note! He was very clear in getting into my head that the key values of a good programmer:

– thinking out of the box. (Try to innovate!)

– accept duality (code & objects are the same things from different points of view!)

– abstractions are good. (Write generic, re-usable code!)

In the closing key-note, along with Joe O’Brian, it was made clear about why we should explain the benefits (and not features) of Ruby to the decision making management – no point in talking about Ruby features like blocks, iterators and TDD/BBD instead of business aspects about how the maintenance cycle of product can be reduced drastically by using Ruby and how Ruby seamlessly integrates with legacy systems.

People had come from all over the globe – from US, Europe, Australia, New Zealand and us from India. If this wasn’t good enough for diversity, the enormous variety of food at lunch made up for it. The venue, Royal College of Physicians was ‘unbelievable’. I sat in the ‘Great Hall’ admiring the ceiling and in the library admiring the expanse of the books lined in the racks beside and the so beautifully architected Amphi-theater.

The talks were well distributed in 3 tracks and each talk had a sizable crowd. There were 3 talks which discussed how to write good code (‘write bad code’ by Gwyn Morfey, ’10 most errors in Rails’ by Exceptional and ‘Your doing it wrong’ by Tammer Saleh). Each had a different style – my personal favourite being Tammer’s talk – it seems straight from the heart. Gwyen’s talk was very well staged to show how to ‘really’ work and it was really funny.

Tyler McMullen’s talk on distributed rack applications was amazing and new way to look at things. Imagine different sections on a web-page being serviced by different application servers — no! Not using ajax requests but genuine HTTP distributed requests for rendering a single page! He demonstrated it really well and there were some of us in the audience who installed the distributed_demo gem from gemcutter to see it all in action!

Joesph Wilk’s talk was precision perfect on scaling cucumber test cases – there is indeed a lot of potential here. Scott Chacon ‘showed off’ the git internal plumbing with an excellent and very cool presentation – I am inspired now and do plan to use a single git repository for distributed deployment on different machines with different roles.

Lightening talks were really good !! I shall be using RedCar (the Ruby IDE) and Showoff, the Rails presentation gem from now on. Ryan’s talk of 2500 words (speed reading) was an awesome insight into the life-cycle of a software programmer who found the worth of testing his code!

Keeping in line with the pomodoro technique, I shall stop now and write another detailed post on the conference soon 😉 I stretched this post to 35 minutes — hell nobody’s perfect!

Pune Rails Meetup#3 ideas?

At Josh, We did some brainstorming and came up with some ideas that our team suggested would be good learning topics at Rails meetup.

To put that into effect, we hope to raise the bar for Pune Rails Meetup#3 and take some technical sessions ( basic and advanced). People are expected to come with their laptops configured with Rails.

Please add comments to this post with a list of topics if you are interested in learning about something in Ruby or in Rails! We need to shortlist these topics and we need feedback. We then need to finalize the comprehensive topic list – maybe get some voting on the topics from Rubyists.

Ruby

1. Ruby meta-programing (advanced)

2. Regular Expressions in Ruby (basic)

3. forking using spork (advanced)

4. Ruby threads (basic)

Rails

1. polymorphic associations & STI (basic)

2. writing a rails plugin (advanced)

3. Social networking connects – authentication & sharing via openIDs (basic)

4. DelayedJob  (basic)

5. Ruby event machine / nanite / RabbitMQ (advanced)

6. Deployment in the cloud (advanced)

Hosting the event

1. We can pool in some funds (or get some sponsors) and host this event at PYC, deccan. If there is some company or venue that can host this event, please contact us.

Metaprogramming with RadiantCMS

My short stint with Radiant in the near past was actually quite interesting ! — No I did not design websites or webpages. What I am talking about is the internals – making radiant plugins and creating radius tags, the overwhelming use of meta-programing and the ease of overriding and extending the base or extensions functionality.

First-up, Radiant is NOT for newbies. If you are looking to design some static webpages or basic content, I would probably not recommend radiant – there are other tools for this. Radiant however is very exciting if you are planning on things which are more complicated than static web-pages, like authorization based views, galleries, custom website configuration, etc. etc. If you are well versed and aware of page-parts, snippets, layouts and how they all interact with each other, then you can look at Radiant

Though radiant supports ALL basic HTML tags, it also supports radius tags — loaded via extensions. The excellent open source contributions as radiant extensions leaves very little to do on your own 😉 but then again – there is nothing called a free lunch.

Its easy to override any existing functionality in core radiant or its extensions by some simple meta-programing. Some good insight here

module MyExtension
  module SiteControllerExt
    def self.included(base)
      base.class_eval {
        def new_method
          ... # whatever new functionality you want to add to the site controller
        end
      }
    end
  end
end

You can then activate the SiteController by adding these additional methods in the meta-class.

def activate
  SiteController.send(:include, MyExtension::SiteControllerExt)
end

You can modify existing behavior of any method by using the alias_method_chain as in the following example:

module MyExtension
  module SiteControllerExt
    def self.included(base)
      base.class_eval {
        def site_with_clone
          ... # pre-processing
          site_without_clone
          ... # post-processing
        end

        alias_method_chain :site, :clone
    end
  end
end

The alias chain method ensures that we can write some code before or after the core method. Similarly you can override the views. More details here The above are some very simple basic rules of meta-programing.

base.class_eval {

This line of code is actually accessing the meta-class and changing it. In other words, we evaluate the Class object and update the methods in it! This helps us add or modify the functionality of the class.

Radiant Multihosting

Radius and multisite hosting is cool — but you should know how it works to reap the benefits. The following extensions help out: multi_site, scoped_admin.

Both these ensure that all pages are site_scoped. Using some basic meta-programing, each site now has its own ‘scope’. Remember the all important Page.current_site !! Sandip and  I busted our heads trying to figure out why the following was ‘wierd’. On the console

>> Site.find 13
=> #<Site id: 13, homepage_id: 74, position: 4, ... >
>> s = Site.find(13).layouts
=> []

This implies that The Site (with id 13) does not have any layouts! However, a direct find_by_sql had a different answer in store for us !

>> Layout.find_by_sql("select * from layouts where site_id=13")
=> [#<Layout id: 29, name: "Basic", content: "<html>\r\n ... >
>>

This implies that the data is there in the database BUT not available from the associations! So frigging wierd. Then we got into the details of multi_site plugin and realized the ‘error in our ways’.  Radiant is by default single site scoped. When we work with multi_site, we need to fool Radiant and tell it which site we are working with! The following works:

>> Page.current_site = Site.find(13)
>> s = Site.find(13)
>> s.layouts
=> [#<Layout id: 29, name: "Basic", content: "<head> ... lock_version: 0, site_id: 13> ]

So, it extremely important to ensure that the Page.current_site is set when working with multi-hosting. So, why did this happen or how is it so, you ask? The multi_site extension, uses some pretty cool metaprograming to override (using alias_method_chain) find, count, max, min, and other ActiveRecord methods. It defaults to ‘default_site’ if the current_site is not set !

Extension Loading and Configurations

Using Radiant extensions is what makes Radiant radiant! 🙂 There are plenty of extensions that can be easily installed in the radiant setup. However, its important to remember the following:

• environment.rb config.extensions is the file to ensure ‘extension load ordering’. This is very important especially if you want to override some extension functionality.
• The :all symbol in config.extensions implies the remaining extensions. BUT remember that these extension are then loaded in ALPHABETICAL order! So, be careful to either ensure your call your extension ‘zzz_myextension’ (LoL) or ensure that any extensions you override load before you load your own extension.
• The ‘settings’ extension is pretty cool for managing configurations. To add a configuration, you can simply do this:

Radiant::Config["key"] = value

• Recommended way of adding default configurations is the have yml file use YAML::load_file in the extension activate function.
• Remember, the configuration is saved in the database — so if you override the value from the GUI, the database gets updated — NOT the yml file. So you have to be careful.

Radius tags

Radius tags look something like this:

<r:content />
<r:gallery id="id> ... </r:gallery>

Radius tags help in managing the programing part of dynamic pages. You can add your own radius tags to ease the effort of inserting HTML code. Some cool examples are:

<r:gallery:lightbox_stuff />

This generates the HTML for loading JS and CSS for lightbox — simple. To create a lightbox:

<r:gallery id="3">
 <r:gallery:lighbox />
</r:gallery>

To add your own radius tags, you need to do the following in a module.

module MyExtensionModule
 module CustomTags

 include Radiant::Taggable

 tag "tractor" do |tag|
  desc "description of the tag"
  tag "tagname" do |tag|
    ... #custom logic
  end
 end

In the extension loader file, if you want to use the radius tags in the Pages, ensure this:

Page.send :include, MyExtensionModule::CustomTags

Enjoy the radiance!

PHPcamp – CakePHP Vs Ruby On Rails

On 9th Jan, ’10,  I presented my topic “CakePHP Vs Rails” at PHPCamp. It was well received and it felt nice to have been able to present such a topic to PHP devs. I was expecting a ‘fight’ (infact quite a few people did too) but I did keep the tempo low and to a discussion and not a debate – expect the unexpected !

Having said this, the controversy started long before my talk though – right from the choice of the title. My opinion is that almost 90% of PHP development is web-development, so the topic – ‘PHP Vs Ruby On Rails’ was on the button. However, Amit Singh, Dhananjay Nene disagreed – they felt it was comparing apples to oranges. I complied to their request and changed the title – Fair enough. BUT how do you explain that there were 2 hands raised in a room of about 80, who said that they used PHP earlier for non web-development ;). All in all, I did finally compare Ruby with PHP and CakePHP with Rails.

PHPcamp had 2000+ registrants and the venue was ‘packed’ – it was very exciting to go a PHPcamp as a rubyist. I wondered if people sensed a wolf among the sheep (couldn’t resist that taunt, 😉 my apologies ) and when I went up to the wikiwall to note my topic, I did get a few curt nods and some smiles and a Ah-you-are-the-rails-guy looks!

I kept my talk after Aditya’s talk on ‘Introduction to CakePHP’ because it would help my cause for people to know more about CakePHP when I compared it with Rails. The talk was really good — and I could see a lot of my effort reducing in explaining MVC, RESTfulness, relationships etc. the rails way. The slides are posted here.

Before I did a deep-dive, I wanted to grasp the type of audience I was dealing with and realized a decent mix of professionals, students and even a few entrepreneurs. I decided to keep my talk a little less-technical, should people sleep off ;).

I started with a brief comparison of PHP and Ruby. The Pure OO Ruby, mention of blocks and meta-programing. The idea of blocks of code being sent as parameters to methods and the idea of defining a method on an object seemed to have intrigued the crowd — however, given that I had to keep the talk basic and not too technical, I am sure it did go a little over the top. Never mind – at least I had got everyone’s attention. In a lighter moment, I did mention the use of throwing shoes as opposed to rotten tomatoes if required. I did not have the pleasure of getting as famous as famous presidents as nobody lost any shoes !

I then gave a brief comparison of CakePHP and Rails which brought me to the main discussion. CakePHP has a directory structure similar to rails and I believe it has been modeled around Rails (at least in the beginning). The model based relationships are similar to Rails, except for HasAndBelongsToMany. In Rails it has been changed to has_many :through =>  relationship. I particularly liked the ‘on the fly’ binding in CakePHP – its pretty cool. In rails this is achieved by a little bit of meta-programing and adding class methods to the model to load new functionality.

Both frameworks preach being model-heavy which is excellent. However I was a little surprised with Aditya mentioning that scaffolding should not be used extensively. I do believe that the RESTful code generators in Rails are excellent and scaffolding should be used extensively to get a jump-start.

RESTfulness was skimmed through the CakePHP talk but I re-iterated in my talk the importance of inherent RESTfulness that helps in interoperability. I could see a heads nodding when I used an example of user account creation URL — got plenty of answers, /add_user, /create_users, etc. and the look of enlightenment when I explained how HTTP verbs are used to ensure that /users (with GET) will get the list of users and /users (with POST) will create a user.

There was finally a person who raised the question of speed and performance in pure PHP code Vs Ruby Code. Now, it is indeed undeniable that PHP code interprets faster then Ruby, in fact as the gentleman said – “The 10 lines of PHP code will run faster than the 3 line of similar Ruby code”.  Agreed!

I was filled with pride that he mentioned the really cool part of Ruby coding — LESS CODE, which in turn means less bugs. (Yes – I know there will be a lot of PHP enthusiasts who will flame me for this — but it is a fact!). Ok – back to the speed and performance issue — in my opinion, CakePHP would run much slower than Rails (courtesy internal caching). However, pure PHP code would run a tad faster than Rails. This is definitely debatable but I would forgo some performance and speed in favour of lesser lines of code. In addition to this, the ruby interpreters (MRI and Jruby) are reaching epic proportions of speed in interpreting.

I then moved onto what I found missing (or lacking) in CakePHP as opposed to Rails.

• The one thing that stands out is the use of migrations. I was shocked to see event the examples on the dev site, showing SQL file for creating and updating the database.
• The lack of deployment information was understandable (standard Apache with mod_php I guess) but I think this is lacking. Capistrano is something I did not mention in the talk but its things like this that make deployment simple! I was talking to some of the experts later, Amit, Aditya and Shardul after the talk and they told me that most still rely on scp and ftp for production deployment! I hope that is incorrect, but was a little taken aback never-the-less.
• There does not seem to be much information about the use of caching in CakePHP — it seemed to be more out of the do-it-yourself manual via memcached. Rails internal caching is superb – object caching, query caching and fragment caching really get things going. Though I found information about caching in CakePHP, I wonder how much people use it.
• The Rails magic of instance variables in controllers directly being accessible in views is cool. CakePHP requires the use of set() to ensure this. Decent enough though lot of scope of mistakes, IMO.
• Routing and RESTful routing seemed a little behind compared to rails. Aditya did mention that default RESTful routes are now available in CakePHP but from the documentation and usage it seemed lacking. I love the following routing in Rails which gives me all the RESTful CRUD automagically.

map.resources :users

• Polymorphic associations were missing (but I may be wrong here — I am sure its supported).

To conclude, the fact is that CakePHP makes life unbelievably happier for PHP web development – I would call it a boon! I personally do see a lot of similarity between CakePHP and Rails – in fact I did tweet that that what Rails does, CakePHP follows. However, surprisingly, Larry E. Masters disagreed with me. His opinion is very important in the context as he is the Co-founder of CakePHP! Check out his interesting replies to my tweet here and here Now, that is spunk and attitude ideal for competitive development – Its highly appreciated as it will make things better! I would like to ‘twibate’ on it sometime, but I think that would digress from this topic 🙂

All in all, I say to PHP devs, — Try the real thing!

Rails Vs CakePHP

View more presentations from gautamrege.

Pune Rails Meetup #2

It was great to be a part of the Pune Rails Meetup which was held yesterday (19th December, 2009) at ThoughtWorks, Pune. It was an idea initiated by Anthony Hsiao of Sapna Solutions which has got the Pune Rails community up on their feet. Helping him organize was a pleasure!

It was great to see almost 35 people for this meet — it was a probably more than what we expected. It was also heartening to see a good mix in the crowd – professionals in rails, students working in rails and students interested in rails – not to forget entrepreneurs who were very helpful.

Proceedings began with Vincent and Anup from ThinkDRY gave an excellent presentation on BlankApplication – a CMS++ that they are developing. I say CMS++ because its not just another CMS but has quite a lot of ready-to-use features that gets developers jump-started. There were interesting discussions regarding how ‘workspaces’ are managed and how its indeed easier to manage websites.

After this technical talk, I spoke next on my experience at the Lone Star Ruby Conference in Texas. I tried to keep the session interactive with the intention of telling everyone how important it is to know and use Ruby effectively while working in Rails. Dave Thomas’s references to the ‘glorious imperfection’ of Ruby did create quite a buzz. To quote a little from Dave’s talk:

name {}

This is a method which takes a block as a parameter but the following line is a method which takes a has as a parameter! A simple curly parenthesis makes all the difference!

name ( {} )

Similarly, the following line is a method m() whose result is divided by ‘n’ whose result is divided by ‘o’

m/n/o

but add a space between this and its a method m() which takes a regular expression as a parameter!

m /n/o

It was nice to see everyone get involved in these interactive sessions. More details about my experience at LSRC is here.

After this there was another technical talk about a multi-app architecture  that has been developed by Sapna Solutions. Anthony and Hari gave a talk on this and it was very interesting to see it work. Using opensource applications like shopify, CMS and other social networking apps to work with a shared-plugin and a single database, its possible to create a mammoth application which is easily customizable and scalable.

Hari did mention a few problems like complexity in migrations and custom routes which they currently ‘work-around’ but prefer a cleaner approach. Some good suggestions were provided by Scot from ThoughtWorks regarding databases. I suggested some meta-programing to align models. Working with git submodules and ensuring rake scripts to sync up data, this indeed seems to have a lot of potential.

There were some new entrepreneurs from VectorBrook who have already developed a live application in Merb which they discussed and explained details of. It was good to hear about how they managed performance and scalability testing. The Q&A forum which was the next event was extremely interactive. Some of the discussions were:

Which are really great CMS in Rails?

There were some intense discussions regarding RadiantCMS, Adva and even BlankApp. The general consensus was a ‘programmable CMS’ Vs WYSIWYG. Those who prefer more of the content management prefer CMS’s like Drupal, Joomla. Those who prefer more customization via programing and code, prefer Radiant. This topic could not close and is still open for discussion.. Do comment in your views – I am a radiant fan 😉

What about testing? Cucumber, Rspec, others?

Usually its still adhoc – testing is expensive for smaller firms — so adhoc blackbox testing is what is done. I opined that cucumber and rspec ROCK! Cucumber is great for scenario testing and testing controller logic and views. Rspec is great for Direct Model Access and Cucumber can make great use of Webrat for browser testing.

In Rpsec, when do we use mocks and stubs?

It was suggested that mocks and stubs should be used when there are no ready model and code. If the code is ready, its probably just enough not to use mocks and stubs directly. Comments welcome on this!

How do you do stress testing?

Stress testing, concurrency testing and performance testing can be done using http-perf. It was interesting to note that ____ have actually done their own implementation for stress and concurrency testing. I recommended they open source it.

How are events, scheduled job and delayed jobs handled?

This was my domain 🙂 Using delayed_job is the way to go. Following the leaders (github) and using Redis and resque would be great too but definitely not backgrounDrb or direct cron!

What project management tools do you use? Pivotal Tracker, Trac, Mingle?

Pivotal tracker suits startup needs. Mingle rocks but becomes expensive. Scott ? 😉 Dhaval from TW mentioned how easy it was to co-ordinate an ‘mingle’ with their 200 strong team over distributed geographies.

Which SCM do you use? git, svn, cvs?

People have been very comfortable with git and more and more are migrating from svn to git.  It was heartening to see that nobody uses CVS 🙂 Jaju (I have have misspelt) gave an excellent brief about how code and diffs can be squished and ‘diff’ed with another repository before the final merge and push to the master. Dhaval gave an idea about how they effectively used git for managing their 1GB source code (wow!)

Some pending questions – probably in next meet-up

1. Which hosting service do you use and why?
2. TDD or BDD?

Suggestions are welcome!

Migrating an existing deployment to Heroku

It was fun — pure unadulterated fun!!

Now, I started looking at heroku since our current deployment on a linode was getting a little bulky. Our client has been complaining of ‘suddenly things slowing down’ and ‘site not working’. The site is used at a hit rate of 36 Requests per minute and we need to keep our hosting costs down — so what is the solution???

Currently, its hosted on a linode (740MB) memory and and nginx + 3 thin cluster. There is monit which ensures that ANY thin server going above 140MB is restarted. BackgrounDrb is taking a lot of memory too ( an unavoidable 160MB) and we have ferret to make things more complicated !! 🙂

Enter Heroku – first thing that got me hooked was a ‘git push heroku’ DEPLOYS your application! Fantastic. I have been a capistrano freak till now and loved it – but its all gone now! (wonder if thats good or bad though).  LoL 🙂

So, to get started, you sign-up on heroku and follow instructions on your My Apps page:

$ heroku create
$ git init && git add . && git commit -m"initial checkins"
$ git push heroku

I used the Blossom account (5 MB space) and was pleasantly surprised to see it work ‘out of the box’. It created some wierd named sub-domain for me which I changed using

$ heroku rename <new-name>

Digging deeper, I wanted to deploy the application ‘as is’ but that was not to be:

Setup Woes

Heroku uses PostGres and we had used MySQL – you need to ensure that there are no special data types being used. As goes Murphy’s law – we had used blob and had to change that the bytea (bytearray in PG). The interesting thing I found was that there is NO way to drop the database and re-create it. The answer lies in:

$ heroku rake db:schema:load

Production environment is assumed. I was not sure which environment I am in till I issued the command (after reading up) that I was in production mode:

$ heroku config
RACK_ENV => production

I was also curious to see how I get the logs and check the console and it really ROCKS:

$ heroku logs
$ heroku console

Alas I could not find a way to access the dbconsole (like what I do sometimes with script/dbconsole -p). Well, that is the way it is!

Ferret Woes

Heroku does not use ferret – it uses solr and sunspot. I had to install these additional gems using the gem manifest i.e. the .gems file located at RAILS_ROOT.

rupport
acts_as_ferret
prawn
$ git commit
$ git push heroku

Since acts_as_ferret is installed, it loads the Rails models and ‘expects’ the database to be in place. So, I had to comment out ALL the references to acts_as_ferret to be able to run a

$ heroku rake db:migrate

So far so good. Now, since ferret requires to index files and heroku mounts the deployment on a read-only filesystem, we need to tweak the configurations to ensure that the index is created in tmp/ (the are where we can write stuff). Add the following lines to the config/environments/production.rb

require 'acts_as_ferret'
ActsAsFerret.index_dir = "#{RAILS_ROOT}/tmp/index"

Now, default ferret production environment is to start a Drb server on 9010 port — and Heroku will not allow this! So, we have to comment out all the configuration in config/ferret_server.yml to ensure that this loads like it does in development. I hate this but no alternative for now. I do plan to migrate to the WebSolr that Heroku is offering but its at 20$ per month! Lets see how that works out.

This should at least get you started on ferret – phew!

BackgrounDrb woes

BackgrounDrb comes with its own baggage I must say. Heroku proposes the use of DelayedJob (yippee) and I have just this plan, so rather than even try to figure out a way to get backgrounDrb to work on Heroku, I simply commented out the config/backgroundrb.yml settings (to ensure it does not get initialized).

I also had to comment out all the MiddleMan references in the controller. The backgrounDrb workers are easily  modified so that they can work as DelayedJob objects ;). I have mentioned this in my earlier posts Moving from BackgrounDrb to DelayedJob

Once this was done, I did not face any problems and after some quick code fixes, git commits and git push’es, I was able to get my heroku application up and running.

All in a days work! Thats really amazing. Now, the next step is to get WebSolr  and DelayedJob integrated and see if I can move onto a bigger storage setup — a couple of months will let me know if this move was right.

RPCFN#3 Short-circuit

It was great to see so many solutions for the problem statement I had set. After an initial ‘shock’ from all electrical engineers, the problem was clearly understood as a variant of the ‘shortest path algorithm’. I hope I was able to derive a laugh from the cryptic clue ‘short-circuit’.

Speaking of ‘derive’, I did see most solutions going into a class implementation of  ‘class Graph’ or ‘class Vector’. Though its pretty cool to see a complete packaged solution, I believe this was a slight overkill. This was NOT the basis of judging but I do feel that ruby has its essence in getting the job done in far lesser code. i.e. less LOC. So, I personally did not foresee 3-4 classes with inheritance in them.. I do totally agree its an excellent way to show-case one’s skills  😉

Djikstra’s algorithm seems to be the popular hit for solving this problem – however, I do feel a recursive solution than an iterative one is more appropriate for this. Again this is just an opinion but to get a sense of the power of ruby programing, a recursive program probably paints a better picture. Its also very concise and readable.

A lot of effort was spent in ‘initializing’ the data-structure. This was really nice to see — unlike me, who took a short-cut. The aim was to see the algorithm but I was really proud to see ‘complete’ solutions. Test cases were written in most solutions and it was a pleasure to see them run.

It was interesting to see various forms of Infinity:

Infinity = 1 << 64
Infinity = 1 << 32
Infinity = 1.0 / 0 # simple and ideal
Infinity = 1000000 # incorrect

It was interesting to note that very few catered for multiple shortest paths, though the question was raised in the comments earlier. Not trying to be a hypocrite here, I should say that I too did not implement multiple shortest paths – LoL.

My solution to the problem is provided at http://github.com/gautamrege/short_circuit and its merely a representation of my style of programing. Creating a gem was for kicks but lib/short_circuit.rb has the core code. It was great fun AND learning while doing this and I realized that I will always be a student.

Cheers!

Capistrano + Nginx + Thin deployment on Linode

This was long lost post I had written about 8 months ago (converted from wiki to HTML – so pardon typos if any)

Terminologies

Capistrano is a ruby gem which helps in remote deployment. As against widely known convention, Capistrano can be used for any deployment, not just a rails app!

Nginx is a web-proxy server. This is simply a light weight HTTP web-server which received requests on HTTP and passes them to other applications. This is way more preferable than Application servers like Apache! Moreover, nginx is very easily configurable and can support multiple domain-names very easily. It has an in-build load-balancer which can send requests to apps based on its internal load-balancing mechanism.

Thin is the next-generation lean, mean rails server. Its much faster, lighter in memory than mongrel. Its has an internal event based mechanism for request processing and a very high concurrency performance ratio than other rails servers.

Linode is a VPS (a Virtual Private Server) that is hosted by www.linode.com. As the name suggests ;), its a “Linux Node”.  We are using Ubuntu 8.10 (Tip: To find Ubuntu release, issue command: lsb_release -a) NOTE: In the linode we had, it was a raw machine with no packages installed. Please read Linode RoR package installation for details.

Steps

Capistrano Configuration Follow the steps provided by Capistrano for basic instructions: Capistrano – From The Beginning Some modifications that you may need (as I needed for deployment):

• Edit Capfile and add the following to it. This ensures that remote capistrano deployment does not fork a remote shell using command “sh -c”. Some hosting servers do not allow remote shells.

default_run_options[:shell] = false

• In addition to changes mentioned in Capistrano tutorial, add the following to config/deploy.rb. This ensures that “sudo” is not used (default for Capistrano) and the user is “root”. Not usually a good practice.. but what the hell!

set :use_sudo, false            set :user, "root"

• Since capistrano uses default script/spin and script/process/reaper, we need to override the deploy:start, deploy:stop and deploy:restart to ensure that we can start/stop the thin service and the ferret_server. I know that in deply:restart, there is a copy-paste involved but I am trying to find out how to invoke a rake task from another rake task.

namespace :deploy do
    desc "Custom AceMoney deployment: stop."
    task :stop, :roles => :app do

        invoke_command "cd #{current_path};./script/ferret_server -e production stop"
        invoke_command "service thin stop"
    end

    desc "Custom AceMoney deployment: start."
    task :start, :roles => :app do

        invoke_command "cd #{current_path};./script/ferret_server -e production start"
        invoke_command "service thin start"
    end

    # Need to define this restart ALSO as 'cap deploy' uses it
    # (Gautam) I dont know how to call tasks within tasks.
    desc "Custom AceMoney deployment: restart."
    task :restart, :roles => :app do

        invoke_command "cd #{current_path};./script/ferret_server -e production stop"
        invoke_command "service thin stop"
        invoke_command "cd #{current_path};./script/ferret_server -e production start"
        invoke_command "service thin start"
    end
end

Thin Configuraion I looked up most of the default configuration of Thin and Nginx on Ubunto at Nginx + Thin. Some extra configuration or differences are mentioned below.

• The init script for starting thin and nginx during startup is configured during package installation. Leave them as they are.
• The following command generates the /etc/thin/acemoney.yml for 3 server starting from port 3000. Note that the -c option specifies the BASEDIR of the rails app. Do NOT change any settings in this file as far as possible.

thin config -C /etc/thin/acemoney.yml -c /home/josh/current --servers 3 -e production -p 3000

• Starting and stopping thin is as simple as

service thin start
service thin stop

• This will read the acemoney.yml file and spawn the 3 thin processes. I noticed that each thin server took about 31MB in memory to start with and with caching went upto ~70MB. On the contrary, a mongrel server (tested earlier) started with 31MB but exceeded 110MB later!

Nginx Configuration Installation on nginx is simple on Ubuntu 😉

apt-get install nginx

Configure the base /etc/nginx/nginx.conf. The default configuration are fine but I added / edited a few more for as recommended at Nginx Configuration

        worker_processes  4;

        gzip_comp_level 2;
        gzip_proxied any;
        gzip_types  text/plain text/html text/css application/x-javascript
                    text/xml application/xml application/xml+rss text/javascript;

According to this configuration above, nginx will spawn 4 worker threads and each worker thread can process 1024 connections (default setting). So, nginx can now process ~4000 concurrent HTTP requests !!! See performance article of thin at Thin Server

Configure the domainname, in our case acemoney.in. Ensure that acemoney.in “A record” entry points to this server! Check this by doing a nslookup or a ping for the server. In /etc/nginx/sites-available create a file by the domainname to be hosted. So I added /etc/nginx/sites-available/acemoney.in. In /etc/nginx/sites-enabled create a symbolic link to this file.

ln -s /etc/nginx/sites-available/acemoney.in /etc/nginx/sites-enabled/acemoney.in 

Now add the contents in /etc/nginx/sites-available/acemoney.in This is the key configuration to hook up nginx with thin.

upstream thin {
     server 127.0.0.1:3000;
     server 127.0.0.1:3001;
     server 127.0.0.1:3002;
}

server {
    listen 80;
    server_name acemoney.in;

    root /home/josh/current/public;

    location / {
      proxy_set_header X-Real-IP $remote_addr;
      proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
      proxy_set_header Host $http_host;
      proxy_redirect false;

      if (-f $request_filename/index.html) {
        rewrite (.*) $1/index.html break;
      }
      if (-f $request_filename.html) {
        rewrite (.*) $1.html break;
      }
      if (!-f $request_filename) {
        proxy_pass http://thin;
        break;
      }
    }

    error_page 500 502 503 504 /50x.html;
    location = /50x.html {
      root html;
    }
}

To analyze this configuration, here are some details:

The following lines tell nginx to listen on port 80 for HTTP requests to acemoney.in. The ‘root’ is the public directory for our rails app deployed at /home/josh/current!

server {
    listen 80;
    server_name acemoney.in;

    root /home/josh/current/public;

Now, nginx will try to process all HTTP requests and try to give the response.. for static HTML’s it will automatically give the data from the ‘root’. If it cannot find the HTML file, it will ‘proxy_pass’ it to thin. “thin” in the code below is an ‘upstream’ directive that tells nginx where to forward the current request it cannot directly serve.

if (!-f $request_filename) {
        proxy_pass http://thin;
        break;
      }

The upstream code is where load-balancing plays a role in nginx. The following code tells nginx which all processes are running on which different ports and it forwards requests to any of the servers based on its internal load balancing algorithm. The servers can be on different machines (i.e. different IP addresses) if needed. In AceMoney, we have started 3 thin servers on 3 different ports!

upstream thin {
     server 127.0.0.1:3000;
     server 127.0.0.1:3001;
     server 127.0.0.1:3002;
}

Performance Statistics Nothing is complete without them. Here is what I found out for 3 thin servers and 1 ferret_server.

top - 14:06:10 up 7 days, 22:58,  2 users,  load average: 0.00, 0.00, 0.00
Tasks:  84 total,   1 running,  83 sleeping,   0 stopped,   0 zombie
Cpu0  :  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu1  :  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu2  :  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu3  :  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:    553176k total,   530868k used,    22308k free,    16196k buffers
Swap:   524280k total,     2520k used,   521760k free,    87280k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
12424 mysql     18   0  127m  42m 5520 S    0  7.9   0:23.01 mysqld
18338 root      15   0 77572  70m 4392 S    0 13.1   0:06.79 thin
18348 root      15   0 71176  64m 4388 S    0 11.9   0:06.51 thin
18343 root      15   0 68964  62m 4384 S    0 11.5   0:07.20 thin
18375 root      18   0 70912  54m 2660 S    0 10.0   2:34.24 ruby
 8141 www-data  15   0  5176 1736  820 S    0  0.3   0:00.07 nginx
 8142 www-data  15   0  5176 1724  816 S    0  0.3   0:00.01 nginx
 8144 www-data  15   0  5152 1720  816 S    0  0.3   0:00.06 nginx
 8143 www-data  15   0  5156 1656  784 S    0  0.3   0:00.00 nginx

As can be seen:

• Each thin server takes around 70M
• The Mysql server takes 41M
• Ruby process (18375 above) is the ferret_serve which takes 54M
• 4 nginx threads take about 1.7K in memory.

 Overall: (3 thin server cluster + Mysql + ferret): 300MB