The secret project I’m working on is taking shape. And because of the looming deadline (Liftconference coming up fast), I had to give some thoughts on the hardware, I’m deploying this application on…

I have been a Textdrive (affiliate link) customer since ever they existed. I have seen them grow from a small hosting company with only a couple of servers to being bought by Joyent (of which there will be talked more soon). Joyent/Textdrive has spent the last couple of months moving their complete hosting infrastructure from Dell Servers running FreeBSD to Sun Servers running OpenSolaris. They have told numerous tales about how much their hosting experience has improved.

They have been offering something they call “Containers”, pieces of a SunFire 4100 server with 4 Dual Opterons, 16 GB of RAM. So for this project, I decided to use their 1/16 container. This basically guarantees me that I always have 1/16 of the resources of this machine available. If no one else is using CPU, I can have it all… Wonderful :-)

Setting up the container with a Apache / Mongrel / Rails / PostgreSQL stack took about 3 hours, most of the time researching the differences between Linux and Solaris command line parameters and getting used to the different layout. There are lot of helpful posts on the Textdrive forum, that got me going.

The the big disappointment. My app took multiple seconds to load – not quite what I was expecting from this machine. Network latency was (and continues to be) one problem: I see roundtrip times of about 200ms to this machine. But the real culprit was the Apache / Mongel combo on serving static files. I had heard good things about nginx, a russian made server/loadbalancer and decided to use it. I got it up and running quite easily:

./configure --with-http_ssl_module --sbin-path=/opt/csw/bin --prefix=/opt/csw/nginx
make 
make install

which drops it into the /opt/csw tree. I then used Ezra’s config file for nginx. Finally, I modified an existing SMF file to get nginx to (re)start on reboots and being monitored by the Solaris Management System SMF. Here’s my nginx.smf file:

<?xml version='1.0'?>
<!DOCTYPE service_bundle SYSTEM '/usr/share/lib/xml/dtd/service_bundle.dtd.1'>
<service_bundle type='manifest' name='export'>
  <service name='network/nginx' type='service' version='0'>
    <create_default_instance enabled='true'/>
    <single_instance/>
    <dependency name='fs' grouping='require_all' restart_on='none' type='service'>
      <service_fmri value='svc:/system/filesystem/local'/>
    </dependency>
    <dependency name='net' grouping='require_all' restart_on='none' type='service'>
      <service_fmri value='svc:/network/nfs/client'/>
      <service_fmri value='svc:/network/loopback'/>
    </dependency>
    <dependent name='nginx' restart_on='none' grouping='optional_all'>
      <service_fmri value='svc:/milestone/multi-user'/>
    </dependent>
    <exec_method name='start' type='method' exec='/opt/csw/bin/nginx -c /opt/csw/nginx/conf/nginx.conf' timeout_seconds='60'>
      <method_context working_directory='/opt/csw/var/log'>
        <method_credential user='root' group='root'/>
        <method_environment>
          <envvar name='PATH' value='/usr/bin:/bin:/opt/csw/bin'/>
        </method_environment>
      </method_context>
    </exec_method>
    <exec_method name='stop' type='method' exec=':kill' timeout_seconds='60'>
      <method_context/>
    </exec_method>
  </service>
</service_bundle>

it can then be imported and enabled with the following commands:

# svccfg import nginx.smf
# svcadm refresh network/nginx
# svcadm clear network/nginx
# svcadm enable network/nginx

And boy did that make a difference. The load time dropped down a lot, and serving of static files is much, much faster (and with caching enabled) almost instantaneous.

Now we’ll have to see, how the container holds up, when the masses are unleashed upon the application I’m writing…

Technorati Tags: joyent, nginx, textdrive, rails, smf, solaris