Joyent Server, Tomcat, Apache Web Server and mod_jk Connector

I just setup a new Joyent Server Smart Machine. I have to admit it was a challenge for me to get it up and running. I am a novice to Linux so this setup came with a steep learning curve. But I say it was worth it from how much I learned. Here is what I did – I hope that something here helps you.

Server Details:
I am running the SmartOSPlus (32bit) Image. This version is recommended for those who want a quick setup and minimal configuration. This package comes with most Linux based tools. Note that a few are different since these machines run Sun Solaris.

SmartOS vs Linux.
At Joyent you have the option of a Linux or Solaris (SmartOS) installation. Here are some factors to consider that might help you make a decision on which OS to choose.

Linux instances run on top of KVM Hypervisor on SmartOS in the Joyent Cloud.  By doing this, a lot of the built in performance/instant scalability features of SmartOS are disabled as they do not work beyond the hypervisor.  The specifics are listed out below:

SmartOS gives you several advantages over Linux.  They include:

1. ARC for caching of files in memory on each server
2. CPU bursting for flash incidents that require more compute – no cost for this.
3. Live resize w/o reboot of RAM (up and down) via API or my.joyentcloud.com dashboard
4. Full and deep integration of Cloud Analytics.  Linux does not have as deep of integration as SmartOS does.
5. SmartOS is still UNIX so your typical LAMP stack works the same way.
6. SmartOS security patching is done by Joyent as the maintainer of the OS.  Patching of Linux would be your responsibility
7. SmartOS machines can come preloaded with LAMP stack and has a large repository of additional packages precompiled.
8. Many GNU tools come loaded to give it the ease of use that Linux has.

My Objective:
Setup a web server. My goal is to migrate from Windows Server running Visual Basic .net web pages to a Linux based platform. I’m freeing my dependency on Microsoft products one day at a time. Yeah! We all will get there when ready, right?

I will discuss each of these:
– ssh Connection
– Apache Web Server
– Tomcat 6
– Tomcat Auto Startup/Shutdown
– mod_jk Connector
– MySQL Database Server
– MySQL Workbench
– Website Setup
– SFTP Connection

Useful Links to use during Setup

• Joyent Wiki
  http://wiki.joyent.com/display/jpc2/JoyentCloud+Home
  This is the wiki provided by Joyent. It provides updated information that pertains to the SmartOS machines.
• Joyent Administration Settings
  https://my.joyentcloud.com/
  The administration control panel for billing, server credentials, and SSH keys.
• Virtual/Webmin login
  MY.PUBLIC.IP.ADDRESS:10000
  The virtualmin control panel is located on port 10000 of your public IP address provided by Joyent or any domain name you have pointed to your Joyent Server.

Terms that might be useful:

Here are a couple of terms that you might find beneficial. These terms are specific to Joyent Servers and configurations (im not trying to insult you).

ip address:
Joyent provides two IP addresses. One is the private and the other is public. These can be found in your Administration Settings (my.joyentcloud.com). Login to account settings, click “Machines” Tab, click on your smart machine to view the details. You will see both public and private IP addresses. Your private IP address will start with 192.168.xxx.xxx. The other is your public IP address.

private ip address:
The private IP address is for internal use such as connecting to a dedicated database server, mail server, etc. If you only purchased one server; I dont believe you will be using the private IP address.

public ip address:
The public IP address is the IP address used for pointing all websites you manage. This is also used in SSH connections, MySQL workbench connections, and SFTP connections.

Server reboot.
There are two ways to reboot your server on a Joyent machine. You can reboot from the Joyent Administration control panel (my.joyentcloud.com). Login to your Joyent control panel, navigate to Machine Details and click the “reboot” button. You can update the status by clicking the “refresh status” button as well.

The second option is when logged in a bash shell prompt. This is the better option of the two. This is a graceful reboot. Finish editing any files and get to the point where you are ready for server to reboot.Type:

sudo shutdown -y -i6 -g0

Apache reboot.
To reboot Apache Web Server on Joyent Server. See Apache HTTP Server Control Interface Docs or use one of the commands below:
Reboot Apache:

sudo apachectl restart

Stop Apache:

sudo apachectl stop

Start Apache:

sudo apachectl start

Create SSH Key for Joyent Server

An SSH key consists of a pair of files. One is the private key, which you should never give to anyone. The other is the public key. You will need a public key to log into most machines you provision.

To generate an SSH key in Ubuntu 11, follow these steps.

1. Enter the following command in the Terminal window.

   gregg@workstation1:~$ ssh-keygen -t rsa

   This starts the key generation process. The -t option specifies what type of SSH key to generate. When you execute this command, the ssh-keygen tool prompts you to indicate where to store the key.

   Generating public/private rsa key pair. Enter file in which to save the key (/home/gregg/.ssh/id_rsa):

2. Press the ENTER key to accept the default.
3. The ssh-keygen utility prompts you for a passphrase. Enter a phrase you can remember. You can use a key without a passphrase, but this is not recommended.

   Enter passphrase (empty for no passphrase):

4. You need to enter the passphrase a second time to continue.

   Enter same passphrase again:

   After you confirm the passphrase, the command generates the key pair

   Your identification has been saved in /home/gregg/.ssh/id_rsa. Your public key has been saved in /home/gregg/.ssh/id_rsa.pub. The key fingerprint is: ae:89:72:0b:85:da:5a:f4:7c:1f:c2:43:fd:c6:44:38 gregg@workstation1 The key's randomart image is: +--[ RSA 2048]----+ |                 | |         .       | |        E .      | |   .   . o       | |  o . . S .      | | + + o . +       | |. + o = o +      | | o...o * o       | |.  oo.o .        | +-----------------+

Your private key is saved to a file named id_rsa in the .ssh directory.

Never share your private key with anyone!

Your public key is saved to a file named id_rsa.pub. This file contains the information you will add to your JoyentCloud account.

cat ~/.ssh/id_rsa.pub

Boot Tomcat at Server Startup/Reboot

I am running Sun Solaris 11 on a Joyent Machine. This may not apply to typical Tomcat setups.

My resources that helped achieve this. Both Peter Black and Peter Tribble are much more articulate in technical writing. So have a look:
1. Joyent Wiki, i forget which article.
2. Peter black: Solaris 10 manifest for starting up tomcat.
3. Peter Tribble: Peter’s Solaris Zone

Create a Manifest File which will be registered with Solaris. Mine looks like this:

<?xml version="1.0" ?>
<!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1">
<service_bundle type='manifest' name='Tomcat6'>
  <service name='application/servers/tomcat' type='service' version='1'>
    <create_default_instance enabled='false' />
       <single_instance/>
       <exec_method type='method' name='start' exec='/opt/local/java/tomcat6/bin/startup.sh' timeout_seconds='30' >
         <method_context>
           <method_credential user='tomcat' group='tomcat' privileges='basic,net_privaddr' />
           <method_environment>
             <envvar name="CATALINA_BASE" value="/opt/local/java/tomcat6" />
             <envvar name="CATALINA_HOME" value="/opt/local/java/tomcat6" />
             <envvar name="CATALINA_TMPDIR" value="/opt/local/java/tomcat6/temp" />
             <envvar name="JAVA_HOME" value="/opt/local/java/sun6" />
           </method_environment>
         </method_context>
       </exec_method>
       <exec_method type='method' name='stop' exec='/opt/local/java/tomcat6/bin/shutdown.sh' timeout_seconds='60' >
         <method_context>
           <method_credential user='tomcat' group='tomcat' />
           <method_environment>
             <envvar name="CATALINA_BASE" value="/opt/local/java/tomcat6" />
             <envvar name="CATALINA_HOME" value="/opt/local/java/tomcat6" />
             <envvar name="CATALINA_TMPDIR" value="/opt/local/java/tomcat6/temp" />
             <envvar name="JAVA_HOME" value="/opt/local/java/sun6" />
           </method_environment>
         </method_context>
       </exec_method>
       <stability value='Unstable' />
       <template>
         <common_name>
           <loctext xml:lang='C'> Tomcat </loctext>
         </common_name>
       </template>
   </service>
</service_bundle>

Download this example Manifest File

How to customize this manifest file:

Give the service manifest a meaningful name. I chose tomcat6. This can be found in the xml node:

<service_bundle type=’manifest’ name=’Tomcat6‘>

Set the path to file (i think). Here I’ve created the servers directory. And file name is tomcat.xml Note that the .xml extension isnt included in name attribute.
<service name=’application/servers/tomcat‘ type=’service’ version=’1′>

This file is stored in the vendor recommended location on the filesystem:

/var/svc/manifest/application/

And the complete path including the directory and file that I manually created:

/var/svc/manifest/application/servers/tomcat.xml

Next fill in all information regarding your configuration. Such as startup.sh, shutdown.sh scripts and environment variables. Read the Peter Black article for specifics.

Register the Manifest File.

Once the manifest file has been created and saved in the /var/svc/manifest/application/… directory; register it with the Service Management Facility (smf).

To begin, import the file. At terminal command prompt:

svccfg -v import /var/svc/manifest/application/servers/tomcat.xml

A successful import may look like the following:

svccfg: Taking “initial” snapshot for svc:/application/servers/tomcat:default svccfg: Taking “last-import” snapshot for svc:/application/servers/tomcat:default svccfg: Refreshed svc:/application/servers/tomcat:default svccfg: Successful import

Debugging a Manifest File

Originally I had some errors. This required some exploration to understand. Here is what i concluded and this is where Peter Tribble’s Solaris Zone is extremely useful. Specifically he helped me understand how to “correct a fault in a manifest file”.

I attempted to start tomcat svcadm enable tomcat in which nothing happened. So to begin isolating the error I looked at the log file for my service. Take a look:

First attempt to enable my Tomcat service. At this point the manifest file above has been imported. Using the svcadm tool, lets enable tomcat service, check the status, notice its in maintenance mode which means its not running. Then check the logs to determine why its not running and fix accordingly.

svcadm enable tomcat

svcs -lp tomcat

svcs -lp tomcat
fmri svc:/application/servers/tomcat:default
name Apache Tomcat 6.0.35
enabled true
state maintenance
next_state none
state_time February 5, 2012 08:59:59 PM UTC
logfile /var/svc/log/application-servers-tomcat:default.log
restarter svc:/system/svc/restarter:default contract_id

As you can see in the code above, the Tomcat service is in maintenance mode. So Tomcat isnt running. Time to view the log file to see what the issue is. In my case, the environment variables weren’t defined.

cat /var/svc/log/application-servers-tomcat:default.log

I see the log entry below:

Neither the JAVA_HOME nor the JRE_HOME environment variable is defined
At least one of these environment variable is needed to run this program
[ Feb  5 20:59:59 Method “start” exited with status 1. ]

My fix was to add the environment variables in the manifest file shown above. So hopefully you will not see this error. I was able to add the environment variables to the manifest file and startup the service without any other problems.

At this point, I believe that adding environment variables to any of the traditional files (.bashrc,  .bash_profile, /etc/.profile, etc) is not necessary.

Some commands that might be useful:

Start/Stop Tomcat service.
svcadm enable tomcat
svcadm disable tomcat

View Status of a specific service.
svcs -lp tomcat

View Status of all services.
svcs -a

When the tomcat service is enabled, Apache Tomcat Server will start up when the Server is rebooted.

To correct errors in a manifest file:

1. Edit the manifest file.
sudo vi /var/svc/manifest/application/servers/tomcat.xml

2. Disable the entry:
svcadm -v disable svc:/application/web/tomcat:default

3. Delete the entry:
svccfg -v delete svc:/application/web/tomcat:default

4. Import updated manifest file:
svccfg -v import /var/svc/manifest/application/servers/tomcat.xml

5. Start up entry.
svcadm enable tomcat

Setup Tomcat on Solaris 11

Here are my steps for installing Tomcat 6 on Solaris 11 server. Begin by securely logging into the server via SSH with administrative control. I am able to login using “admin” user. Admin user has ability to perform most operations without being root. I have chosen to install Tomcat 6 with java 6. I was fortunate to have Apache Web Server already installed when my machine was imaged.

ssh admin@MyIPAddress

Part 1: Find Java on the server

Check Java Version:
java -version

This command simply checks the version of Java. For example, my output looks like:

java version "1.6.0_26" Java(TM) SE Runtime Environment (build 1.6.0_26-b03) Java HotSpot(TM) Client VM (build 20.1-b02, mixed mode, sharing)

Find Java Location:
find / -name java 2>/dev/null

This command searches the file system for files with the name java in it. All other output is sent to null. So only the results you are interested in are printed to the terminal screen. I find where java is located on my server:

/opt/local/java/sun6/bin/java

therefore, JAVA_HOME=/opt/local/java/sun6

Confirm Java Location:
For fun, confirm this is correct by running the java -version command using the absolute path. Since the java binary should be located in the bin directory we should be able to check the version and get the output. If successful, we have what we are looking for. This step probably isn’t necessary. Its nice to have the confirmation.

/opt/local/java/sun6/bin/java -version

The results are:

java version "1.6.0_26" Java(TM) SE Runtime Environment (build 1.6.0_26-b03) Java HotSpot(TM) Client VM (build 20.1-b02, mixed mode, sharing)

Write down the path to our java directory. We will add this to the Manifest entry later as an environment variable.

Download Tomcat 6

Now download Tomcat 6.
Using firefox browser: go to Tomcat website and pick a download location. Here is one for example: http://apache.cs.utah.edu/tomcat/tomcat-6/v6.0.35/bin/apache-tomcat-6.0.35.tar.gz

Copy the link. Return to SSH terminal and wget to download the compressed zipped tarball file. Type wget command. Right click and paste the url.

wget http://apache.cs.utah.edu/tomcat/tomcat-6/v6.0.35/bin/apache-tomcat-6.0.35.tar.gz

The apache-tomcat-6.0.35.tar.gz tarball should be in the user home directory.

Checksum – validate the download
Verify your download has not been tampered with using the checksum. I will use the md5 encryption key. The checksum keys are found beside your binary download link on the tomcat website. For example, when running the md5sum command; we should have a matching checksum value.

md5sum apache-tomcat-6.0.35.tar.gz

The output from the command above is:

171d255cd60894b29a41684ce0ff93a8  apache-tomcat-6.0.35.tar.gz

When comparing this against the checksum value saved in the md5 link on the tomcat website we see these match. So it is safe to install software on our server.

Extract and move tomcat directory to permanent location
Its time to move tomcat directory to a meaningful location. Note I am new to Linux and Solaris so I may take additional (and unnecessary) installation steps. For example, I extract and move the tomcat directory as root user. Then I change the ownership of the directory. This may be an extra step. That is ok with me at this time. Please do as you see fit.

Different sources on the Internet say to put tomcat directory here or there. I chose to put in with java directory. I figure its just a location. You chose what makes sense to you please.

I will store in the directory: /opt/local/java

sudo mv apache-tomcat-6.0.35.tar.gz /opt/local/java/apache-tomcat-6.0.35.tar.gz

Verify it moved:

cd /opt/local/java; ls;

Extract it.

sudo tar xzvf apache-tomcat-6.0.35.tar.gz

The switches for the tar command are:
x – Extract the contents of the archive.
z – Decompress the files using gzip utility.
v – Verbose, display the output of stdout so you can see what is happening.
f – Filename, specifices that we are providing a file name.

Rename directory:
Once extracted, rename the directory to something meaningful. I choose tomcat6.

sudo mv apache-tomcat-6.0.35 tomcat6; ls

The tomcat directory should now be /opt/local/java/tomcat6.

Create Tomcat User and Group

Now its time to create the tomcat user who belongs to the tomcat group.

Create a tomcat group:

groupadd tomcat

Create a tomcat user:

useradd -s /bin/bash -g tomcat -m tomcat

The switches for the useradd command are:
s – Specifices the absolute pathname to the shell used for the user account.
g – Specifies a new primary group for the user account.
m -Specifies that a home directory should be created for the user account.

Assign a password to tomcat user:

passwd tomcat

To edit the tomcat user settings using /etc/passwd file:

sudo vi /etc/passwd

Change ownership of Tomcat directory:

Go to the directory where tomcat6 is stored. I am logged in as admin so i will run sudo commands to change ownership (chown and chgrp).  Note the -R switch is used to change permissions recursively throughout the tomcat6 directory tree.

cd /opt/local/java
sudo chown -R tomcat6 tomcat
sudo chgrp -R tomcat6 tomcat

Tomcat is ready.

Verify Tomcat Server is running:

Turn on Tomcat 6:
sudo /opt/local/java/tomcat6/bin/startup.sh

If you’ve setup environment variable for CATALINA_HOME:

sudo $CATALINA_HOME/bin/startup.sh

Immediate output might resemble:

sudo sh startup.sh
Using CATALINA_BASE:   /opt/local/java/tomcat6
Using CATALINA_HOME:   /opt/local/java/tomcat6
Using CATALINA_TMPDIR: /opt/local/java/tomcat6/temp
Using JRE_HOME:        /opt/local
Using CLASSPATH:       /opt/local/java/tomcat6/bin/bootstrap.jar

View in browser: http://localhost:8080.

To shutdown Tomcat Server.

sudo sh shutdown.sh

Now its time to create a service to start/stop Tomcat Server when the Joyent Server is rebooted.