Insert Something Creative

Setting up KVM on Ubuntu 10.04 (Lucid Lynx)

2011-11-28T15:42:00.002+08:00

After doing a KVM install on Debian Squeeze and trying to get a VM up and running, the hassle convinced me to go back to Ubuntu and their vm-builder package, which allow ones to create VMs relatively easy once the setup is complete. There is a vm-builder port for Debian, though that only works for building older versions of Ubuntu and I want to run the latest, Ubuntu 11.10 (Oneiric Ocelot).

Starting with a bare-metal Ubuntu 10.04 LTS (Lucid Lynx) 64-bit, below is the list of commands and instructions to install and set up the KVM. Details on these instructions can be read in the Ubuntu community documentation, KVM Installation and KVM Networking:

sudo apt-get install qemu-kvm libvirt-bin ubuntu-vm-builder bridge-utils

virsh -c qemu:///system list (To verify installation, should have no errors)

sudo apt-get install libcap2-bin

sudo setcap cap_net_admin=ei /usr/bin/qemu-system-x86_64

sudo vi /etc/network/interfaces
- Original file: # The loopback network interface auto lo iface lo inet loopback # The primary network interface auto eth0 iface eth0 inet static address 10.10.3.140 broadcast 10.10.3.143 netmask 255.255.255.248 gateway 10.10.3.137 # default route to access subnet up route add -net 10.10.3.136 netmask 255.255.255.248 gw 10.10.3.137 eth0
- Modified file: # The loopback network interface auto lo iface lo inet loopback # device: eth0 auto eth0 iface eth0 inet manual # The primary network interface auto br0 iface br0 inet static address 10.10.3.140 broadcast 10.10.3.143 netmask 255.255.255.248 gateway 10.10.3.137 bridge_ports eth0 bridge_stp off bridge_fd 9 bridge_hello 2 bridge_maxage 12 # default route to access subnet up route add -net 10.10.3.136 netmask 255.255.255.248 gw 10.10.3.137 eth0 up route add -net 10.10.3.136 netmask 255.255.255.248 gw 10.10.3.137 br0

sudo /etc/init.d/networking restart

Running ifconfig lists the following interfaces br0, eth0, lo, virbr0

This completes the KVM installation and creation of a bridge for the VMs. Up next is replacement of the vm-builder. The one in the Ubuntu packages is faulty and also will not allow you to install Ubuntu 11.10 (Oneiric Ocelot). So I updated to the latest, downloading the source, building and installing it. The steps below can be found in this accepted answer:

sudo apt-get install bzr

sudo apt-get install epydoc (big install here, ~400mb)

bzr branch lp:ubuntu/vm-builder ubzr-vm-builder

cd ubzr-vm-builder

fakeroot debian/rules binary

sudo dpkg -i ../*vm-builder*.deb

With that, everything is installed and vm-builder is ready to run. The easiest way is to use a script so that vm creation can be set once and repeated as desired. The only changes required being hostname, ip and maybe memory. Obtain the Ubuntu 11.10 64-bit server iso and put it in the same place as the script. The directory I used is ~/vm/basekvm:

cd ~/vm/basekvm

sudo vi create_vm.sh
- File: #!/bin/bash # Configure this before running the command HOSTNAME=myhostname MEMORY=2048 IP=192.168.122.10 # -- End of configuration vmbuilder kvm ubuntu \ --destdir=/var/lib/libvirt/images/$HOSTNAME \ --ip=$IP \ --hostname=$HOSTNAME \ --mem=$MEMORY \ --suite=oneiric \ --flavour=virtual \ --arch=amd64 \ --iso=/root/vm/basekvm/ubuntu-11.10-server-amd64.iso \ --mirror=http://de.archive.ubuntu.com/ubuntu \ --libvirt=qemu:///system \ --domain=localdomain \ --part=/root/vm/basekvm/vmbuilder.partition \ --bridge=virbr0 \ --gw=192.168.122.1 \ --mask=255.255.255.0 \ --user=myusername \ --name=myname \ --pass=mypassword \ --tmpfs=- \ --addpkg=vim-nox \ --addpkg=acpid \ --addpkg=unattended-upgrades \ --addpkg=openssh-server \ --firstboot=/root/vm/basekvm/fboot.sh \ -o

sudo chmod 700 create_vm.sh

sudo vi fboot.sh (Optional)
- File: # This script will run the first time the virtual machine boots # It is ran as root. # Expire the user account passwd -e administrator # Install openssh-server apt-get update apt-get install -qqy --force-yes openssh-server

sudo chmod 777 fboot.sh

sudo vi vmbuilder.partition
- File: root 8000 swap 4000 --- /var 8000

cd ~/vm

ln -s /var/lib/libvirt/images/ images

The create_vm.sh is basically a template script. You can modify it to accept console input so that you don't need to go and edit the file values, that is left for another time. The symbolic link shows the directory where the VM disk images are located once created. Below is how you would use it to create a VM:

sudo cp basekvm/create_vm.sh create_vm_myvmname.sh

sudo vi create_vm_myvmname.sh. Edit the HOSTNAME, IP and MEMORY as desired

sudo ./create_vm_myvmname.sh

virsh start myvmname

And that's it! A VM has been successfully created and started up. Give it a few minutes and then you can log in through ssh using the information in the script. If the ssh is slow to connect, try this.

KVM host with gateway guest using port-forwarding

2011-11-25T22:35:00.004+08:00

Using the 3 rules listed here and below, a KVM host can forward all http and ssh traffic to a specified gateway guest VM:


iptables -t nat -I PREROUTING -p tcp --dport 80 -j DNAT --to-destination 10.0.0.1:80
iptables -t nat -I PREROUTING -p tcp --dport 22 -j DNAT --to-destination 10.0.0.2:22
iptables -I FORWARD -m state -d 10.0.0.0/24 --state NEW,RELATED,ESTABLISHED -j ACCEPT

To make it permanent, requires one to go through this page and use the following commands:


sudo sh -c "iptables-save -c > /etc/iptables.rules" (after applying the 3 commands above)
sudo vi /etc/network/if-pre-up.d/iptablesload

The /etc/network/if-pre-up.d/iptablesload file will have the following text:


#!/bin/sh
iptables-restore < /etc/iptables.rules
exit 0

The KVM host will now have VM creation as its sole focus. Ensure the host's ssh port have been changed to make it accessible from outside, otherwise it can be accessed from the gateway guest.

All redirection and VM access are transferred to the the gateway guest. The guest will need to install nginx so it can act as a http proxy for other VMs. All ssh access use the gateway guest as a stepping stone to the other VMs.

Following the 3 rules, it is found the traffic essentially loops back to the gateway guest. This makes it incapable of reaching the other VMs. Applying 1 more rule after the 3 above solves this. The rule accepts all packets from the VM ip range and does not do any forwarding:


iptables -t nat -I PREROUTING -p tcp --source 10.0.0.0/24 -j ACCEPT

A Certain Minimal QR Scanner iPhone app

2011-02-07T09:54:00.003+08:00

A QR Code is similar to a barcode, except it contains more information and looks like a pixellated square Rorschach test.

There are a number of free QR readers in the appstore like NeoReader and RedLaser. I especially like the scanning GUI for RedLaser, which seems more streamlined than any of the apps I have tried.

A number of these apps use the open-source ZXing ("Zebra Crossing") scanner library. The library is in Java but has a number of ports which include iPhone.

To start things off, you need to download or checkout the source. Create a new "View-Based Application" project in Xcode. Inside the iphone folder of the source, follow the README on how to include the ZXingWidget project into yours.

Pay attention to the instructions, especially the direct dependency, header search path and the fact the file with the ZXing has to be a .mm instead of .m. If it does not build, you're probably missing something. You can look at the sample projects to see how they include the widget.

One last thing before moving on to the code, put the beep-beep.aiff file from the ScanTest project into your project. This is to get audio confirmation of a scan.

Inside the sole viewController of your project:


#import "ZXingWidgetController.h"
#import "QRCodeReader.h"
#import "ResultParser.h"
#import "URLResultParser.h"
#import "ResultAction.h"

- (void)viewDidLoad {
    [super viewDidLoad];
    [ResultParser registerResultParserClass:[URLResultParser class]]; 
}

- (void)viewDidAppear:(BOOL)animated {
    [super viewDidAppear:animated];
    ZXingWidgetController *widController =
  [[ZXingWidgetController alloc] initWithDelegate:self showCancel:NO OneDMode:NO];
    QRCodeReader *qrcodeReader = [[QRCodeReader alloc] init];
    NSSet *readers = [[NSSet alloc] initWithObjects:qrcodeReader,nil];
    [qrcodeReader release];
    widController.readers = readers;
    [readers release];
    NSBundle *mainBundle = [NSBundle mainBundle];
    widController.soundToPlay =
 [NSURL fileURLWithPath:[mainBundle pathForResource:@"beep-beep" ofType:@"aiff"] isDirectory:NO];
    [self presentModalViewController:widController animated:YES];
    [widController release];
}

#pragma mark -
#pragma mark ZXingDelegateMethods
- (void)zxingController:(ZXingWidgetController*)controller didScanResult:(NSString *)resultString {
    [self dismissModalViewControllerAnimated:YES];
    ParsedResult *parsedResult = [[ResultParser parsedResultForString:resultString] retain];
    NSArray *actions = [[parsedResult actions] retain];
 
    if ([actions count] == 1) {
        ResultAction *theAction = [actions objectAtIndex:0];
        [theAction performActionWithController:self shouldConfirm:YES]; 
    } else {
        UIAlertView *alertView = [[UIAlertView alloc] initWithTitle:@"Text Found:"
            message:resultString
            delegate:nil
            cancelButtonTitle:@"OK"
            otherButtonTitles:nil];
        [alertView show];
        [alertView release];
    }
}

The code in viewWillAppear is lifted off from the sample projects. This sets up the scanning video camera with the appropriate reader. If a scan is successful, the delegate method didScanResult will execute. The result is parsed to see if it is a URL. You set which parser to use in viewDidLoad. A parsed result can have default actions associated with it, the URLResultParser opens up the url in Safari as default. Otherwise the result is treated as text and displayed.

This app can now scan QR codes and open up URLs in Safari. There are a number of other things you can add to this, eg you can switch out the ResultParser with a UniversalResultParser that includes all the parser classes. You should take a look in the Classes folder of the ZXingWidget project to see what is available.

Setting up mail for your Ubuntu server

2010-04-07T12:12:00.003+08:00

Need your bare-bones Ubuntu server to send out email notification and the like? Well, first you need to check what your hostname is and modify accordingly:

hostname -f

So my server's hostname is microdude, you might also get it as microdude.localdomain. You'll want to convert it into an FQDN (Fully Qualified Domain Name). That means I need to change my microdude to microdude.favoritemedium.com.

sudo vi /etc/hostname
sudo vi /etc/hosts

The output of hosts:

127.0.0.1 localhost
127.0.1.1 microdude.localdomain microdude

Would now be:

127.0.0.1 localhost
127.0.1.1 microdude.favoritemedium.com microdude

Reboot and check that the hostname has been changed to microdude.favoritemedium.com:

sudo reboot
hostname -f

After that, you're supposed to set up RDNS (Reverse Domain Name System), this helps avoid your mails falling prey to the all-powerful spam filter. You can check on this RDNS with dig which can be installed from this package:

sudo apt-get install dnsutils

This is optional, though if your mails start going into the spam folder, you'd best revisit this in the links provided.

Now to install the actual mail agent onto the server with a one-liner operation:

sudo tasksel install mail-server

The process stops at two points for user input. The first is to pick the postfix configuration for the server, select the default Internet site. The second is to set the main domain name, this should be already be filled with microdude.favoritemedium.com and all you need to do is accept.

Once finished, the postfix daemon should be running now. You need to know about two files, the main configuration file and the mail log. You don't need to mess around with the config file, since that should be set up correctly. The mail log is useful to check on whether mails have been sent or other information. Their locations:

/etc/postfix/main.cf
/var/log/mail.log

Now you'll want to test whether mails can be sent out. An email can be typed out in the terminal using the mail command:

mail testing@mailaddress.com
Subject: Do not panic, this is a test
Panic panic panic panic panic
.
Cc:

Check the email account you sent the mail to and you should see that it is present.

Congratulations, you can now spam to your heart's content =^_^=.

From:

Moving Drupal from one host to another

2010-04-06T18:32:00.004+08:00

You've got a Drupal site running and now want to move it to another server which is bare-bones. This server is luckily running Ubuntu, which makes installing the rest of the LAMP (Linux, Apache, MySQL and PHP) stack a one-line operation:

sudo tasksel install lamp-server

The install process will stop at one point to ask for the password of the root user for MySQL. Once the install completes, use the MySQL command-line client to add in the same database (an empty one) and user that Drupal uses.

Use mysqldump to dump out the Drupal database data into an sql file. Load it in the new database, populating it. You'll want to tar up the whole directory where the Drupal files reside (not forgetting the .htaccess) and expand it out in the same location within the new machine. You may (not) need to edit sites/default/settings.php or .htaccess for any host information changes. settings.php is also where you edit the database login information, if you decide to use different ones.

With Apache running, you should be able to hit the new Drupal site. It looks to be running but there are a few gotchas. If you are (most likely) running Clean URLs, the links don't work anymore. You'll need to disable it to get navigation working at http://hostname/?q=admin/settings/clean-urls, maybe going through http://hostname/?q=user to log in first. To set up Clean URLs again, you need to run this line:

sudo a2enmod rewrite

And edit /etc/apache2/sites-available/default, changing the AllowOverride None to AllowOverride All inside the directory with the path /var/www, or where the Drupal files live. Restart the server and you find that you can enable Clean URLs now.

The other gotcha is that the status report is complaining it can't find the PHP GD library. Apparently PHP library is installed but not configured so you run these lines:

sudo apt-get install php5 (optional?)
sudo apt-get install php5-gd

Lastly, cron needs to be set up to update the Drupal site. I use curl so the lines are:

sudo apt-get install curl
crontab -e
0 * * * * curl --silent --compressed http://localhost/cron.php

The last line is done inside the vi editor so after that, save and quit.

Congratulations! Your Drupal site is now running fine on the new machine.

Google Analytics not showing Firefox and Internet Explorer hits, only Safari and Chrome

2010-03-30T09:31:00.005+08:00

If you find your Google Analytics is not showing any results for Firefox and Internet Explorer and everything is correct with no problems on the Javascript side, you might want to take a closer look at the GA code. It might be that all the code is in one script block like so:


<script type="text/javascript"> 
var gaJsHost = (("https:" == document.location.protocol) ? "https://ssl." : "http://www.");
document.write(unescape('%3Cscript src="'+gaJsHost+'google-analytics.com/ga.js" type="text/javascript"%3E%3C/script%3E'));

try {
    var pageTracker = _gat._getTracker('UA-XXXXXXX-X');
    pageTracker._trackPageview();
}
catch (err) {}
</script>

The code that GA gives you to use is no mistake, you need to separate them out into two blocks:


<script type="text/javascript"> 
var gaJsHost = (("https:" == document.location.protocol) ? "https://ssl." : "http://www.");
document.write(unescape('%3Cscript src="'+gaJsHost+'google-analytics.com/ga.js" type="text/javascript"%3E%3C/script%3E'));
</script>
<script type="text/javascript">
try {
    var pageTracker = _gat._getTracker('UA-XXXXXXX-X');
    pageTracker._trackPageview();
}
catch (err) {}
</script>

The first code block is responsible for downloading the ga.js. That file needs to be loaded up before the second code block can execute properly. If they are in the same block, the try code will fail and no exception or alert is given.

Below are three separate posts covering the same question:

GA not tracking Mozilla or Firefox. Only shows Chrome and safari Clicks

Reports not showing some browsers

Analytics not showing visitors using Internet Explorer

The last comment here lets you know what to look for in troubleshooting Google Analytics. The GA cookies are named __utma, __utmb, __utmc, __utmz, and __utmv and if you do not have any of them, GA is not working.

Groovy Tutorial for MongoDB

2009-08-04T16:28:00.002+08:00

Java Tutorial (http://www.mongodb.org/display/DOCS/Java+Tutorial)

Since Groovy is based on Java, you can make use of the Java driver for MongoDB as well as go through the tutorial shown above. The code can be used as-is or modified to look more groovy.

Create a groovy file (mongo.groovy) and place the Java driver in the same directory. Edit it so the code looks like:

this.class.classLoader.rootLoader.addURL(new File("mongo-0.6.jar").toURL())

import com.mongodb.*

DBAddress address = new DBAddress("localhost", "mydb")
Mongo db = new Mongo(address)
db.authenticate('testUser', 'testPassword')

DBCollection coll = db.getCollection("testCollection")

BasicDBObject doc = new BasicDBObject()
doc.put("name", "MongoDB")
doc.put("type", "database")
doc.put("count", 1)

BasicDBObject info = new BasicDBObject()
info.put("x", 203)
info.put("y", 102)
doc.put("info", info)

coll.insert(doc)

for(i in 1..100) {
coll.insert(new BasicDBObject().append("i", i))
}

println coll.getCount()

DBCursor cursor = coll.find()
while(cursor.hasNext()) {
println cursor.next()
}

You can see it's just a rehash of part of the Java tutorial, the output will be a count of the number of inserted objects followed by the objects themselves.

The first line of code loads the jar file, this is useful if you're not looking to put the jar in a classpath or one of groovy's configured library paths like '~/.groovy/lib'.

Running the script is accomplished by 'groovy mongo.groovy' in the terminal. You can also type in 'groovyConsole' to load the console then opening the script and running inside it.

Installing MongoDB on Ubuntu 9.04 Jaunty Jackalope

2009-08-04T11:22:00.000+08:00

For the installation, I decided to build MongoDB (http://www.mongodb.org/) from source. The documentation regarding this is thorough and getting it running was a cinch. A Javascript engine called Spider Monkey is used and you need to build it with UTF8 support.

The documents referred to:
http://www.mongodb.org/display/DOCS/Building
http://www.mongodb.org/display/DOCS/Building+Spider+Monkey
http://www.mongodb.org/display/DOCS/Building+for+Linux

The all-in-one command line version:
Dependencies:
sudo apt-get install curl tcsh git-core scons g++
sudo apt-get install libpcre++-dev libboost-dev libmozjs-dev

Spider Monkey installation:
curl -O ftp://ftp.mozilla.org/pub/mozilla.org/js/js-1.7.0.tar.gz
tar zxvf js-1.7.0.tar.gz
cd js/src
export CFLAGS="-DJS_C_STRINGS_ARE_UTF8"
make -f Makefile.ref
sudo JS_DIST=/usr make -f Makefile.ref export

MongoDB installation:
git clone git://github.com/mongodb/mongo.git
scons all
sudo scons --prefix=/opt/mongo install

Finishing up:
add /opt/mongo/bin to your path
create a /data/db directory structure and chown it to your user or add read-write permission.
http://www.howtogeek.com/howto/ubuntu/how-to-add-a-program-to-the-ubuntu-startup-list-after-login/
System -> Preferences -> Startup Applications -> Startup Programs
click add, click browse and select /opt/mongo/bin/mongod
add run to the end of the line so that it reads '/opt/mongo/bin/mongod run'
enter name and comment then save

And you're done. mongodb will start up with no authentication as default at localhost:27017, the web information interface is at localhost:28017. You can use mongo to enter the interactive shell, similar to mysql's shell client.

Some links to get you started:
http://www.mongodb.org/display/DOCS/Overview+-+The+MongoDB+Interactive+Shell
http://www.mongodb.org/display/DOCS/File+Based+Configuration
http://www.mongodb.org/display/DOCS/Command+Line+Parameters
http://www.mongodb.org/display/DOCS/Security+and+Authentication
http://www.mongodb.org/display/DOCS/Manual

Delegation for iphone-google-maps-component

2009-03-17T17:56:00.003+08:00

The iphone-google-maps-component is a way for you to use Google Maps on the iPhone through the use of an extended UIWebView and an html file on a server somewhere that does the map loading.

The present code overrides the default delegate UIWebViewDelegate with its own, MapWebViewDelegate. You cannot use the setCenterWithLatLng to establish the starting location until the view has finished loading. With the override, the webViewDidFinishLoad delegate method is unavailable.

Going by this post, the following code changes will allow the use of both delegates.

MapWebView.h
1) Change protocol definition.
@protocol MapWebViewDelegate < UIWebViewDelegate >

MapWebView.m
2) Remove the synthesize line.
//@synthesize delegate;

3) Replace with own methods, to hook into super.delegate.

- (id ) delegate {
       return (id )super.delegate;
}

- (void) setDelegate :(id ) delegate {
       super.delegate = delegate;
}

webViewDidFinishLoad should be accessible, as well as the other methods in the UIWebViewDelegate.

Another way of setting the starting location is through the loadMap method in MapWebView.h. The URL string can accept two more variables, latitude and longitude.

Change this:

NSString *urlStr = [NSString stringWithFormat:
        @"http://www.wenear.com/iphone-test?width=%d&height=%d&zoom=%d",
        width, height, DEFAULT_ZOOM_LEVEL];

To this:

NSString *urlStr = [NSString stringWithFormat:
   @"http://www.wenear.com/iphone-test?width=%d&height=%d&latitude=%f&longitude=%f&zoom=%d",
   width, height, location.coordinate.latitude, location.coordinate.longitude, DEFAULT_ZOOM_LEVEL];

Here, location is a CLLocation, part of the CoreLocation framework. The loadMap method is publicly defined in the interface and it accepts location as its variable. The call in didMoveToSuperview should be properly handled.

The latitude and longitude are doubles and can be hardcoded instead.

XML and the Copyright symbol, ©

2009-02-09T17:16:00.002+08:00

For anyone parsing an XMl file and wondering what in the world is an org.xml.sax.SAXParseException: Invalid byte 1 of 1-byte UTF-8 sequence error message, check to see whether the XML file has any special characters like the copyright symbol, ©. The default encoding of UTF-8 will choke on these.

Solved my woes by changing the encoding to <?xml version="1.0" encoding="iso-8859-1"?> to get the character to display correctly. You can also filter it out or replace the symbol with its character reference.

Google Friend Connect & Facebook Connect

2009-01-05T15:06:00.003+08:00

Google Friend Connect

A Google, Yahoo, AIM or OpenID account can be used for authentication.

You start it off with logging into your Google account and going to http://www.google.com/friendconnect/.

From there, click on 'Set up a new site' and enter your site name and url.
You'll be given two files, canvas.html and rpc_relay.html, to copy onto your web site.
The last step is testing for those two files.

After that, it's a matter of choosing the gadgets you wish to add, customizing it and then copying the code generated into one of your site pages. The code is normal html and javascript.

From the current gadgets available, you can add on to your site to make it more 'sociable', like knowing the site members, adding in a chat box and reviewing/rating stuff.

You can create your own custom gadgets from these sites:
http://code.google.com/apis/gadgets/
http://code.google.com/apis/opensocial/

Facebook Connect

Only a Facebook account can be used for authentication.

Login into your Facebook account and follow the instructions located here:
http://wiki.developers.facebook.com/index.php/Trying_Out_Facebook_Connect

The main page for developers for Facebook Connect is http://developers.facebook.com/connect.php.

Facebook uses it's own Facebook Markup Language (FBML), in contrast to normal HTML, to facilitate creation of all Facebook applications. Facebook Connect is therefore embedded into user pages using FBML.

Facebook Connect is more of an expansion of Facebook onto other sites, depending on the application, it allows for data on a user site to be displayed back in Facebook.

HOW TO: Add Facebook Connect to Your Blog in 8 Minutes

Clickpass plugin for Grails, Part 2

2008-12-02T13:21:00.000+08:00

Go to Clickpass and create an account. Once in the account, go to the "My Details" tab and check the "Show software developer tools" under "Additional Preferences". There should now be a "developer" tab present, select it and push the "Create new site" button.

1. Register site

Type in the site name.
URL: http://localhost:8080/makedo. This is for running a grails app on your own machine named makedo.
Description and logo are optional.

2. Log users in

OpenID trust root: http://localhost:8080.
begin_openid_login: http://localhost:8080/makedo/openid/login?success_controller= clickpass&success_action=welcome&error_controller=&error_action=. This link is where Clickpass will access, providing the OpenID details, after one has passed the bar that should be in the app.
Leave the OpenID parameter label alone.
Submission method: POST.
SSL is optional.
Once you have saved the changes, the page will change and display a choice of clickpass bars and the html code for it down below. Copy the html code into _clickpass.gsp.

3. Merge accounts

begin_add_openid_to_user URL: http://localhost:8080/makedo/clickpass/merge. This is after the authentication and the check whether the OpenID belongs to any user on the database, bringing him to the registration form. If the user selects to login a pre-existing account, the username and password he provides will be sent to this link. If successful the OpenID will be associated with that account.
user_id label: Username. This is for the login page Clickpass will provide before accessing the URL above. Merely cosmetic.
Adding the connect bar is optional.

4. Register new signups

process_openid_registration_url: http://localhost:8080/makedo/clickpass/register. This is after the authentication and the check whether the OpenID belongs to any user on the database, bringing him to the registration form. The user provides information to Clickpass which will be passed along to the link. If successful a new user account with the associated OpenID will be created and saved in your app.
new_openid_completion_url: http://localhost:8080/makedo/clickpass/welcome. This is for when authentication is successful and all other things have been taken cared of, e.g. registration or merging.
The others are optional except for the fields. Changes in the fields may require changes in the register method of the Clickpass controller.
The URL for the registration form should be left for last as it changes due to the fields. Once you're satified and have saved the changes, you can copy and paste the string into urlString in the welcome method of the Clickpass controller.

Aftermath
Once the settings are done, you can use Clickpass.

Clickpass plugin for Grails, Part 1

2008-12-02T09:15:00.003+08:00

Clickpass works by handling the OpenID authentication and sending it back to the site. This means that any Grails app using this plugin must also install the OpenID plugin. Also the site will need to be registered with Clickpass and some settings need to be set online. When the authenticated details are sent back, the user is logged in and sent to the welcome page, sent to a form for details to complete registration or asked to login to attach the OpenID to an existing user account.

grails create-plugin clickpass.
Add ClickpassController.groovy to grails-app/controller.
Add User.groovy to grails-app/domain. This is a dummy file required for the plugin to be packaged.
Add ClickpassTagLib.groovy to grails-app/taglib.
Add _clickpass.gsp to grails-app/views.
Add welcome.gsp to grails-app/views/clickpass

ClickpassController.groovy

class ClickpassController {
    def openidService

    def index = { response.sendError(404) }

    def welcome = {
        def user = null
  
 if (openidService.isLoggedIn(session)) {
            user = User.findByOpenid(openidService.getIdentifier(session))
            if (user == null) {
                def urlString = "http://www.clickpass.com/process_new_openid?
                    site_key=Xr2zelexGn&process_openid_registration_url=
                    http%3A%2F%2Flocalhost%3A8080%2Fsocialr%2Fclickpass%2F
                    register&new_openid_completion_url=http%3A%2F%2F
                    localhost%3A8080%2Fsocialr%2Fclickpass%2Fwelcome&
                    requested_fields=nickname%2Cemail&required_fields=&
                    nickname_label=Nickname&email_label=Email"
                redirect(url:urlString)
            }
        }
    }

    def merge = {
        def user = User.findByUsernameAndPassword(params.user_id, params.password)
        def urlredirect = 
            java.net.URLDecoder.decode(params.clickpass_merge_callback_url) + 
            "&userid_authenticated=false"   
      
        if (user == null) {
            redirect(url:urlredirect)
        }

        user.openid = openidService.getIdentifier(session)

        if (user.save()) {           
            redirect(controller:'clickpass',action:'welcome')
        }
        else {
            redirect(url:urlredirect)
        }
    }
  
    def register = {
        def user = new User()
        user.username = params.nickname
        user.email = params.email
        user.openid = openidService.getIdentifier(session)
        user.password = ''
      
        if (user.save()) {
            flash.message = "Thank you for signing up."
            redirect(controller:'clickpass', action:'welcome')
        }
        else {
            flash.message = "There is a problem signing up."
            redirect(controller:'clickpass', action:'index')
        }
    }
}

index: Nothing for the index to do so set up a 404.
welcome: When a user is authenticated by OpenID, a search is made to see whether that OpenID is in the database, if not then redirect to the registration page. The urlString is obtained from Clickpass so this needs to be changed for each site
merge: A user, authenticated by OpenID, who wants to attach the ID to his account.
register: Create a new user account with the OpenID and save it.

User.groovy
Nothing in it except for the class declaration.

ClickpassTagLib.groovy

class ClickpassTagLib {
    static namespace = "clickpass"
 
    def bar = { attrs, body -> 
        try {
            out << render(template:"/clickpass", contextPath:"${pluginContextPath}")
        }
        catch(Exception e) {
            log.error(e)
        }
    }
}

The namespace allows one to use it, i.e <clickpass:foobar/>, in the code instead of the generic <g:foobar/>
For usage in an app, all that is required is <clickpass:bar/> to be inserted into the html.
The / in the template name is necessary to revert the path to the project base, otherwise it would go looking in the main app views folder.
The contextPath:"${pluginContextPath}" is not mentioned in the documention but after setting the path to the base, this sets it to the correct plugin views folder.

_clickpass.gsp
The html for this is generated by Clickpass, after registering your site, you can just copy and paste. This creates the Clickpass login bar.

welcome.gsp

<html>
    <head>
        <link rel="openid.server" href="http://www.clickpass.com/openid_server" />
        <link rel="openid.delegate" href="http://clickpass.com/public/iseec" /> 
        <title>Welcome</title>
    </head>
    <body>
        <h1>Welcome</h1>
        <openid:identifier /> 
        <openid:ifLoggedIn>
            <openid:logoutLink>Logout</openid:logoutLink>
        </openid:ifLoggedIn>
    </body>
</html>

This is a simple welcome page after the user successfully logs in and his account is verified.
One can replace it with their own or redirect from this page.
The OpenID is displayed as well as a logout link.

Aftermath

grails package-plugin to get it all into a zip file.
grails install-plugin /path/to/plugin/grails-example-0.1.zip to install it into an app.

IPhone MoviePlayer Sample Code 4

2008-10-24T10:50:00.003+08:00

MyOverlayView

Interface

Instance methods:

-(void)awakeFromNib;
-(void)dealloc;
-(void)touchesBegan:(NSSet *)touches withEvent:(UIEvent *)event;

Implementation

Imports:

#import "MyViewController.h"

-(void)awakeFromNib
Rotate the overlay 90 degrees clockwise then translate it to the centre of the screen. This will match the orientation the movie player uses, full-screen landscape mode.

-(void)dealloc
Call super dealloc.

-(void)touchesBegan:(NSSet *)touches withEvent:(UIEvent *)event
Similar to the MyImageView implementation. This creates a OverlayViewTouchNotification and fires it off.

IPhone MoviePlayer Sample Code 3

2008-10-24T10:11:00.002+08:00

MyImageView

Interface

Imports:

#import "MyViewController.h"

Instance variables:

mViewController

Instance methods:

-(void)touchesBegan:(NSSet *)touches withEvent:(UIEvent *)event;

Implementation

-(void)touchesBegan:(NSSet *)touches withEvent:(UIEvent *)event
Receives a set with the touches and the event. Gets a UITouch from the set and check it's phase. If it's UITouchPhaseBegan, play the movie.

IPhone MoviePlayer Sample Code 2

2008-10-23T14:11:00.004+08:00

MyViewController

Interface

Imports:

#import < MediaPlayer/MediaPlayer.h >
#import "MyOverlayView.h"

The line 'extern NSString * const OverlayViewTouchNotification' creates a global constant that will be used for touches to the overlay view. Any class importing MyViewController can access this variable.

Instance variables:

mMoviePlayer (MPMoviePlayerController)
mMovieURL (NSURL)
mOverlayView (MyOverlayView)

Instance methods:

-(NSURL *)movieURL;
-(void)initMoviePlayer;
-(IBAction)playMovie:(id)sender;
-(void)overlayViewTouches:(NSNotification *)notification;
-(IBAction)overlayViewButtonPress:(id)sender;

Implementation

Imports:

#import "MoviePlayerAppDelegate.h"

Initialise constant OverlayViewTouchNotification with the string 'overlayViewTouch'.

#pragma is a compiler directive. However the compiler ignores it if #pragma mark is used. #pragma mark helps with the organisation of your methods in the Xcode dropdown list.

'#pragma mark -' divvies up the methods by creating a spacer line while '#pragma mark Movie Player Routines' creates a bold label with the text 'Movie Player Routines'.

Movie Player Routines
- (void) moviePreloadDidFinish:(NSNotification*)notification
- (void) moviePlayBackDidFinish:(NSNotification*)notification
- (void) movieScalingModeDidChange:(NSNotification*)notification
These are events. When they are triggered is evident from the names. There is no code inside them.

-(void)initMoviePlayer

Register MPMoviePlayerContentPreloadDidFinishNotification (named after the definition in MPMoviePlayerController.h) with NSNotificationCenter.
The movie player object, mMoviePlayer, is created by allocating MPMoviePlayerController and initialising it with movieURL.
Register MPMoviePlayerPlaybackDidFinishNotification and MPMoviePlayerScalingModeDidChangeNotification, giving mMoviePlayer as the notification sender.
Obtain the MoviePlayerAppDelegate object and use its instance variables to set mMoviePlayer.
Register OverlayViewTouchNotification.

-(IBAction)playMovie:(id)sender

Play mMoviePlayer.
Get the application windows in an array.
Get the movie player window.
Add mOverlayView as the subview of the movie player window.

View Controller Routines
-(NSURL *)movieURL

Check mMovieURL whether it's null or not.
Passing will return mMovieURL.
Failing will get the main bundle, look for 'Movie.m4v', check it and convert the path to NSURL before assigning it to movieURL.

-(BOOL)shouldAutorotateToInterfaceOrientation:(UIInterfaceOrientation) interfaceOrientation
-(void)didReceiveMemoryWarning
-(void)overlayViewTouches:(NSNotification *)notification
-(IBAction)overlayViewButtonPress:(id)sender
Trivial or empty methods.

-(void)dealloc
Remove all registered notifications and release the movie player.

IPhone MoviePlayer Sample Code

2008-10-23T10:15:00.007+08:00

Deconstructing the MoviePlayer app to see how the Media Player Framework ticks.

MoviePlayer has 4 classes:

MoviePlayerAppDelegate
MyImageView
MyOverlayView
MyViewController

A new IPhone application automatically creates 2 classes, with project name MPC, MPCAppDelegate and MPCViewController. Therefore, MyImageView and MyOverlayView are additions while MyViewController has been renamed.

MoviePlayerAppDelegate

Interface

There are 2 different ways of including a file/class:

#import "MyViewController.h"
@class MyViewController;

The interface lists 3 instance variables of type NSInteger aside from the default UIWindow and ViewController:

backgroundColor
controlMode
scalingMode

The @property declares the accessor methods for the 5 variables. It has attributes which modify the accessors.

Example: @property (nonatomic, retain) UIWindow *window;

nonatomic means the accessor returns the value directly, the default is to lock, get value, unlock which is meant for multithreaded environments.

assign, retain or copy must be explicitly set if garbage collection is not used. assign is the default and means the setter uses a simple assignment. retain keeps the object variable in-memory and cannot be deallocated without your permission. copy means that a copy of the object should be used for assignment and the previous one is sent a release message.

3 instance methods:

-(void)setUserSettingsDefaults;
-(void)applicationDidFinishLaunching:(UIApplication *)application;
-(void)dealloc;

Implementation

3 NSString keys:

kScalingModeKey = @"scalingMode"
kControlModeKey = @"controlMode"
kBackgroundColorKey = @"backgroundColor"

@synthesize generates the accessors according to the attributes listed. If implemented manually, the code must follow what has been specified in the @property. @synthesize will only generate accessors that don't exist so if a variable has a setter already, then only a getter will be generated.

-(void)setUserSettingsDefaults
Sets up a test calling NSUserDefaults to see if there's a match. Passing will set the scalingMode, controlMode and backgroundColor variables. Failing will create the values based on the settings bundle info and then set the variables:

Get the path to the main bundle of the project.
Append Settings.bundle to the path using stringByAppendingPathComponent, which takes care of adding separators.
Append Root.plist to the path, same as before.
Create a dictionary using dictionaryWithContentsOfFile, giving the path to Root.plist.
Create the prefSpecifierArray array from the dictionary using the key, PreferenceSpecifiers, which is a tag in Root.plist.
- Root.plist is an XML file with dict, array and key tags. May be an easy way to read XML.
Declare 3 NSNumber variables and a dictionary, prefItem.
Loop prefSpecifierArray with the current value going into prefItem.

Set keyValueStr with the dictionary using the key, 'Key'.
Set defaultValue with the dictionary using the key, 'DefaultValue'.
Check if keyValueStr is equal to any NSStrings specified above.
Set the numbers with defaultValue if any is correct.

Create a dictionary containing the default values and their keys.
Register the dictionary with NSUserDefaults using registerDefaults.
Updates and writes to the persistent domains using synchronize.

-(void)applicationDidFinishLaunching:(UIApplication *)application
Add the ViewController's view as a subview of the UIWindow. Call self with setUserSettingsDefaults to get the default movie player settings. Lastly, call the ViewController with initMoviePlayer to create a movie player object.

-(void)dealloc
Release the UIWindow and the ViewController. Call the super's dealloc method.

Objective-C Classes

2008-10-21T16:16:00.005+08:00

Objective-C (Obj-C) is C with Object Oriented Programming tacked on. The object syntax takes some time to get used to, as it is decidedly different from the normal C++/Java syntax.

There are two parts for an object in Obj-C, which are the interface and the implementation. The parts should be stored separately, the interface in a header file (*.h) and the implementation is a *.m file.

The interface stores the instance variables and the prototypes for the class and instance methods. The implementation defines those methods.

Sample code:


*** CounterController.h ***
#import < Cocoa/Cocoa.h >
@interface CounterController : NSObject {
IBOutlet id display_update;
int count;
}
- (IBAction)minus_button_pushed:(id)sender;
- (IBAction)p10_button_pushed:(id)sender;
- (IBAction)plus_button_pushed:(id)sender;
@end

*** CounterController.m ***
#import "CounterController.h"
@implementation CounterController
- (IBAction)minus_button_pushed:(id)sender {
count--;
[display_update setIntValue:count];
}
- (IBAction)p10_button_pushed:(id)sender {
count+=10;
[display_update setIntValue:count];
}
- (IBAction)plus_button_pushed:(id)sender {
count++;
[display_update setIntValue:count];
}
@end

Things of note:

Obj-C uses import instead of include.
The class CounterController is a subclass of NSObject.
Inside the brackets are the instance variables display_update and count.
The IBOutlet before display_update means the display_update is an output on the GUI display in the Interface Builder.
id is a general type, meaning any object.
(IBAction)minus_button_pushed:(id)sender translates to IBAction minus_button_pushed(id sender).
The - before the methods classify them as instance methods, + for class methods.
The brackets [] indicate messaging, [display_update setIntValue:count] translates to display_update.setIntValue(count).

http://www.gmonline.demon.co.uk/cscene/CS7/CS7-05.html
http://en.wikipedia.org/wiki/Objective-C
http://www.otierney.net/objective-c.html

Grails - JSecurity plugin

2008-08-22T09:40:00.002+08:00

JSecurity is a Grails plugin that adds authentication and authorization to a web application. First we go about creating a project and installing the plugin:


grails create-app project_name
grails install-plugin jsecurity

Then we use the quick-start option, available once the plugin is installed, to generate JSecurity domain classes, controller and realm.


grails quick-start

We can follow the JSecurity Quick Start guide to create a functional security application or use the files present to customize and only take what is required.

Authentication

For the login authentication, we'll need the User class, the Auth controller and the DbRealm.

The User class will need a username and password field. The password field must be hashed, SHA1 is the default.

For the Auth controller, the important methods are signIn and signOut. For signIn, the important lines are the authToken definition, the jsecSecurityManager login and the try-catch block.

Authentication occurs in the DbRealm. The important lines in the authenticate method are the account definition and the credentialMatch. You may customize the authToken class by changing the static line, replacing the default with your own class.

credentialMatch defaults to SHA1 hashing, to change it, modify grails-app/conf/spring/resources.groovy and add the line credentialMatcher(org.jsecurity.authc.credential.hashMatcher). Replace hashMatcher with the hash class of your choice.

Roles

Roles depend on the hasRole method of the realm. This method accepts a principal, usually a username, and a rolename. The output is a simple true or false boolean, making the logic inside the method completely up to the programmer. The classes generated by the quick-start are not all necessary. We can easily use a variable in the user class to determine the role, modifying the hasRole method as required, thus no additional classes are needed.

For this project, we are using the default hasRole method and keeping the Role as well as the UserRoleRel class. This, along with the User class, allows us to have users and roles, assigning the users to a role in a many-to-one relationship. Refer to the JSecurity Quick Start guide for the way to bootstrap it.

Authorization

Having users and roles, we can now set up proper access control by creating a security filter file, grails-app/conf/SecurityFilters.groovy. Filter rules cannot overlap in actions, just modify the accessControl to include the desired roles. The rules return true or false, allowing or denying access.

Security Filter Example

import org.jsecurity.SecurityUtils
// Customise the interceptor behaviour by implementing onNotAuthenticated() and onUnauthorized() methods in your filters class.
// Default behaviour remains as before (redirect to login page for an unauthenticated user, redirect to unauthorized page for unauthorized access).
class SecurityFilters {
 def filters = {
//   Ensure that all controllers and actions require an authenticated user,
//   except for a few controllers
    auth(controller: "*", action: "*") {
       before = {
//            Exclude these controllers.
          if ((controllerName == "search" && actionName == "unboundSearch") || (controllerName == "business" && actionName == "show") ||
          (controllerName == "user" && actionName == "signup")) return true
            
//            This just means that the user must be authenticated. He does
//            not need any particular role or permission.
          accessControl { true }
       }
    }

//      Roles return true or false
//      Add reviews to businesses. Only Administrators or Users can do this
    addReview(controller: "review", action: "create") {
       before = {
          accessControl {
             role("Administrator") || role("User")
          }
       }
    }

//      Add pets to users
    addPets(controller: "pet", action: "(create|save)") {
       before = {
          accessControl {
             def user = User.findByUserName(SecurityUtils.subject.principal)
             def allowAccess = false

             if (role("Administrator")) {
                allowAccess = true
             }
             else if (params.id.toLong() == user.id) {
                allowAccess = role("Business User") || role("User")
             }
             return allowAccess
          }
       }
    }
 }
}

Permissions

Permissions offer finer access control, like read, update, delete. Roles can be considered as named collections of permissions, like administrator, user, superuser. Nothing was done with permissions, it was entirely a role-based affair. Refer to the documentation for usage.

GSP training

2008-07-16T08:38:00.002+08:00

Brought Fauzan up to speed on the project work that's been done, explaining the search controller and the gsp code.

He made a simplified version of the search page, recreating the basic functionality.

The CSS was not touched on. I figure one can just strip it from the page and into a file then figure out where to put and call the file.

Similarly the javascript, dealing with the Google map functionality, may be moved to a file and referred to. It may not work due to the gsp tags inside the code.

TWiki Replication

2008-06-26T17:24:00.002+08:00

Setup a backup TWiki by using rsync, cron and public key authentication. Had problems getting cron to run on root since a normal user does not have the permissions necessary. Solved them when I realized I do need the account active (as I had ignored the expired warning when logging in as root).

Made little headway with regards to the MixTape build on hudson (reminder: change the SVN location once migration is finished). Installed the Flex 3 SDK and used it to compile the mxml files. However the swf files do not seem to work.

The grails search plugin sets up easily. Run 'grails install-plugin searchable', add 'static searchable = true' to any domain class, add data and run the app. The default allows you to search the fields of the domain class, objects may need a specific mapping though.

In the process of migrating SVN projects from one repository to another.

Flex File Upload using Grails Backend

2008-05-28T12:41:00.005+08:00

This file upload application is based off the one by Nocturnal at Coding Cowboys, a Flex client with a PHP backend. The client has been stripped down to it's core functions and PHP replaced with a Grails backend.

The Grails backend is a file controller with an action defined:


def upload = {
  if(request.method == 'POST') {
    Iterator itr = request.getFileNames();
     
    while(itr.hasNext()) {
      MultipartFile file = request.getFile(itr.next());
      File destination = new File(file.getOriginalFilename())
      
      if (!file.isEmpty()) {
      file.transferTo(destination)
//       success
      }
      else
      {
//    failure
      }
     }
     
//     Trigger an Event.COMPLETE event, notifying the Flex client
     response.sendError(200,'Done');
 }
}

This is capable of uploading multiple files at once and can be tested using a view gsp:


<html>
    <head>
        <meta http-equiv="Content-Type" content="text/html; charset=UTF-8"/>
        <meta name="layout" content="main" />
        <title>Test Upload</title>
    </head>
    <body>
        <div class="body">
            <h1>Upload File</h1>
            <g:form method="post" action="upload" enctype="multipart/form-data">
             <input type="file" name="file_1"/><br/>
             <input type="file" name="file_2"/><br/>
             <input type="file" name="file_3"/><br/>
             <input type="file" name="file_4"/><br/>
             <input type="file" name="file_5"/><br/>
             <input type="submit"/>
            </g:form>
        </div>
    </body>
</html>

The Flex client has been pared down to 3 main functions, adding files to the upload list, removing them from the list and finally uploading the files to the server.

First the imports, global variables and initializer:


import mx.controls.*;
import mx.managers.*;
import mx.events.*;
import flash.events.*;
import flash.net.*;
   
// Convention: Underscore to differentiate global and local variables
private var _refAddFiles:FileReferenceList;
private var _refUploadFile:FileReference;
private var _arrUploadFiles:Array;
private var _numCurrentUpload:Number;

private function initApp():void {
 _refAddFiles = new FileReferenceList();
 _refAddFiles.addEventListener(Event.SELECT, onSelectFile);
 _arrUploadFiles = new Array();
 _numCurrentUpload = 0;
}

The add-files function opens the browse window and enable the selection of one or more files to add to the upload list:


// Called to add file(s) for upload
// The fileList property is populated anew each time browse() is called on that FileReferenceList object.
private function addFiles():void {
 _refAddFiles.browse();
}

// Called when file(s) are selected
private function onSelectFile(event:Event):void {
 // Add files to _arrUploadFiles
 if (_refAddFiles.fileList.length >= 1) {    
  for (var k:Number = 0; k < _refAddFiles.fileList.length; k++) {
   _arrUploadFiles.push({
    name:_refAddFiles.fileList[k].name,
    size:formatFileSize(_refAddFiles.fileList[k].size),
    file:_refAddFiles.fileList[k]});
  }
  listFiles.dataProvider = _arrUploadFiles;
  listFiles.selectedIndex = _arrUploadFiles.length - 1;
 }
}

The remove-files function:


private function removeFiles():void {
 var arrSelected:Array = listFiles.selectedIndices;
 if (arrSelected.length >= 1) {
  // Null selected files
  for (var i:Number = 0; i < arrSelected.length; i++) {
   _arrUploadFiles[Number(arrSelected[i])] = null;
  }
  
  // Remove the null entries
  for (var j:Number = 0; j < _arrUploadFiles.length; j++) {
   if (_arrUploadFiles[j] == null) {
    _arrUploadFiles.splice(j, 1);
    j--;
   }
  }
  listFiles.dataProvider = _arrUploadFiles;
  listFiles.selectedIndex = 0;     
 }
}

The upload function is where the point of integration with Grails is, in the request calling up the controller. Although the Grails backend can handle multiple files, Flex can only initiate the upload one file at a time. The event listener and the return response in the Grails controller allow the upload function to loop:


private function uploadFiles():void {
 if (_arrUploadFiles.length > 0) {
  listFiles.selectedIndex = _numCurrentUpload;
  
  var request:URLRequest = new URLRequest();
  request.data = sendVars;
     request.url = "http://localhost:8080/App/file/upload";
     request.method = URLRequestMethod.POST;
     _refUploadFile = new FileReference();
     _refUploadFile = listFiles.selectedItem.file;
     _refUploadFile.addEventListener(Event.COMPLETE, onUploadComplete);
     _refUploadFile.upload(request, "file", false);
 }
}

private function onUploadComplete(event:Event):void {
 _numCurrentUpload++;
 if (_numCurrentUpload < _arrUploadFiles.length) {
  uploadFiles();
 } else {
  _numCurrentUpload = 0;
  _arrUploadFiles = new Array();
  listFiles.dataProvider = _arrUploadFiles;
  listFiles.selectedIndex = 0;
  Alert.show("Files have been uploaded", "Upload Complete");
 }
}

And lastly the UI:


<mx:Panel title="Files Upload" width="100%" height="100%">
 <mx:DataGrid id="listFiles" allowMultipleSelection="true" verticalScrollPolicy="on"
  draggableColumns="false" resizableColumns="false" sortableColumns="false" width="100%" height="100%">
  <mx:columns>
   <mx:DataGridColumn headerText="File" dataField="name" wordWrap="true"/>
  </mx:columns>
 </mx:DataGrid>
 
 <mx:ControlBar>
  <mx:Button label="Add" click="addFiles()"/>
  <mx:Button label="Remove" click="removeFiles()"/>
  <mx:Button label="Upload" click="uploadFiles()"/>
 </mx:ControlBar>
</mx:Panel>

The Flex mxml/swf files must be moved to the web-app folder of the Grails project and the embedded Jetty activated by 'grails run-app' in order to run this application.

The files will be uploaded to the project directory and thus are not accessible by the webapp. The line to amend is in the Grails controller:


File destination = new File("web-app/" + file.getOriginalFilename())

Note this only works while the project is in development mode. For war files, the base directory would depend on where the server is started, usually in the server home directory. So for a standalone Jetty server, the path to add would be "webapps/App/".

Using the war file in a Tomcat server has been problematic. Only one file can be successfully uploaded to the server, there is no looping.

Links:
Coding Cowboys: Flex upload component

Flex Client:
http://livedocs.adobe.com/flex/3/langref/flash/net/URLRequest.html
http://livedocs.adobe.com/flex/3/langref/flash/net/FileReference.html
http://livedocs.adobe.com/flex/3/langref/flash/net/FileReferenceList.html
http://livedocs.adobe.com/flex/3/html/help.html?content=17_Networking_and_communications_7.html

Grails Backend:
http://www.grails.org/Controllers+-+File+Uploads
http://docs.codehaus.org/display/GRAILS/File+Upload

Flex Training - Last bit

2008-05-23T16:36:00.002+08:00

Used list-based controls that accept a data array and display out depending on design. Then dealt with data forms, accepting user input, validating it and supposedly entering it into the database.

The last bit was how to create a release version of the Flex application.

Flex Training - Dynamic Data

2008-05-22T17:16:00.002+08:00

Read an XML file using a HTTPService and displayed it. Autogenerated a PHP XML service based on database contents previously inserted. The Flex application was supposed to use the service to access the data in the database and display it on the page. However seems there's a problem with the service.

The videos make use of MAMP, a Mac-Apache-MySQL-PHP all-in-one webserver. There may be conflicts with the MySQL that was previously installed and the MAMP's version.

Flex Training - Debugging and Event Handling

2008-05-21T18:19:00.002+08:00

Debugging - How to set breakpoints and traces (logging to console). The breakpoints suspend the application, allowing one to see the variables and objects present at that particular time.

Event Handling - Dealing with events and how to listen for them. One can also create custom events. With a breakpoint, the anatomy of an event can be explored, showing the specific variables and the inherited ones that make up the event.

Flex Training - 3 chapters down

2008-05-20T18:12:00.002+08:00

Learned layouts, CSS and layered views.

VBox, HBox, Canvas and Panel layouts place their content vertically, horizontally or using absolute positioning.

A feature of the Flex CSS is embedding font(s) into the application so that the look remains the same for all. The drawback to that is the increased size of the application.

There are three layered view components, the ViewStack, TabNavigator (which extends ViewStack) and Accordian. The latter two contain their own switching mechanism, the first has to include in custom code utilizing the selectedIndex or selectedChild properties or one of these four bars: TabBar, LinkBar, ButtonBar and ToggleButtonBar.

Flex Training - Of custom components and state changes

2008-05-16T16:51:00.003+08:00

Went through the creation of a basic custom component and how to use it in a application project.

More information on the Design perspective, explaining the different areas such as the properties and controls. Designed view states that allow a login form to become a register form by a push of a button and vice versa.

Flex Training - Day 2

2008-05-14T16:39:00.002+08:00

Going into interesting territory. An explanation of the current programming language used, ActionScript 3, is given along with how MXML translates to it and vice versa. It then shows the visual design and their controls, correlating it with the mxml source that shows up as you drag and drop controls on the design.

Found out what was up with the bindable tags that are appended to ActionScript variables. Bindable enables a variable to broadcast its content when updated, alerting the elements that listen for and display or use such content. By default ActionScript variables are not bindable and each and every one must be appended for all to be bindable. MXML variables (eg. mx:Number) are automatically bindable however.

Flex Training

2008-05-13T17:08:00.002+08:00

Assisting in VMWare Server, Ubuntu 8.04 Server edition and Drupal installation took up the better part of the day. The Server edition was not necessary, however the Desktop edition downloaded cannot be used due to corruption/incomplete transfer. The advantage of the Server edition is that it allows the setup of the mail, LAMP and OpenSSH servers automatically, thus mail works and ssh to the installed OS is possible. The downside is that the user interface is the command-line. This is rectified by following How to install GNOME on Ubuntu Server
specifically these lines:


sudo apt-get update

sudo apt-get install ubuntu-desktop
sudo apt-get install gdm
sudo /etc/init.d/gdm start
sudo dpkg-reconfigure xserver-xorg

The resulting download is over 300MB, so it will take some time to finish with the process. In the meanwhile, one will have to suffer with the command-line ;).

After that, Flex training took up the rest of the day. Started out on the easy side, going over the 'Introduction' and 'Getting Started' videos. Mostly reinforcing what was already known, though there were a few interesting bits, like making full use of a two-monitor setup by detaching some of the Flex windows from the main application and putting it in the spare screen area. Then one can save that perspective and reuse it. It is applicable to Eclipse as well, seeing that the Flex Builder is built on it.

Fisheye

2008-05-07T17:36:00.002+08:00

Found the effect we're looking demoing here. The actual component is found here. Now to discover how to include it in the project.

Links of Interest:
Example: Flex Recipe App MXML for Drupal

Flex and Drupal

2008-05-06T16:45:00.003+08:00

The services module is responsible for exposing the Drupal data to any external app, with the amfphp module adding that particular functionality. The image, img_assist, pathauto and token modules are used to add images and text descriptions to Drupal. The views module has methods available in the services module, making it convinient to call.

Pulling the data from drupal yields a filepath for the image, the actual file is displayed by adding the IP address to the filepath, e.g. http://10.10.3.131/drupal5/files/images/pic.jpg.

Links of interest:
Reading And Writing Drupal With Flex
Howto: Ubuntu Linux convert DHCP network configuration to static IP configuration

Setting up a test Drupal site

2008-05-05T16:51:00.002+08:00

The old Gutsy Gibbon VM was totally screwed up so the Hardy Heron server edition was installed on a new one. It has Drupal 5.7 in the apt-get packages and so installation was a breeze. One thing it should have mentioned was the need to hit the install page, http://ip/drupal5/install.php, to initialize the db and everything else.

Turning clean urls on was as simple as typing two lines of code:

sudo a2enmod rewrite
sudo /etc/init.d/apache2 restart

Then testing the clean urls and turning it on. This may be Ubuntu-specific though.

A bunch of modules were downloaded and installed:
amfphp
cck
image
img_assist
pathauto
services
swfaddress
token
views

The amfphp is a plugin that requires the implementation available here.

Dupal CMS with Flash/Flex Frontend

2008-05-02T16:18:00.002+08:00

Found a site, cascadingStyle, that talks about Druplash and Druplex (Drupal-powered Flash and Flex) sites and has a presentation pdf available on that subject. The pdf lists two sites, Steven W. Merrill, Tenor and Martin Engineering, running on Drupal+Flash.

Three Drupal modules, SWFAddress, Services, AMFPHP are required to go about 'flashing' the site.

Here is another take on the subject, using the same modules.

TaskMaster - Day 10

2008-04-30T16:49:00.003+08:00

The new user can be added to the project or task and data already entered is preserved. This is done by removing the form's default save action and implementing actions for each and every submit button. The submit button will dump the data into the params map and all that is required is to store/forward it along then retrieve it back afterwards.

The new user is to be notified by an email containing a link back to the project/task. The messagingService is responsible for this but has not yet been created.

More information here :)

TaskMaster - Day 9

2008-04-29T16:53:00.003+08:00

When in the task or project creation page, a new user may be created given just an email address and name. Unfortunately, redirecting to the user creation page wipes out any data entered in the task/project pages. So far there has been no way of saving the data.

More information here :)

TaskMaster - Day 8

2008-04-28T16:24:00.003+08:00

Finally added the user members to the project. Used a custom tag here that displays a list of checkboxes.

Also cleared the validation and a few other stuff.

More information here :)

TaskMaster - Day 7

2008-04-25T16:45:00.003+08:00

Managed to clear a minor issue but hit a wall with the adding of members to a project, there does not seem to be any good examples covering the view interface for a 1:m relationship. Much frustration.

So right now there are three major issues, the messaging service, instant creation of a user for task or project inclusion and addition of members to projects.

More information here :)

TaskMaster - Day 6

2008-04-24T16:53:00.003+08:00

Spruced up the views, changing or removing elements and code.

Aside from the MessagingService, which can't take off without a mail server, there are a few story issues. One is the adding of members to a project, figuring out how to do it. Another is to auto-create a non-existing user so that he may be added to a project or assigned a task, later on the user needs to complete his registration.

Still to tackle the validation problem.

It lives! :)

TaskMaster - Day 5

2008-04-23T16:10:00.003+08:00

Added in some sample data. Made a few changes to the login and registration pages as well as creating a navigation bar. Validation of the registration data does not seem to follow the constraints listed in the domain model. The project creation page needs to be tweaked, to allow members to be added in.

The views in Grails are where all the presentation elements are present, with the controllers chipping in the model and the business logic. Whereas in Wicket, the presentation elements are coded and inserted through the placeholder ids which had been seeded in the HTML. Grails entails more time in HTMLland than is the case with Wicket.

Creating the war file for the project is a simple one-liner: grails war. This beats having to fuss around your own ant script and making sure all dependencies are being added. Now to find out how to change the version number on the war file, being stuck at 0.1.

Reading up on the messaging service, this seems a simple way of doing it.

The other blog is dead :)

TaskMaster - Day 4

2008-04-22T08:54:00.003+08:00

Two services were created, AuthenticationService and MessagingService. The former service is used for logging in users (especially later on when the authentication methods increase, e.g LDAP, OpenID) and the latter service is to email users for whatever purposes.

A BaseController was created to add in an interceptor and the authorization to all controllers via inheritance:


def beforeInterceptor = [action:this.&auth, except:['login', 'logout', 'register']]

def auth() { 
    if(!session.user) {
        redirect(controller:'user',action:'login') 
            return false 
    } 
}

Cobbled together a working rough login and registration page. The controllers and views will be the focus from now on.

The other blog is on hiatus for the moment :)

TaskMaster - Day 3

2008-04-21T16:23:00.003+08:00

Refined testing, splitting the various CRUD activities into different test methods. Said activity took up the whole day, squashing failures as they turn up.

Regarding the dates previously, switched to using the dateCreated and lastUpdated variables as GORM handles the timestamping with those variables automatically. There is another date variable that now works fine, making the past problem a head-scratching puzzle.

Next up is the service tier. On that note, found a nice Grails CRUD example at http://infoq.com/minibooks/grails. The PDF at that site shows how to create a Racetrack CRUD.

More information here :)

TaskMaster - Day 2

2008-04-18T08:51:00.002+08:00

Today was testing day. Having previously done no testing on Grails, there was some reading involved. Much frustration was involved in trying to get the test to run. There was light at the end of the tunnel when this, under the section 'Testing Domain Classes' was found. Basically the need to flush while saving, to make sure it all flows to the database.

The light that was the flush was quickly snuffed out by the bold, hated 'FAILURE' from the test run. Much despair ensued. After some gritting of teeth and grinding of stone, the reason for the failed test surfaced. Apparently dates are not welcomed and with their exclusion, 'SUCCESS' was assured.

Now to properly build the test classes...

More information (or not) here :)

TaskMaster - Day 1

2008-04-17T17:06:00.002+08:00

TaskMaster is an issue-tracking system, in the image of Bugzilla, to be developed using Grails. This is mostly a development project for training.

Eclipse is used as the development IDE. Integration with Groovy and Grails needed to be set up.

The first day was spent sketching out the domain model, with three primary classes; User, Task and Project. Users are assigned Tasks to do and Projects are user groups for a particular purpose.

A few problems came up regarding testing and checking into SVN.

More information here :)

J2ME Development on OSX

2008-04-15T11:36:00.005+08:00

This follows the tutorial on java.net, which shows how to compile and package J2ME apps manually via the command-line.

First, a wireless toolkit (J2ME SDK) is needed. The mpowerplayer toolkit supports OSX and it comes with a preverifier. The MicroEmulator toolkit has a better emulator than mpowerplayer though.

Once a source file is provided, it is compiled by:
javac -bootclasspath /path/cldc-1.1.jar:/path/midp-2.0.jar /package/path/Source.java

The bootclasspath option uses the specified library files for the compilation, instead of the normal Java environment.

The next step is to preverify the class:
mpowerplayer-folder/osx/preverify/preverify -classpath /path/cldc-1.1.jar:/path/midp-2.0.jar package.name.Source

The preverifier default is to create a folder called output with the same package structure in the current directory. The preverified file is placed there.

In the output folder, create a Manifest.mf with the following:


MIDlet-Name: Source
MIDlet-Version: 1.0.0
MIDlet-Vendor: NameMe

Also in the output folder, create a jar file:


jar cvfm Source.jar Manifest.mf ./com

The penultimate step is to create a jad file descriptor, Source.jad, with the following:


MIDlet-1: Source, , package.name.Source
MIDlet-Name: Source
MIDlet-Version: 1.0.0
MIDlet-Vendor: NameMe
MIDlet-Jar-URL: Source.jar
MIDlet-Jar-Size:
MicroEdition-Profile: MIDP-2.0
MicroEdition-Configuration: CLDC-1.1

The last step is to fill in MIDlet-Jar-Size with the actual number of bytes the jar file takes up. This completes compiling and packaging.

Testing the packaged app in an emulator:
java -jar microemulator-folder/microemulator.jar Source.jad

Allowing the default behaviour of downloading the jad file, which then installs the jar file, set the MIDlet-Jar-URL to:
http://domain/url/Source.jar

Sites of Interest:

J2ME Development on OS X

J2ME Development on OS X, revisited

Syncing your Google Calendar with the Samsung BlackJack using OggSync

2008-03-27T09:42:00.004+08:00

Firstly, you will need to go to OggSync Mobile Install and download the 2 files there. Most likely you would not need the .NET CF 2.0 SP2 file if you're running WM6 on the BlackJack. Using IE to download, it will automatically ask you whether you wish to install when the download completes, otherwise you will have to call up the File Explorer and go to /My Documents.

Once installed, OggSync will show up as a new application in the Start menu. When started, you will need to enter the GMail authentication details and complete the initial settings, such as picking the default Google Calendar to sync with and the timezone. After that, you can also set how far in the past and the future you want to sync with 6 months and 5 years given as the maximum respectively.

With the freeware version, OggSync allows you to sync to only one calendar manually. The Pro version allows multiple calendar syncing and you can set a time interval for it to sync automatically.

AMF and BlazeDS

2008-02-14T11:07:00.002+08:00

Action Message Format (AMF) is a serializing format like XML and JSON. Where the latter two sends formatted text, AMF sends binary data. This allows one to transfer your own domain objects instead of creating a data transfer scheme for XML or JSON. BlazeDS, an open source project from Adobe, is the 'enabler' for AMF.

This site compares the data loading performance between AMF, XML, JSON and a few others. The screenshot shows that AMF took the least time and bandwidth to load the data.

Sites of Interest:
BlazeDS and what it means for the developer community
BlazeBench: Why you want AMF and BlazeDS
Ajax and Flex Data Loading Benchmarks

Flex

2008-02-13T14:42:00.002+08:00

Flex is a presentation layer in a SWF package using Flash, residing in the client and interacting with the server to obtain data. It is very similar to Wicket, in that you would lay out UI components in a design and add business logic behind the scenes. While Wicket does it with HTML and Java, Flex does it with MXML and ActionScript. Flex is more towards application development than AJAX, but both can use each other.

A Flex interface requires a server API to talk to. This site shows four ways (ColdFusion, PHP, Java and ASP.NET) to go about it. The data returned by the API would be in XML for ease of use.

Sites of Interest:
What is Flex?
Flex and Ajax: So Happy Together
From Tags to Riches: Going from Web 1.0 to Flex

Food for thought: Endless learning

2008-02-11T11:27:00.000+08:00

The Mythical 5% by Bruce Eckel started the brain juices flowing on how to be a better software engineer, following on one of the things the article mentions - listening to podcasts. It should definitely bring in new ideas, though the question is where to start? Followed some links and trying out The Java Posse and Software Engineering Radio. Now I need to get my MP3 player working again...

One of the things I would like to experience would be the pair-programming and working in a team. Being the sole developer has a lot of advantages, such as being able to do with the source code as you will, but it leads to an entrenched viewpoint and blindspots abound.

There is always plenty to learn, expanding on my Java knowledge would be a good thing. And I would like to delve deep in some other language as a counterweight, exposing me to different coding techniques and styles.

Spring 2.1 Annotated Dependency Injection

2007-09-07T14:29:00.000+08:00

Annotation-Driven Dependency Injection in Spring 2.1: This page shows how one can use annotations to entirely inject dependencies. This allows the role of the application context xml file to be minimised, with no bean definitions necessary. Instead the file can be set to an annotation context with scanning of the classes done in the package specified.

The downside to this is the POJOs are now infected with Spring configuration. For a large project with dozens of files, this would make changes difficult.

Wicket 1.3

2007-09-06T10:38:00.001+08:00

Going over the Migrating to Wicket 1.3 page and looking at the API changes which affect the FilmFestival project:

Using a Filter: Change the assignment of ServletContext in the WebApplication subclasses. Change the web.xml listing.

IModel change: The FilmFestival's DetachableBlogEntryModel need to extend some other class as per the migration mapping due to AbstractReadOnlyDetachableModel having been removed. Note the method name changes.

Interesting bit on CompoundPropertyModel. The project uses CompoundPropertyModels to automatically bind data elements to their page counterparts. As the approach used is different, alteration should not be required.

Validation: The API for validation has been moved from package wicket.markup.html.form.validation to package org.apache.wicket.validation.

Repeaters: This API has been moved from extensions to the core, requiring only a Ctrl-Shift-O to relink the classes affected in the import section.

DatePicker: This component has been removed. Alternatives listed need to be used.

Custom resource loading: Would this allow the html files to be separated from their java counterparts? The instructions here does not seem to use it.

One of the new features in Wicket 1.3 is Enclosure. Which, as shown here and here, is a way of toggling markup visibility for components.

Installing JIRA, a bug & issue tracker

2007-08-29T11:36:00.000+08:00

Installing the WAR version
Obtain and extract the WAR file:

wget http://www.atlassian.com/software/jira/downloads/binary/atlassian-jira-enterprise-3.10.2.tar.gz
gunzip atlassian-jira-enterprise-3.10.2.tar.gz
gtar -xf atlassian-jira-enterprise-3.10.2.tar.gz

Tomcat is installed through Blastwave. The path is /opt/csw/share/tomcat5. Follow the Tomcat installation instructions. The transaction factory in the WAR's edit-webapp/WEB-INF/classes/entitymanager.xml is set to Tomcat and does not need to be changed. There are only two settings to be changed for use with PostgreSQL. Build the distribution and copy from dist-tomcat/ to the web server.

Copy and edit the jira.xml file:
cp {WAR}/dist-tomcat/tomcat5.5/jira.xml /opt/csw/share/tomcat5/conf/Catalina/localhost

The resource should now have four lines:

username="jira_user"
password="{password}"
driverClassName="org.postgresql.Driver"
url="jdbc:postgresql:jiradb"

Obtain the JDBC driver for Tomcat and move it to common/lib:
wget http://jdbc.postgresql.org/download/postgresql-8.1-410.jdbc3.jar

Extract the jars to Tomcat's common/lib/ directory:
wget http://www.atlassian.com/software/jira/docs/servers/jars/v1/jira-jars-tomcat5.zip

Start Tomcat and go do the setup wizard. LDAP authentication is not tightly coupled therefore you must create JIRA users with the same name as the LDAP users for this to work. A bulk-import of LDAP users can be done if desired.

Grinder 3, a Java load tester

2007-08-29T09:59:00.000+08:00

The file download is available here. Once downloaded, you can extract it out to any location. It is used through the java command as explained at this site and here as well.

To get it running as soon as possible, run TCPProxy and set the browser proxy to the port 8001. The proxy will record the actions you take through the browser. Once complete, close the proxy and start the console. Create a grinder.properties from the example given and start the agent. Use the console to start the processes and stop it when you have had enough.

The results obtained are mostly time statistics. One would have to create or edit a script to obtain more specific details. Testing against a local webapp using Tomcat revealed a Java heap memory exception in the Tomcat logs. Against the same webapp on a public server, there was no errors in the Grinder logs.

Installing XWiki

2007-08-28T09:09:00.000+08:00

Before installing the XWiki WAR version listed here, you'll need to install a servlet container and a relational database, in this case Tomcat and PostgreSQL, following the instructions given.

After configuring XWiki (the concluding step), you will need to import the default skin for functionality. It can be downloaded at the same place as the WAR file.

The LDAP Authentication is basically straightforward. Replace the values with your own and you are set. LDAP users have to login to XWiki before they have a presence there. XWiki groups have to be assigned as the LDAP groups are not imported in.

Installing Confluence

2007-08-24T09:42:00.000+08:00

Follow the install guide here. Going through the standalone version, ensure that the Java JDK is installed. Obtain the latest Confluence archive, note: 2.5.6 has a bug that will not allow LDAP authentication to work properly. Also following trying to use the old database with LDAP authentication will open up this bug.

With Solaris, there is no need for the X11 libraries. The home directory is where Confluence data will be kept, Confluence itself is in the archive. Go to the Confluence Setup Wizard Guide. Create the admin user.

LDAP Authentication
Follow this. Note Step 3.2. Step 5 must be successful. For Step 6, refer to this and this. There is a problem with linking LDAP users with their groups, currently both are shown but are not connected. Current solution is to create a Confluence group and use that.

LDAP Authentication with TWiki

2007-08-21T11:00:00.000+08:00

Using LdapContrib for transparent authentication. Follow the installation instructions, first installing the required dependencies either through the script or manually. This explains more on what LdapContrib does. The LdapNgPlugin and NewUserPlugin may be desired.

Installing TWiki

2007-08-17T09:52:00.000+08:00

This post is regarding the installation steps of TWiki on a remote Solaris 11 server. This assumes the installer has root access. The main reference for this post is here - Installing TWiki 4.x on Solaris 10.

Create the following (arbitrary) directory structure in the filesystem:

/apps/twiki-root
/apps/twiki-root/bin
/apps/twiki-root/twiki - A symlink to the directory below
/apps/twiki-root/twiki-4.1.2
/apps/twiki-root/apachemodules
/apps/twiki-root/perlmodules

Download and unpack the latest TWiki version. Create the LocalLib.cfg file in /apps/twiki-root/twiki/bin. Modify it to set the twikiLibPath and the path for TWiki related Perl modules:

$twikiLibPath = "/apps/twiki-root/twiki/lib";

@localPerlLibPath = ('/apps/twiki-root/perlmodules', '/apps/twiki-root/perlmodules/i86pc-solaris-64int');

Building CPAN Perl modules on Solaris 10

Go over to CPAN and download the CGI::Session and Digest::SHA1 modules. Unpack them into temporary directories and use these commands:

$ /usr/perl5/bin/perlgcc Makefile.PL [LIB=/apps/twiki-root/perlmodules]
$ make
$ make test
$ make install

Blastwave for GNU grep, diff and rcs

Follow Blastwave installation instructions to enable pkg-get and install grep, diff and rcs. They will be available in /opt/csw/bin. Create symlinks for egrep(->ggrep), fgrep(->ggrep) and diff(->gdiff).

Apache

Create the /etc/apache2/httpd.conf from the example in that directory. Use the ApacheConfigGenerator to configure a twiki.conf file. Settings used for the generator:
Enter the full file path to your twiki root directory (mandatory):
/apps/twiki-root/twiki

Enter the IP address range or hostnames that will have access to configure - separate with spaces
localhost

Enter the list of user names that are allowed to view configure
Empty

Enable mod_perl
Unchecked

Choose your Login Manager:
None - No login

Prevent execution of attached files as PHP scripts if PHP is installed:
No PHP Installed

Block direct access to viewing attachments that ends with .htm or .html
Unchecked

Block direct access to viewing attachments in Trash web
Unchecked

Append the file to httpd.conf or otherwise have Apache load it. Restart the Apache Web Server:

svcadm disable apache2
svcadm enable apache2

Check /var/svc/log/network- http:apache2.log to check whether the server is up or it failed to start. Troubleshoot as required.

Browser configuration

Go to http://hostname/twiki/bin/configure to continue configuring TWiki. Use /opt/csw/bin to add to the path. Also use it or create a symlink for rcs. Complete the setup.

Done. More details as forthcoming.

LDAP Authentication with SugarCRM

2007-08-10T09:42:00.000+08:00

Both SugarCRM and LDAP are installed. LDAP is to be used to authenticate users in SugarCRM.

Login to SugarCRM as admin. Click on the 'Admin' button at the top of the page and then 'System Settings' in the main page.

Set 'Enable LDAP'. The default LDAP server and port number are localhost and 389 respectively. Change as required. The base dn is the location where the search for users begin, in this case dc=nodomain.

For the bind attribute, disregarding the example text, it should be dn. More information here. The login attribute is the username to be used. Any attribute can be used (e.g uid, sn, cn). An important thing to note is with 'Auto Create Users' set, a change in login attribute will create new users with that attribute as the username. If 'Auto Create Users' is not set, authentication will fail as the user may be present in LDAP but not in the SugarCRM database due to the username.

Authenticated user and password is the LDAP account to be used for searching. The 'Auto Create Users', as mentioned before, creates new users from their LDAP information if they are not present in SugarCRM. The encryption key may be left empty.

svnsync: A Subversion Mirror

2007-08-07T11:13:00.000+08:00

The steps to mirror a subversion repository are detailed here

My take on those steps:

$ svnadmin create /var/svn/backup
$ echo '#!/bin/sh' > /var/svn/backup/hooks/pre-revprop-change
$ chmod +x /var/svn/backup/hooks/pre-revprop-change
$ svnsync init file:///var/svn/backup svn+ssh://[username]@svn.sixpaq.com/home/[username]/svn/projects
$ svnsync sync file:///var/svn/backup

Other places of interest are here, here and here

Python strings (vs. Java strings)

2007-07-18T12:32:00.000+08:00

Python strings are more or less similar to Java strings. A difference is the formatting of Python strings are done in the strings themselves. Also Python strings have more methods than their Java counterparts.

Methods:
The Python methods, find (the same as index) and rfind (rindex) have their Java opposite, indexOf and lastIndexOf. So does lower and upper in the Java methods toLowerCase and toUpperCase. split, replace, startsWith and endsWith are common to Python and Java.

join, the opposite of split, does not have a Java equivalent. The join method concatenates the members of a string sequence together with a specified divider. Neither do these methods: count, isalpha, isalnum, isdigit, islower, capitalize, isupper, title, istitle, swapcase, expandtabs and translate.

Formatting:
The string formatting operator, the percent (%) sign, does all the work as an example from the Beginning Python: From Novice to Professional book shows:

>>> format = "Hello, %s. %s enough for ya?"
>>> values = ('world', 'Hot')
>>> print format % values
Hello, world. Hot enough for ya?

Templates strings are another way of formatting. Basically these are strings with variables that are substituted with the actual value. Below are two examples from the book:

>>> from string import Template
>>> s = Template('$x, glorious $x!')
>>> s.substitute(x='slurm')
'slurm, glorious slurm!'

>>> s = Template("It's ${x}tastic!")
>>> s.substitute(x='slurm')
"It's slurmtastic!"

A third example uses a dictionary for substitution with value-name pairs:

>>> s = Template('A $thing must never $action.')
>>> d = {}
>>> d['thing'] = 'gentleman'
>>> d['action'] = 'show his socks'
>>> s.substitute(d)
'A gentleman must never show his socks.'

Creating a Drupal module

2007-07-04T08:42:00.000+08:00

First, went to the module developer's guide page and did the tutorial for Drupal 5.x. The block module in the tutorial, when completed, is full of code that integrates the module into the Drupal CMS. We don't need much of that code for now.

We want a quick and dirty email notifier module. First we create the info file that contains information which the CMS will display. The module's name is Messenger so the file will be messenger.info:

; $Id$
name = Messenger
description = "Sends out an email whenever new content has been created."

Then we have the module file, messenger.module. We add the hook_help and hook_perm functions from the tutorial. Through the Hooks API, we use the hook_nodeapi function which will run itself whenever there is node activity. This function is where we put in the mail command:

function messenger_nodeapi(&$node, $op, $a3 = NULL, $a4 = NULL)
{
 switch($op)
 {
   case 'insert':
     $query = "SELECT mail FROM users ".
              "WHERE name = 'admin'";
     $queryResult =  db_query($query);

     $user = db_fetch_object($queryResult);
   
     $to = $user->mail;
     $subject = "New content available!";
     $body = "Someone's been adding new content! To ensure it's nothing illegal, please go check it out.";
     $from = "drupalcares@whocares.com";
     $headers = array();

     $mail_success = drupal_mail('messenger', $to, $subject, $body, $from);
     if(!$mail_success)
     {
       drupal_set_message(t('There has been an error.\\nTo: '.$to.'\/nSubject: '.$subject.'\rBody: '.$body.'\nFrom: '.$from.'\nHeaders: '.$headers));
     }
     else
     {
       drupal_set_message(t('A mail has been sent to the admin. Prepare yourself! :)'));
     }
     drupal_set_message(t('Yay, you just inserted something! :)'));
     break;
 }
}

One thing to check is whether sendmail is available on the Ubuntu system. If sendmail is not working then the module will not work.

The module is functional now. Further improvements can be done; sending the mail to more than one admin or even to users, restricting the notification to just pages or blog entries and making it configurable through the CMS.

Drupal Modules

2007-06-26T13:02:00.000+08:00

Drupal requires a few modules before it can be truly functioning to one's expectation. The three sites listed below are more than enough for the beginner. As it is, some of the modules are only relevant if the website to be built is of a particular type - community, commercial, blog, etc. That said, the one module that must be installed would be the WYSIWYG editor. The default Drupal content creation way is to type in the page wholesale with all the markup which is a pain!

Modules:
Top 10 Drupal modules
10 Drupal Modules You Can't Live Without
Drupal Podcast No. 40: Top 40 Projects

WYSIWYG Editor:
TinyMCE WYSIWYG Editor
widgEditor - A WYSIWYG editor
FCKeditor

PHP HTTP Authentication using database with PEAR

2007-06-22T09:39:00.000+08:00

With Ubuntu 7.04 (Feisty Fawn) and a Synaptic-installation of php5, you'll have to install the Auth_HTTP package as in the book and then the DB package.

To obtain data read from the database, must use the getAuthData("column_name") method on the Auth_HTTP object.

PHP HTTP Authentication

2007-06-21T09:17:00.000+08:00

The HTTP Authentication for PHP comes in two versions, “Basic” and “Digest”.

Basic sends the username and password across the net in plain text.
Digest uses some clever one-way encryption techniques (eg. MD5 or SHA1) to prove that both client and server are exactly who they say they are without actually sending the password at all (in fact the server doesn't even need to store the password itself).

A problem with HTTP Authentication is that there is no way to easily logout, in fact there is no support for logging out. The browser can access the restricted material until such time that the cached credentials are flushed. This usually happens when the browser is closed.

Beginning PHP and MySQL 5 - Chapters 1-7

2007-06-15T09:14:00.000+08:00

Ch 1 - An Introduction to PHP
Basically the history of PHP and why you should use it.

Ch 2 - Installing and Configuring Apache and PHP
Useful in getting started up. There are a lot of configuration options listed out. Using default values so skipped over it for now.

Ch 3 - PHP Basics
Learned the syntax that are available, quite a few including a short form version. Went through comments, outputting to html, datatypes, variables and constants, expessions and control structures like if, if-else and switch.

Ch 4 - Functions
All about functions, the prototype for a function is similar to a method:

function function_name(parameters)
{
   function-body
}

There is no return type required. If needed, it can be added to the body.

Variable functions are functions which are evaluated at execution time to retrieve their names:

$function()

Given a URL parameter, this allows one to bypass the tedious if statement in order to know which function to call:

if($trigger == "retrieveUser")
{
   retrieveUser($rowid);
}
else if($trigger == "retrieveNews")
{
   retrieveNews($rowid);
}
else if($trigger == "retrieveWeather")
{
   retrieveWeather($rowid);
}

to the much more compact:

$trigger($rowid);

There are security risks in using this so care must be taken.

Reuseable functions can be stored in a function library and called up by:

include("function.library.php");

Ch 5 - Arrays
Of note is the natural sorting function, natsort(), which sorts (10, 1, 20, 2) to (1, 2, 10, 20) instead of (1, 10, 2, 20).

Ch 6 - Object-Oriented PHP
Normal OO stuff that can be found in C or Java. The Properties feature in PHP can be used to extend the class by adding in new properties. A potential gotcha for constructors is that they do not call the parent constructor automatically.

Ch 7 - Advanced OOP Features
Again more of the same. However there are OOP features that are not supported such constructor, method and operator overloading. There is a section on reflection, showing how to obtain the class type, methods and parameters.

Initial thoughts on PHP

2007-06-13T13:21:00.000+08:00

Learning PHP through a book entitled Beginning PHP and MySQL 5: From Novice to Professional, Second Edition by W. Jason Gilmore.

PHP is a loosely-typed language due to its origin as a web counter script. This is inherent in the quick and dirty way it does things, one can put in a variable and assign and reassign it to various data types like strings, integers and floats. There is no need for declarations and type-casting, though it is available. Coming from a Java background, that is very off-putting. Acclimation would be easy but care must be taken to prevent any bad programming habits from creeping in.

The OOP aspect of PHP is a mix of C and Java coding styles. In that respect, it is pretty conventional and a Java programmer can take easily to it. Relatively simple websites should be easy to prototype quickly.

Installing CruiseControl - a continuous integration framework

2007-06-07T13:59:00.000+08:00

The first step in installing CruiseControl (CC) is going to the SourceForge site and downloading the distribution. There are two flavors, a binary and a source distribution. You should choose the binary distribution as it is a trimmed version of the source distribution and is much simpler to use. After the download, extract the file to where you want to place it.

From here, you will want to follow the getting started guides for the binary and source distributions in order to set up CruiseControl. The binary distribution lets you run it 'out of the box' and you can see how a functional CC is supposed to work. Once you have the hang of it, you can try out putting your own project under CC. Using the guide for the source distribution, under the "Running the Build Loop" section, you can specify a place to store all your work.

Now the binary distribution sets the root of the CC directory to where the sh/bat executive file is called from. The requirement is that CC's lib and dist directories must be there. So the default is to call cruisecontrol.sh from the extraction place, that will also bind the project storage to be there. If you call the sh file from the specified place in the previous paragraph, CC will complain about not finding the lib and dist directories. The solution is to edit cruisecontrol.sh and hardcode the CCDIR to the extraction place, then call it from the project storage. Note that the binary distribution uses a projects directory instead of a checkout directory.

After that, follow the guide in creating config.xml and build-.xml, making changes as necessary. CC makes use of the Ant build tool so all the project compiling, testing and archiving are done using that. This site tells you how to make use of SVN in Ant instead of CVS.

Links:

Continuous Integration

2007-06-06T10:22:00.000+08:00

Continuous Integration (CI) is a software development practice of having team members integrate their work into the project frequently. Every integration is automatically built and tested so integration errors can be flushed out.

From Martin Fowler's article on CI, the practices to follow are:

Maintain a Single Source Repository
Automate the Build
Make Your Build Self-Testing
Everyone Commits Every Day
Every Commit Should Build the Mainline on an Integration Machine
Keep the Build Fast
Test in a Clone of the Production Environment
Make it Easy for Anyone to Get the Latest Executable
Everyone can see what's happening
Automate Deployment

Current practice at the moment only make use of steps 1 and 2. With step 2, though an Ant build script is present, the Eclipse IDE's automatic build is favored.

Step 3 - Tests are coded but currently they are run by hand, not in conjunction with the build. Step 4 - This is not adhered to, usually committing is done when a large task has been completed.

Steps 5, 6 and 7 are not relevant with the current practice.

Step 8 - Current practice is to ask developer for the latest code, use the source repository to transfer the code then build it. If the code is transferred to a new machine, custom settings have to be supplied before it can be built. Generally a lot of hassle and potential pitfalls.

Step 9 - Only the developer knows what's happening, anyone else has to be informed by that person.

Step 10 - The Ant build script has a deployment task. However that task only deals with production deployment which is set to a particular server and is not suitable for testing.

CI is supposed to lessen integration impact and allow for better communication between team members who are working on a project. Being a development team of one, there are no major issues. Anything that occurs is the result of that single developer and must subsequently be resolved by that person. However that will change as the team grows.

Wicket (vs. Tapestry)

2007-04-25T15:33:00.000+08:00

Wicket, as with Tapestry, is a view technology that uses plain HTML and Java code to render web sites. However where with Tapestry you have the controlling logic split between the Java code and html/page files, Wicket thrusts all logic into the Java code and use the html pages only to define the position of its components.

In addition, there is no xml configuration needed aside from the inclusion of the Wicket servlet into the web application. Any configuration is done in Java code. As such the components in Wicket can be reused and also inherit from other components, normal Java behaviour.

After trying Wicket for a few days, it is very much easier to use than Tapestry. The split logic in Tapestry forces one to constantly refer to the html pages for what components are used and their settings and the Java for the logic behaviour. Keeping all the logic in Java enables one to have a clearer picture on what the components are and what they do, and the html pages only need their wicket ids to function.

Another thing is the strict enforcement of wicket ids referencing their respective components. If one has a wicket id that is not referencing a component somewhere in the Java code, the webapp cannot render it and gives an error page. Similarly for a component that tries to add itself to a page without an id.

Shifting from Tapestry, one had to change their coding mindset from a single Java file for a html page. Wicket also requires that, however a base page can be defined and, taking advantage of inheritance, be extended by other pages. Components in one page can be reused in another and so on.

Maven

2007-04-10T16:31:00.000+08:00

http://maven.apache.org/

Maven can be run straight out of the archive, once the systempath variables are set. Following the 5 minutes guide, I created a 'Hello World' app, compiled, tested and finally packaged it into a jar. Going further, using 'mvn eclipse:eclipse' allowed it to be imported into the Eclipse IDE. There is also a plugin available for tight integration between Maven and Eclipse.

Dependency management is one prime reason for using Maven. The dependencies are downloaded from a remote repository and stored in a local repository that Maven creates. So for several projects, there will be only be one dependency copy. The default remote repository can be changed to an internal repository that will house a private copy of dependencies.

In regards to Ant, Maven makes for a more standardized build process, with little or no configuration needed to start creating, building and testing projects. It provides a common directory layout for each Maven project, easing orientation.

Repository Pattern in lieu of DAO Pattern

2007-02-21T08:51:00.000+08:00

Domain Driven Design - Inject Repositories, not DAOs in Domain Entities

Instead of the usual domain-DAO-database, a Repository is added so that it becomes domain-Repository-DAO-database.

The Repository contains domain-centric methods and uses the DAO to interact with the database.

Domain Driven Design : Use ORM backed Repository for Transparent Data Access

With Hibernate or JPA taking care of persistence and simplifying the DAO, the Repository can interact directly with the ORM and drop the DAO. So it becomes domain-Repository-Hibernate/JPA.

The link gives a general implementation of the Repository, which adopts a Bridge pattern to separate the abstraction (which will be exposed to the client code) and the implementation (which can swapped between Hibernate or JPA). It shows how the client Repository can extend from the general implementation.

The design is attractive, with the general implementation available to be used in any project. However it requires more thought to use it and and the unfamiliarity requires that more groundwork needs to be done before one can be proficient in it.

Ubuntu 6.10, Edgy Eft

2007-02-20T16:27:00.000+08:00

Downloaded and installed the latest version of Ubuntu, 6.10 - Edgy Eft. The ISO is a combination Live CD and installation CD, it loads up the Live CD first and offers an option to install.

Installation is pretty much the same as the procedure listed in the Apress book, "Beginning Ubuntu Linux". After installation, there were a lot of updates to download and install.

Ubuntu comes internet-ready, provided one has an active connection. Firefox is available and browsing can be done after installation.

Playing media files, music and video, however means downloading and installing codecs. The Apress book has a section on it, also there is a unofficial Wiki guide that lists a lot of how-to stuff, including codec installation. Following the guide requires one to be familiar with the command prompt though.

NTFS drives can be mounted to the filesystem in a read-only capacity. One would need to use FAT drives to share data between Windows and Linux.

The GUI offers more than enough configuration managers that the command prompt can be ignored for the basic user.

Spring-JPA-Tomcat

2007-02-15T12:50:00.000+08:00

Created an Eclipse project that follows the 'Introduction to Spring 2 and JPA' pdf. This entails creating POJOs, the business objects of the application, which in this case are the Employee class and the Address class. The POJOs do not contain any business logic, possessing only their data fields and the associated getters and setters. Two constructors are added, a no-arg constructor and another constructor that sets all the fields with data.

The POJOUnitTest class, a Junit test case, is coded to unit test the Employee and Address classes. Their objects are created, inserted with data (via setter methods and also constructor) and then the data is verified in the test.

A service layer containing the business logic to use these POJOs is visualized in the EmployeeService interface, which contains abstract methods for dealing with the Employee class.

Up to now, it's been plain Java coding. Spring and JPA comes into play when one needs to add database support. JPA annotations are added to the POJOs, at the class and field names, to establish the mapping between objects and database counterparts. Spring's JPA support enables one to use its DAO API to code the implementation class which extends JpaDaoSupport and implements the EmployeeService interface, called EmployeeDAO.

Now we use Spring to string these POJOs up as beans in dwspring-service.xml. Here the provider is changed from TopLink to Hibernate and a database change to MySQL by modifying the entityManagerFactory bean's properties as shown below:
From:

property name="jpaVendorAdapter"
bean class="org.springframework.orm.jpa.vendor.TopLinkJpaVendorAdapter"
property name="databasePlatform"
value="oracle.toplink.essentials.platform.database.HSQLPlatform"

To:

property name="jpaVendorAdapter"
   bean class="org.springframework.orm.jpa.vendor.HibernateJpaVendorAdapter"
property name="databasePlatform"
   value="org.hibernate.dialect.MySQLDialect"

Needless to say, the properties of the dataSource bean (driverClassName, url, username, password) need to be changed to reflect the use of the MySQL database.

Another test, an integration test this time, verifies that the EmployeeDAO implementation of EmployeeService works against an actual database. The classname is EmployeeServiceImplementationTest and it extends AbstractJpaTests. This allows implementation of the getConfigLocations() method, where one or more beans configuration files can be parsed by the Spring engine. This enables automatic dependency injection; when Spring loads the EmployeeServiceImplementationTest class, it will discover an unfulfilled dependency - a property of type EmployeeService. The engine looks through the dwspring2-service.xml file for a configured bean of type EmployeeService and injects it via the setEmployeeService() method. The relevant code is below:

public class EmployeeServiceIntegrationTest extends AbstractJpaTests {
   private EmployeeService employeeService;
   private long JoeSmithId = 99999;
   public void setEmployeeService(EmployeeService employeeService) {
       this.employeeService = employeeService;
   }
   protected String[] getConfigLocations() {
       return new String[] {"classpath:/com/ibm/dw/spring2/dwspring2-service.xml"};
}

After that, in order to run the test, one needs to include all the dependency JARs from Spring and Hibernate libraries, which includes the MySQL driver file. A persistence.xml file is required by the JPA specification. It describes a persistence unit, though in this case, it is only there to satisfy specs. The file is placed in the META-INF folder and must be accessible through the classpath. The code is below:

persistence-unit name="dwSpring2Jpa" type="RESOURCE_LOCAL"

Now the web application comes into play. The UI for this is Spring MVC, and the first controller class created is MainController, which handles the initial incoming request. It obtains a list of all the employees in the system. Next the dwspring2-servlet.xml (dwspring2 being the name of the DispatcherServlet) file is configured with all the beans required by Spring MVC. The web.xml file contains the configuration of the DispatcherServlet and is located in the WEB-INF folder. MainController does not display the data it holds, instead it hands the data over to a view which resolves to a jsp file, home.jsp located in the jsp folder. home.jsp displays the id and full name of each employee, it also made links out of the ids. When clicked, the links go to the controller EmpDetailsController which gets all details about a particular employee (in other words, get the employee object). A command class, EmployeeDetailsCommand, is used to parse the link arguments into an object. Here, the only arguments being passed is the employee id, so EmployeeDetailsCommand has only one data field and a getter and setter.

EmpDetailsController passes the employee object it holds to the empdetails view which as before is resolved by the InternalResourceViewResolver by appending the appropriate prefix and suffix and returns /jsp/empdetails.jsp as the view handler. empdetails.jsp displays the employee's details.

home.jsp and empdetails.jsp use the css/dwstyles.css stylesheet to format their HTML. This only affect appearance.

The next step is the Eclipse WTP process, however it has been replaced with an Ant build. This step will compile the code, build a deployable WAR file (for deployment in a J2EE compatible Web tier container) and deploy that file to a Tomcat server. The WAR file consists of the compiled classes, the dependency library jars, the web content and the configuration files (context.xml, persistence.xml, dwspring2-servlet.xml, dwspring-service.xml, web.xml)

Before the WAR file can be deployed, a number of things had to be done. When a Spring JPA application runs on Tomcat, bytecode "weaving" during class loading is required for the JPA support to work properly. The standard Tomcat classloader does not support this. A Spring-specific classloader is needed.

Installing the Spring classloader:
1) Copy spring-tomcat-weaver.jar into the Tomcat's server/lib subdirectory. The spring-tomcat-weaver.jar library can be found in the dist/weaver of the Spring distribution.
2) Configure the context.xml file (located in META-INF) to let Tomcat know to replace the standard classloader for this particular web application.

Spring needed to hook into Tomcat's context loading pipeline so a ContextLoaderListener was added to web.xml.

Datasources are managed by Tomcat and are available through a standard Java Naming and Directory Interface (JNDI) lookup mechanism. The employee system runs as a Web application inside Tomcat and should obtain its datasource through Tomcat's JNDI. To accomplish this, the MySQL driver needs to be copied to Tomcat's common/lib subdirectory. Then configure the context.xml file, adding a JNDI resource. With a resource name of "jdbc/dwspring2", the configuration makes the JNDI datasource available through the name java:comp/env/jdbc/dwspring2. Next, add a resource reference in web.xml, making it available for use within the web application. Finally, dwspring2-service.xml must be modified to use the JNDI datasource.

The Spring engine needs to locate and process the bean configuration file (dwspring2-service.xml) for the POJOs in order to wire them up with the web application. The context parameter in web.xml must be configured with the location of the file.

The final step before deployment is to fill the database with data. The FillTableWithEmployeeInfo class was coded to do this. This class extends AbstractJpaTests. A great feature of tests based on AbstractJpaTests is that all database changes are rolled back upon completion of a test, allowing the next test to run quickly. However calling the setComplete() method within the test commits the transaction instead of rolling it back and makes the changes permanent. Which is what is done with FillTableWithEmployeeInfo.

Finally the WAR file (spring2web.war) can be deployed to Tomcat's webapps subdirectory and loaded. However the deployment was unsuccessful.

From Tomcat's Catalina log, it was a puzzling one-liner error about the context listener. Set up the web application logging by adding log4j.properties in order to find out what was wrong. The error was more verbose, essentially a ClassDefNotFoundException of a class in jasper-compiler.jar, which is located in Tomcat's common/lib subdirectory. Adding that file to the web application library only made it a ClassCastException. The project was stuck for a while.

Got hold of the springjpa project from MemeStorm, and tried to deploy that as well. Whilst doing that, changed the loadTimeWeaver property in dwspring2-service.xml from SimpleLoadTimeWeaver to InstrumentationLoadTimeWeaver to follow the springjpa project. Read that SimpleLoadTimeWeaver was only suited for testing, so for real deployment, InstrumentationLoadTimeWeaver or ReflectiveLoadTimeWeaver should be used. Whatever the problem was, it wasn't the loadTimeWeaver property as both applications still refused to deploy.

Eventually learned that it was the Spring class loader at fault. When Tomcat replaced the standard classloader with the Spring classloader, the Spring classloader does not have the classpath the standard classloader possess in order to access Tomcat's library jars. The solution was to add the attribute useSystemClassLoaderAsParent="false" to the Loader in context.xml. The error was hurdled but a new one popped up.

The new error was that no persistence units were parsed from META-INF/persistence.xml. This one was simple to solve, the META-INF/persistence.xml needed to be in the root of the class package. So a move to the web application's WEB-INF/classes and it is solved.

The next error was talking about a java agent and prior reading indicates this had to do with the InstrumentationLoadTimeWeaver, basically it needs a Spring agent to be loaded into the JVM. To accomplish this in Tomcat, the line set JAVA_OPTS=%JAVA_OPTS% -javaagent:"%CATALINA_BASE%\server\lib\spring-agent.jar" was inserted into catalina.bat in Tomcat's bin subdirectory. The spring-agent.jar mentioned in the line can be found in the dist/weaver of the Spring distribution. The file was copied to Tomcat's server/lib subdirectory.

Success! Deployment went without a hitch and the web application can be accessed, displaying the list of employees previously inserted into the database and their details when their link was clicked.

Messed around with the loadTimeWeaver property, changing it back to SimpleLoadTimeWeaver and even commenting it out as Hibernate apparently does not require it. The web application still runs fine.

One last change was the addition of index.jsp and include.jsp, which were from a previous project. A slight modification to the welcome file in web.xml and one can now access the web application via http://localhost:8080/spring2web without any need for a filename.

The file structure of the spring2web web application:


spring2web
|-css
|  |-dwstyles.css
|
|-jsp
| |-empdetails.jsp
| |-home.jsp
| |-include.jsp
|
|-META-INF
| |-context.xml
|
|-WEB-INF
| |-classes
| | |-com.ibm.dw.spring2.*
| | |-com.ibm.dw.spring2.web.*
| | |
| | |-META-INF
| | | |-persistence.xml
| | |
| | |-log4j.properties
| |
| |-lib
| | |-*.jar
| |
| |-dwspring2-service.xml
| |-dwspring2-servlet.xml
| |-web.xml
|
|-index.jsp

The steps taken in Spring-JPA-Tomcat:
- POJOs
- Unit Test
- dwspring2-service.xml
- Integration Test
- META-INF/persistence.xml
- Spring MVC
- dwspring2-servlet.xml
- web.xml
- META-INF/context.xml (not at the same location as persistence.xml)
- (Tomcat)/common/lib <-- mysql-connector-java-5.0.4-bin.jar
- (Tomcat) /server/lib <-- spring-tomcat-weaver.jar, spring-agent.jar
- (Tomcat)/bin/catalina.bat <-- Java agent command

JSP Front End Testing

2007-02-12T10:33:00.000+08:00

With the JSP front end, all correct operations dealing with valid employee ids will work.

As there is no error correction and no validation, there are a lot of bad cases:
Insert: Anything can be inserted. An employee id with null for every field is possible.

Update: With a valid id, all the fields can be updated with junk data, even nulled. Without a valid id, the server creates an exception.

Delete: Valid ids will be deleted, invalid ones will create an exception report.

Find: Valid ids will be displayed, invalid ones will display a blank page.

Spring in Action

2007-01-29T13:12:00.000+08:00

Inversion of Control

Inversion of Control (IoC) is the reversal of responsibility with regard to how an object obtains references to other objects. Normally, each object is responsible for obtaining its own references to its dependencies. With IoC, objects are given their dependencies at creation time by an external entity which handles all the objects in the system.

Dependency Injection

Dependency Injection is merely a more apt name for IoC, given that dependencies are injected into objects. There are 3 types of IoC:

1) Interface Dependent - Dependencies are managed by implementing special interfaces.
2) Setter Injection - Dependencies and properties are configured via the setter methods.
3) Constructor Injection - Dependencies and properties are configured via the constructor.

IoC allow the Java objects to be loosely coupled, interacting through interfaces. It allows the programmer to set up and configure the objects as desired, while leaving little trace of it in the code itself.

Application design for Spring would be based on interfaces. Overall, the code would be normal POJO, until arriving at the Spring setup; a class using all the objects coded and a Spring configuration file, usually XML.

Native Hibernate vs. Hibernate JPA

2007-01-18T18:06:00.000+08:00

Native Hibernate uses only the Hibernate Core for all its functions. The code for a class that will be saved to the database is displayed below:

package hello;
public class Message {
private Long id;
private String text;
private Message nextMessage;
// Constructors, getters, setters...
}

As can be seen, it is merely a Plain Old Java Object (POJO). The relational mapping that links the object to the database table is in an XML mapping document. The actual code that will create and save the object is below:

package hello;
import java.util.*;
import org.hibernate.*;
import persistence.*;
public class HelloWorld {
public static void main(String[] args) {
// First unit of work
Session session = HibernateUtil.getSessionFactory().openSession();
Transaction tx = session.beginTransaction();
Message message = new Message("Hello World");
Long msgId = (Long) session.save(message);
tx.commit();
session.close();
// Shutting down the application
HibernateUtil.shutdown();
}
}

Session, Transaction and Query (not shown) objects are available due to the org.hibernate import. They allow a higher-level handling of database tasks than DAO using JDBC.

Hibernate JPA is accessed through the use of Hibernate EntityManager and Hibernate Annotations. The Hibernate EntityManager is merely a wrapper around Hibernate Core, providing and supporting JPA functionality. Thus the change in the code can be seen below:

package hello;
import javax.persistence.*;
@Entity
@Table(name = "MESSAGES")
public class Message {
@Id @GeneratedValue
@Column(name = "MESSAGE_ID")
private Long id;
@Column(name = "MESSAGE_TEXT")
private String text;
@ManyToOne(cascade = CascadeType.ALL)
@JoinColumn(name = "NEXT_MESSAGE_ID")
private Message nextMessage;

The XML document with all the relational data has been removed and replaced with inline annotations, which is provided by the javax.persistence import. The only difference between the Hibernate POJO and JPA POJO is the annotations. The code itself will run fine, the annotations make it a persistent entity but will do nothing unless Hibernate goes through it. JPA can glean enough information from them for the ORM and persistence tasks. The HelloWorld code:

package hello;
import java.util.*;
import javax.persistence.*;
public class HelloWorld {
public static void main(String[] args) {
// Start EntityManagerFactory
EntityManagerFactory emf =
Persistence.createEntityManagerFactory("helloworld");
// First unit of work
EntityManager em = emf.createEntityManager();
EntityTransaction tx = em.getTransaction();
tx.begin();
Message message = new Message("Hello World");
em.persist(message);
tx.commit();
em.close();
// Shutting down the application
emf.close();
}
}

The Hibernate import is gone, replace by javax.persistence. The EntityManagerFactory, EntityManager and EntityTransaction run the database tasks.

Both API seem similar and choosing one over the other is a matter of preference. Native Hibernate is the cleaner one, with the relational data put into an XML document. Hibernate JPA is standardised with Java and can be ported easily.

Other JPA implementations:
Open-source:
GlassFish
Apache OpenJPA

Commercial:
SAP NetWeaver
Oracle TopLink
BEA Kodo

HIbernate

2007-01-18T10:15:00.000+08:00

Hibernate is an open-source project that handles the role of the persistence layer, becoming the middleman between the business logic code and the database data. Its expressed purpose is to free developers from the tedious and common coding of database tasks such as queries, insertions and deletions.

Take the previous DAOExercise as an example, where most of the code dealt with inserting, selecting and deleting data from the MySQL database. Hibernate would handle these mundane tasks and allow the developer to focus more on the business logic and rare SQL code exceptions.

HIbernate incorporates ORM (object/relational mapping), which will map objects to their proper tables in the database. Such logic would be used when hand-coding DAOs using raw JDBC, and it would be up to the developer to track and maintain any changes in either object or database table. Use of Hibernate simplifies matters and make maintenance easier.

Object Pool Pattern

2007-01-17T11:48:00.000+08:00

The Object Pool pattern dictates having an object (usually a singleton) maintain a pool of reusable objects that can be checked out and in by clients who will use them. Connections to databases are the perfect candidate for this pattern.

The ConnectionFactory (CF) in the DAOExercise can be the connection-pool manager with the clients being the DAO classes. A DAO class request a connection from the CF for a query. The CF checks its pool; if there is no objects in the pool, it creates one and sends it to the DAO class, else it will pop out the object and return it. Once the DAO class is done with the query, it sends the connection back to CF and the CF puts it into the pool. The pool may have a maximum number of objects, whereby if all objects have been checked out and the CF tries to create one, it cannot and must wait for the return of an object to honour the request by the DAO class.

The Object Pool pattern benefit designs with a resource creation that is expensive, limited in number or slow that must be shared out to clients utilizing that resource. Real-world examples of the design are car rentals and timesharing.

Factory Method Pattern vs. Abstract Factory Pattern

2007-01-17T09:02:00.000+08:00

The Factory Method pattern is to define an interface for creating an object but let subclasses decide the class to instantiate. For example the UIBuilder class contains two method stubs. The subclasses (EnglishUIBuilder, MalayUIBuilder) have to implement these stubs in their own way.

The Abstract Factory pattern is to provide an interface for creating families of related or dependent objects without specifying their concrete classes. This pattern is often used in conjunction with the Factory Method pattern, thus it can be seen as a factory of factory objects.

The UIFactory static method returns a subclass of UIFactory which in turn creates the appropriate UIBuilder object. The UIFactory subclass returned is determined by reading a config file and using the default value.

MySQL Class

2007-01-16T08:36:00.000+08:00

Prior to the addition of the ConnectionFactory, connection to the database was handled by the MySQL class. The class was to return a connection to any caller via a getter method.

The connection was made static in the class, letting only one instance to be created. All the methods were static, thus the class need not be instantiated. With that, a static initializer was added to run once the class was loaded, establishing the connection.

For testing purposes, one connection is enough. In a live environment, the one connection that MySQL has would be swamped with requests. A pool of connections would be best for that sort. So at the time, the choice for a single connection was appropriate however it would not be scaleable outside test conditions.

The static initializer was to initialize and establish the connection. This was done when the class is loaded, in other words when a static method is called. That will be the getter method being called. Initialization can take place inside the getter or another method, however since theus connection is static and only needs to be established once, the static initializer is used.

When one has an open connection, one should provide a way of closing it. That was the closeConnection method. Ideally, once the connection has been used by a class, the class should close it. Going through DAOExercise; discovered that once closed, a connection cannot be reopened. Thereafter the only use for it was when all the tests were done in the AllTests class.

In retrospect, using the static initializer was not a good idea. It would be better to stick it in the getter method:

public static Connection getConnection()
{
if(conn.isClosed() || conn == null)
{establishConnection();}
return conn ;
}

This way, closeConnection can be used.

Addendum: Static reference creates only one instance for the class; all class objects share that static reference. As MySQL is never instantiated, that point is moot. Not so for the DAOExercise objects, two EmployeeDAOImpl objects will share the static connection.

Abstract Factory Pattern for DAOExercise

2007-01-15T13:46:00.000+08:00

The Abstract Factory Pattern allows for multiple factories which share a common theme to be streamlined into one class. In DAOExercise, this class is the ConnectionFactory which will create the appropriate factory for use. This class currently has a concrete implementation for MySQLConnectionFactory and can be further extended (OracleConnectionFactory, OCBCConnectionFactory, etc).

This implementation allows the underlying database (MySQL) to be divorced from the actual DAOExercise (Employee, Address, Dependent) so any database change (e.g Switch from MySQL to Oracle) can be coded and inserted into the ConnectionFactory, touching very little of the code for DAOExercise.

Another way of looking at ConnectionFactory is that it is a factory of factories. The ConnectionFactory determines which factory is to be handed over to the client code via a string which must be set by the client code. The MySQLConnectionFactory will create and return a connection to the MySQL database. This outlook can be confusing when all the client code sees is the ConnectionFactory reference and not the actual factory object.

Using this pattern for DAO creation (a DAOFactory generating EmployeeDAOFactory, AddressDAOFactory and DependentDAOFactory) is impossible with the current design as the implementing classes are not related to each other. The pattern can get around this with the use of the Adapter pattern but that would still require a major rewrite of the design. At best DAO creation is served by a single factory (DAOFactory) which will generate all three classes. Also unlike the ConnectionFactory, where the objects need different initializing data, the DAOFactory objects are self-contained.

As the purpose of a factory is to generate objects for use, a single instance of it would suffice. Coding it so that it complies with the Singleton pattern would enforce this single instance. However, the pattern is not a requirement. Having multiple factories would not be a problem, save for efficiency and design.

In MySQLConnectionFactory, there is a Properties variable which is used to read a file containing all database-specific information. Aside from the connection data, it stores all the SQL queries for that database. These queries are used in the DAO classes. It is not possible to create DAO objects with the file.

DAOExercise Architecture

2007-01-11T15:47:00.000+08:00

Having implemented and written DAO code that accesses a MySQL
database, there need to be changes if that code is to access an
Oracle database. Though both databases use SQL, code that works
for one may be broken or have unexpected results in the other.
So the SQL code needs to be tested and rewritten as necessary.
The driver class needs to be rewritten with the proper commands
and authentication so as to get the right connection to the Oracle
database.

Whenever the database changes, the Java code has to be
rewritten to accommodate it. This is due to the hardcoding
of the SQL and driver information. The only way to not rewrite
all that is to throw it to a go-between which interacts with
the database and leaves the Java code handling objects only.

SQL Injection

2007-01-09T09:57:00.000+08:00

SQL Prepared Statements are apparently not subject to injection attacks. The precompiled code will view the wildcard parameters as data only. Attempts to subvert the code proved futile, with no change to the database. Proper arguments work and the code executes.

JDBC Basics

2007-01-08T15:29:00.000+08:00

Learning how to use SQL via the Eclipse Java IDE and MySQL. Had problems connecting to the MySQL databases until I remembered to start up the service (>_<).

The most crucial part was getting the connection through the DriverManager class and the settings for it. After that was the creation of tables and filling it with data. The tutorial at http://java.sun.com/docs/books/tutorial/jdbc/basics/tables.html was vague on that part and I had to go look elsewhere to do it.

Used SELECT to print out the table values with the help of the ResultSet and Statement classes. Then looked into updating the data via Java methods instead of normal SQL commands.

Prepared Statements are Statements given an SQL command at creation time. With wildcard parameters in the command, one can use it repeatedly, changing the parameters at will. Looked at the joining of two tables.

The last was transactions, how to commit several statements as an atomic action. The Savepoint methods allowed part of the transaction to survive a rollback.

One thing to keep in mind when building strings SQL commands is the spacing.

SCJP Exam

2007-01-08T08:33:00.000+08:00

4th January 2007, the day of the SCJP exam, scheduled for 10:00AM. Arrived at the testing centre, was told to leave all my things in a locker. That included the pens I brought, prompting the question - will I be provided pen and paper? Apparently I won't, they have marker pens and thin mousepad-sized writing-board sheets (1 pen, 2 sheets and a duster per tester). That was a let-down. Though it's understandable why they do so, I would be more comfortable with the old pen-and-paper standard. Using a marker pen to record one's answer is somewhat irritating.

The questions were much simpler than expected, after the torture of going through the twisted self-test questions and mock exams. Even knowing that practice questions were harder than the real ones, I was still startled. 2 hours and 55 minutes is enough time to make an initial leisurely pass answering the 72 questions, bypassing over those with long convoluted code - sure to take some time to understand, a 2nd pass to wrap up unanswered questions and a quick 3rd pass to go through all the questions, minus the drag-and-drop ones. Spending time to record the answers for the drag-and-drop questions is time that could have been better spent elsewhere. Reanswering those drag-and-drop questions is definitely a pain, since the answers are cleared when you want a second look at the questions.

Overall, I think the preparation done for the SCJP exam was sufficient, as most of the questions I answered with confidence. Only with one question did I have really some doubt about what would happen (Oh, for a compiler at that time). So though I entered the exam with trepidation, I ended it with a very confident outlook. Of course, getting 13 questions wrong knocked me down with a good dose of humility, though a pass of 81% is not bad.

Looking over the breakdown of the score, I did pretty good in Declarations, Initialization and Scoping, Collections/Generics, Fundamentals and got perfect marks for Flow Control. The areas where I was lacking were API Contents, Concurrency and OO Concepts. I was surprised about Concurrency, admittedly it is a complex topic but I did not struggle with any questions regarding it except for one which looked like a deadlock situation. API Contents and OO Concepts were no surprise to me, the mocks that did breakdown listed them as problem areas. However I elected to focus on Generics and Collections, feeling I had a shaky understanding of them, and it paid off.

What I got from the entire affair? Aside from the SCJP certification, which I am not sure would be that useful, the one-month-plus training established a solid grounding in the standard Java language. It exposed me to the new features of Java 1.4 and 1.5. I'm not happy with Generics, thinking about it makes me feel like it is an abstract topic (in the literal, not Java, sense). I can use it for collections type-safety, though thinking about infesting wildcards, super and extends into classes and methods drives me off the deep end.

=^.^= At last I can bind the SCJP book with chains, weight it down with rocks and dump it in the deepest, murkiest river I can find, all the while dancing and cackling madly. =^.^=

SCJP Chapter 10

2006-12-22T13:31:00.000+08:00

Development
After the wringer that is threads, this is a light read.

javac and java
The commands javac and java are used to compile and run java files. The command-line options that are available include -classpath (-cp), -d and -D. Multiple options and files must be separated by spaces.
The -classpath or -cp option are paths that will be searched for classes. The dot (.) indicates searching the current directory. Any subdirectories listed will be searched but not the directories above them. With packages, classpaths become trickier. The root of the package must be in a subdirectory that is in the classpath, making it visible. It is important to understand the relative and absolute paths. With JAR files, one must include the name at the end of the path.
The -d option tells the compiler where to put the class file that is generated. A compile error occurs if the destination directory does not exist. On the other hand, with a java file in a package, javac will build the package directories if none exist.
The -D option creates a system property with the syntax of name=value following immediately with no spaces in-between, eg -Dkey=property. Should the value have spaces, it should be enclosed with quotes. Java provides a java.util.Properties class to obtain information on the system as well as adding more user properties. Two methods, setProperty() and getProperty() facilitate setting and getting a property. setProperty() takes two strings for the name-value pair and getProperty() returns a string while either taking one string for the name value or two strings, a name value and a default value if the property does not exist (similar to setProperty()).
For java, exactly one class file is executed every time it is invoked, with no need for the extension. The argument declaration for the main method is no longer fixed to String[] args. The name of the string array can be changed and the var-args syntax may be used. String... x, String port[] and String[] ___ are all legal.

Static Imports
Static imports are a convenience-only feature that allows less typing. Beginning with the syntax: import static, which can only be used on static object references, constants (which are static and final) and static methods. Beware of ambiguously named static members, two classes may have a constant with the same name. If used, Java will not know which class to refer to.

SCJP Chapter 9

2006-12-20T15:02:00.000+08:00

Threads
There are two ways to define, create and run threads. One is to extend from the Thread class, overriding the default run() method with another implementation. The other is to implement the Runnable interface, which only contains one method run(), and put in the run() code. With a Runnable implementor, a Thread instance needs to be created with the implementor in the argument, allowing the code to be threaded.

States
When a thread is created but not started using the start() method, it is in the new state. Once started, it will graduate to the runnable state and enters the runnable pool where threads wait for the scheduler to pick them and run them. The run order of runnable threads is not guaranteed. What is guaranteed is that each thread will start and each thread will run to completion. The running state is what all threads aspire to. However, a thread may be pulled off by the waiting, blocking or sleeping states (If the scheduler does not pull it off first). In which case, the thread would still be alive but not eligible for running. Once a thread's run() method is done, the thread is dead. And once the thread has been started, it can never be started again (Runtime exception).

Priorities
Thread priorities are positive integers ranging from 1 to 10, though it is better to use the constants Thread.MIN_PRIORITY, Thread.NORM_PRIORITY and Thread.MAX_PRIORITY. The scheduler guarantees that the thread running will always have a greater or equal priority to the highest priority threads in the runnable pool.

Synchronization
Race condition is a problem where multiple threads vie for access to a single object and in the process, corrupt the state of the object, producing data which is of no use to any of the threads.

The solution to this is to lock methods or code blocks so that only one thread can access the object. Only code that can change the object needs be synchronized. Threads trying to get the object's lock, but discover it has already been taken, is said to be blocked on the object's lock. They go into a pool for that object and wait until the lock is released, whereby any of them can get it. Care has to be taken that deadlock does not occur. Deadlock is when two threads are blocked, with each waiting for the other's lock.

Thread Interaction
wait(), notify() and notifyAll() are methods that deal with objects' locking status. These methods can be called on any objects, within a synchronized context. A thread cannot call these methods on an object unless it owns that object's lock. The thread that calls the wait() method on an object goes into a waiting room, i.e. sleeps, until the notify()/notifyAll() method is invoked on the same object. Thus wait() and notify() must be used on the same object.

SCJP Chapter 8

2006-12-19T15:09:00.000+08:00

Inner Classes
There are four types of inner classes, your 'regular joe' inner class, method-local inner class, anonymous inner class and the static nested class (which is not really an inner class at all).

Regular Inner Class
Beginning with the regular inner class, which is coded alongside the outer class's variable and method members, one needs to understand that the inner class has access to all of the outer class members. Even private ones. The outer class needs to create an instance of the inner class before it can begin accessing the inner class members. Also when calling from another class, the outer class need to be instantiated before one can instantiate the inner class:

MyOuter.MyInner myinner = new MyOuter().new MyInner();

When in the inner class, the object self-reference is this. The outer class reference is NameofOuterClass.this. The same modifiers (final, abstract, public, private, protected, strictfp and static - though that changes it into a static nested class) that can be applied to instance variables and methods can also be applied to inner classes.

Method-Local Inner Class
An inner class defined within a method, it requires instantiation (below the class definition) inside the method to be available for use. No other method or class can create the method-local inner class aside from the method which holds that class. Like regular inner classes, it can access the outer class members, however it cannot access the local variables inside the method containing it unless these variables are marked final. Local variables exist on the stack and within the method lifetime only. Only two modifiers for the method-local inner class, abstract or final.

Anonymous Inner Class
Nameless inner classes with two versions, regular and argument-defined.

The regular version has two flavors, creating a subclass of any class type and creating a class implementing an interface. Subclassing is used to override the methods of a superclass, however creating a new method is legal though there is no way to invoke it short of a compilation error. The syntax for both flavors is unusual:

class Popcorn {
public void pop() {
System.out.println("popcorn");}
}

interface Cookable{
public void cook();
}

class Food {
Popcorn p = new Popcorn(){
public void pop(){
System.out.println("anonymous popcorn");} };

Cookable c = new Cookable(){
public void cook(){
System.out.println("anonymous cookable implementor");} };
}

Note the semicolon indicating the end of the statement. The semicolon is not present in argument-defined anonymous inner classes as the definition is located in the argument portion of the method call.

Static Nested Classes
A class that is simply a static member of the enclosing class. Meaning it can be accessed without instantiating the enclosing class. Naturally being static, it can only access other static variables and methods. Syntax for it:

MyOuter.StaticInner sinner = new MyOuter.StaticInner();

Or within the enclosing class, MyOuter:

StaticInner sinner = new StaticInner();

SCJP Chapter 7

2006-12-18T16:47:00.000+08:00

The last of the big chapters, Chapter 7 deals with collections and generics.

Collections
There are four interfaces (Lists, Sets, Maps and Queues) encompassing the eleven classes that implement them (ArrayList, Vector, LinkedList; HashSet, LinkedHashSet, TreeSet; Hashtable, LinkedHashMap, HashMap, TreeMap and PriorityQueue). Basically have to know what methods are available and how they are used, most of which is in the code done for the chapter.

One thing to note is the equals() and hashCode() methods and their contracts. The methods come from the Object class and thus is present in every class afterwards.
The default equals() consider two objects equal if the two references point to the same object, thus to get proper equality of objects (datawise), the equals() need to be overridden with the desired implementation. Otherwise, Sets will allow duplicates in and objects with the default cannot be used as keys in a hashtable. Read the contract (reflexive, symmetric, transitive, consistent and that for any non-null x x.equals(null) will return false) and watch out for questions on this and below.
With equals() overridden, hashCode() must be overridden as well. The default will allow duplicates in a Set. The contract for this one is a bit trickier:
For: x.equals(y) == true
Required: x.hashCode() == y.hashCode()

For: x.hashCode() != y.hashCode()
Required: x.equals(y) == false

For: x.hashCode() == y.hashCode()
Allowed but not required: x.equals(y)

For: x.equals(y) == false
Allowed but not required: No hashCode requirements

Generics
Generics enforce compile-time type safety on Collections (and classes and methods declared with generic type parameters). A generic type (eg. Animal) can accept subtypes (eg. Dog, Cat, Wolf). One advantage to type safety is no cast is required when the element type is known. Passing type-safe code into non-type code is dangerous, the compiler will compile it under duress (giving out a warning).

Polymorphism applies to base type, not generic type parameter:
Right: List aList = new ArrayList();
Wrong: List aList = new ArrayList();
The generic type must always be the same for declaration and initialization. Another problem is an ArrayList will not accept any collections of an Animal subtype. This requires further reading.

Wildcard ? allows any type to be assigned, but nothing can be added to the list referred to as List.
With the extends keyword, List ? extends Animal will be able to assign any subtype of List and typed for or anything that extends Animal (either subclasses or implementations), but still nothing can be added.
With the super keyword, List ? super Animal will be able to assign any subtype of List and typed for or any supertype of Dog, and allows a restricted way of adding to the collection.

One last thing to note is the use of generic typing in classes and methods, this allows type-checking to be built into the classes and methods, does away with the need to cast and the need to create several subclasses for different types.

SCJP Chapter 6

2006-12-13T13:15:00.000+08:00

Of strings, I/O, formatting and parsing.

Starting with Strings, which are immutable, there are also mutable StringBuffers and StringBuilders (added as of Java 5). The latter two are the same except that StringBuilders are faster because they are not synchronized (and therefore thread-unsafe). Important stuff to remember; know the methods of all three classes, Strings have length() method and arrays have length variable, know the differences between Strings and the other two, and beware chained methods.

For I/O, there are six classes to contend with; File, FileReader, BufferedReader, FileWriter, BufferedWriter and PrintWriter. Know about File's existential and creation problems. Know about wrapping classes up to enable higher-level usage and which ones are legal. Be aware that PrintWriter, as of Java 5, can now be used by wrapping a File or String.

Serialization is the saving of state of objects into a file for them to be read later (deserialization), reconstituting back the class objects. ObjectOutputStream and ObjectInputStream with their respective writeObject() and readObject() methods are the ones that allow the saving and loading of objects. Being higher-level classes, they need to be wrapped around the lower-level classes FileOutputStream and FileInputStream. A few things to note: all the classes to be serialized must either implement the Serializable interface or have a superclass that did. Any non-serializable object will cough up a runtime exception by the JVM. The transient keyword tells the serialization to skip a variable, not saving it. However, if there is need for that variable's state to be saved, one can override the writeObject() and readObject() to include it (the overriding code has to be in the class to be serialized). Constructors do not run for serialization, however a non-serializable superclass will run its constructor, potentially changing inherited variables. Lastly, serialization does not work for statics.

To work with dates, numbers and currencies, these classes are available: Date, Calendar and Locale from java.util.*, DateFormat and NumberFormat from java.text.*. Calendar, DateFormat and NumberFormat use factory methods to initialize. Knowing how to use all the classes is essential.

The last section of Chapter 6 deals with finding stuff (through regular expressions, of which there are two ways, pattern matching and searching using the Scanner class), tokenizing stuff (through String.split and using the Scanner class) and formatting stuff (Using printf and format, which are identical, C-style).

Chapter 7, the return of the Generics!

SCJP Chapter 5

2006-12-11T08:38:00.000+08:00

This chapter covers flow control; conditional if and switch statements, looping do-while, while and for statements, even exceptions and assertions.

What's there to know about the if statement? It encompasses the if, if-else and if-elseif constructs. Things to beware of: unbraced statements (only one is in the if construct), lost else clause (belongs to the innermost if statement), misleading indentation and illegal expressions (must resolve to a boolean).

switch statements can simulate multiple if statements. A switch's expression takes an enum or variables that can be implicitly promoted to an int (char, byte, short, int) and it can use boxing. The value for a case must be a compile time constant. Cases with the same value will not compile. Neither will a case with a value larger than the type variable in the switch. Without break statements, the code will run top down starting from the execution entry point (the first case constant matched). This dropping down is called "fall-through". default is executed when there is no match and its location does not matter; end, middle or top.

do-while and while loops are the same except for these differences: do-while must run at least once (while may not run at all) and construction is different (note do-while ends in a semicolon).

for loops has a second structure as of Java 5. The "enhanced for loop", a.k.a for-each and for-in, makes it easier to iterate through arrays and collections. The for-each loop has two components (declaration and expression) versus the three components of a for loop (initialization, condition and iteration).

break and continue statements are used to halt a loop, either the entire loop (break) or the current iteration (continue). Labels are added when nested loops are involved, they enable the break/continue statement to apply to the labelled loop, not necessarily the loop the statement is in.

Exceptions are handled by coding in try and catch blocks. A finally block can be added and will always run after the try block, regardless whether an exception is thrown or not. A try block cannot be used on its own, either a catch or a finally block must be present. An exception must find a match with a catch block or its supertype. A method must declare an exception if it throws (directly or indirectly) the exception otherwise it must handle the thrown exception. This is the "handle or declare" rule. Runtime exceptions are exempt from this rule. Need to know which exceptions are thrown by the Java Virtual Machine (JVM exceptions) and which are thrown explicitly by application programmers (Programmatic exceptions).

Assertions test assumptions during development. Their default mode is off and must be specifically turned on. If an assertion is true, the code runs smoothly. If an assertion is false, an AssertionError is thrown, shutting down the application. Assertions have two versions, both versions have an expression that results in a boolean value, the difference is a second expression to be added to the stack for debugging purposes. Inappropriate uses of assertions are validating public method arguments, validating command-line arguments and assert expressions that cause side effects. Appropriate uses of assertions are validating private method arguments and, even in public methods, checking for cases that are not supposed to happen.

SCJP Chapter 4

2006-12-07T13:15:00.000+08:00

A nice small focused chapter on operators. It starts with the assignment operator "=" and its compound kin (+=, -=, *=,/=). With the compound operators, the right side of the = will always be evaluated first, something to take note of.

Relational operators (<, <=, >, >=, == and !=) are relatively straightforward. Comparison of characters use their Unicode value. Equality (==) for objects only check whether variables point to the same object, not whether two objects are the same.

The instanceof operator is used with object reference variables and checks for type. It returns true if the variable IS-A particular class (or superclass) or interface (indirectly or directly implemented). It is legal to test null eg. null instanceof String, but it will always return false. It will not compile if testing a class with no relationship to the other class eg. Dog instanceof Cat. Last note: array IS-A Object.

Arithmetic operators (+, -, *, /, %) are simple to understand. The remainder operator (%) divides two operands and returns the remainder as the result. The + operator can be used to concatenate strings and care must be taken to verify which role it is in. If either one of the operands are strings then it concatenates, otherwise it is a normal addition operator.

Increment and decrement operators (++ and --) have two modes, prefix and postfix. Below shows a code example. These operators can not be used on final variables, doing so results in a compiler error.

int x = 1;

if(++x == 2) // prefix, x is incremented to 2 and
{ return true; }// the expression is evaluated,
// returning true

if(x++ == 2) // postfix, the expression is
{ return true; }// evaluated first, returning true
// then x is incremented to 3

The conditional operator is a ternary operator (with three operands). It is a miniature if statement. The structure is as below.

x = (boolean expression) ? value if true : value if false

Logical operators (&, |, ^, !, &&, ||) are the last topic in the chapter. Short-circuit operators (&& and ||) evaluate up to the operand they need to make a conclusive decision. AND (&&) will return false upon the first operand resolving to false, and does not bother to check any other operands after that. Similarly, OR (||) will return true for the first operand resolving to true.
The non-short-circuit operators (& and |) evaluate all the operands. This is a key point to remember. XOR (^) returns true when EXACTLY one operand is true. The ! operator returns the opposite of the boolean value.

Onto chapter 5, where flow control rules.

SCJP Chapter 3

2006-12-06T14:44:00.000+08:00

A lot of information in this one, a 'foundation' chapter indeed.

Starting off with a look into assigning primitives and object references, which then necessitates casting types to the correct one. Implicit casts are done when there is no loss of data, conversely explicit casts are needed when data is at risk of being truncated.

A brief explanation on scope and lifetimes of variables followed by default initialization of instance and local variables. Watch out for shadow variables that exist in their own scope and override any other mimics.

Arrays: declaration, construction and initialization. 1-D, 2-D or n-D; a lot of logic needed for this one.

Wrapper classes for the primitives, enabling them to used as objects. Autoboxing is a new feature in Java 1.5, and is involved when overloading methods with wrappers. Overload paths - widening, autoboxing and var-args, in that order. Beware combination questions.

Lastly is garbage collection. Once an object has no references to it, it is ripe for collection. A request for garbage collection may be made but execution is not guaranteed. Similarly, the finalize method is also not guaranteed. If finalize runs, it does so only once for an object and may save the object from deletion.

Onto chapter 4!

Blackout

2006-12-05T21:46:00.000+08:00

Power went out for the whole building around 3 o'clock. It seems that TNB (electric company) is to blame. Hopefully, by tomorrow it will be resolved.

SCJP Chapter 2

2006-12-04T16:54:00.000+08:00

Overriding and overloading, potential mix-up. Overriding methods to replace the original code with something different. Overloading methods to increase the number of methods available.

Constructors and their (default) construction. No-arg constructors, particularly default ones, are needed else beware of pitfalls in the super callers.

A brief look on static, static methods cannot access non-static variables and methods.

Misreading questions is hazardous to one's answers.

SCJP Chapter 1

2006-12-04T11:43:00.000+08:00

Dealing with access modifiers and visibility of classes, methods and variables. default (package), public, private, protected are the access modifiers with final and static thrown into the mix.

Shows correct declaration forms, especially for interfaces and abstract classes. Warns of 'compilable' questions.

First day

2006-12-04T11:27:00.000+08:00

The room is fine, comfortable for one person working. The table takes up a third of the space but on the upside, there's more space to place stuff down. Two extra chairs here that won't see much use.

For two persons, will need to replace the table with a smaller one or two. Would be a tighter fit at that time.

Very quiet environment, good for concentration and focus.

Testing 1, 2, 3

2006-11-28T11:56:00.000+08:00

First post for Insert Something Creative.