LEJOS: Programming the NXT in Java

Where you can find my code

2011-03-20T12:58:00.000-04:00

Some of my older posts have outdated links pointing to my old website at the University of Waterloo. My code can be found at

https://sites.google.com/site/lejosprogramming/

I apologize for any confusion that this may have caused. One person wrote asking for the Java implementation of the Box-Muller algorithm. It is included in the zip file, which is located at the link given above.

..Allan

Tutorial on Building a Thermostatic Sensor for your NXT

2010-08-30T02:08:00.002-04:00

In this video, I take you step-by-step through the process of creating a thermostatic sensor for your NXT. Enjoy!

The NXT as a Home Thersomstat using the HiTechnic Experimenter's Kit

2010-08-28T22:33:00.000-04:00

In my next blog entry, I'll show you how to use the NXT with the HitTechnic Experimenter's Kit to construct a household thermostat. I am currently making the video. I'll have code and a circuit diagram to show you. Stay tuned!!!

Monte Carlo Localization and Code Tutorial

2010-08-23T07:15:00.000-04:00

I have been asked to explain how the MCL class works. Rather than giving a long written explanation, I've chosen to give a video tutorial. Remember that the source code that I am referring in the video is available for download.

Enjoy!

Build your own Sound Meter with Lights

2010-08-17T21:30:00.000-04:00

Using the HiTechnic Experimenter's Kit, you can easily create a sound meter with lights. Here is my YouTube video illustrating how this can be done! Download the source code.

Robot Building Instructions

2010-08-16T19:58:00.000-04:00

I have been asked what robot design I use. Actually, I use the Robot Educator Model courtesy of Carnegie Mellon University's Robot Academy. It's a simple design and it can be constructed quite easily and rapidly.

Lejos and the HiTechnic Experiment Kit

2010-08-15T20:33:00.004-04:00

I recently purchased the HiTechnic Experiment Kit. I'm having a lot of fun with it. Even though the code for the eight experiments is written in NXT-G and NXTC, I have found that the kit is perfectly suitable for use with Lejos. You can download my example code for Experiments 1 and 2 as shown in the book that comes with the kit.

Also, please view my video tutorial, which explains Experiment 2 in detail, including the code that supports it. The experiment deals with having the NXT control 6 LEDs based upon the setting of a potentiometer.

It's a product certainly worth purchasing!

Customizing the GUI

2010-08-14T20:32:00.000-04:00

I have been asked how to change the GUI’s connect panel to change the name and Bluetooth address of the NXT or NXTs you want to connect to.

In the ConnectPanel.java file, you need to change these two lines of code:

private static final String[] _robotNames = {"MONTE", "Wayfinder"};

private static final String[] _robotAddresses = {"00:16:53:00:57:37","00:16:53:08:82:3a"};

In my particular case, my NXTs are called MONTE and Wayfinder; their Bluetooth addresses are 00:16:53:00:57:37 and 00:16:53:08:82:3a respectively.

For example, if your NXT is called Delta with a Bluetooth address of 00:16:53:00:00:00, then the two lines of java code should read:

private static final String[] _robotNames = {"Delta"};

private static final String[] _robotAddresses = {"00:16:53:00:00:00"};

Thank you for your feedback.

In my next posts, I’ll show you how to change the NXT’s map, and how to build the robot in Lego.

Video Tutorial

2010-08-09T19:27:00.001-04:00

My video tutorial can be found below. Enjoy!

Download My Code

2010-08-08T12:03:00.003-04:00

You can download my full project code here.

A tutorial video will become available once it's uploaded to YouTube.

Let me know what you think of my project.

Finding a New Place to Post my Code

2010-08-03T22:42:00.000-04:00

I know that some of you have been sending me emails requesting my project code. I am currently looking for a place where the code could be hosted, ideally for free. Unfortunately, apart for images, this blogging service does not permit the posting of documents.

Once I figure out where I can post my project code, I'll let everyone know.

Best regards,

NXT Bluetooth Tutorial

2010-07-31T20:13:00.002-04:00

This blog is the first of many as a tutorial on using the NXT with Bluetooth. First, I’ll speak to the issue of programming the robot because it’s the easiest to understand. The following program is intended to be run on the NXT. This code will not run on your PC.

The steps for your NXT are quite simple

1.    Declare all of the class’s instance variables.
2.    Display a message on the NXT’s LCD display that the NXT is waiting for a connection. I also like the NXT to sound two beeps in case the NXT is not near enough to read the LCD.
3.    Make a blocking call to wait for a Bluetooth connection from your PC. I'll explain in future blogs how to make the PC initiate the Bluetooth connection.
4.    Once the connection is made, set up the input and output streams. I also like the NXT to sound a tune so that I know that the connection is ready.
5.    Next, send the NXT into a while loop controlled by a Boolean flag. The stuff inside the while loop is what you want the NXT to do.
6.    When your PC signals the NXT that it is through with the Bluetooth connection, the NXT should set the Boolean flag mentioned in step 5 to false. Consequently, the while loop will cease.
7.    Close the input and output buffers to close the Bluetooth connection.

Step 1: Declare all of the class’s instance variables

import lejos.nxt.*;
import lejos.nxt.comm.*;
import lejos.robotics.navigation.*;

import java.io.*;

public class Robot {

    public static void main(String [] args) throws Exception
    {
    int[] command = new int[3];
    int[] reply = new int[8];
    boolean keepItRunning = true;
    String connected = "Connected";
      String waiting = "Waiting...";
      String closing = "Closing...";
      DataInputStream dis = null;
      DataOutputStream dos = null;
      BTConnection btc = null;
      UltrasonicSensor sonic = new UltrasonicSensor(SensorPort.S1);
      Pilot piloting = new TachoPilot(56.0f, 121.0f, Motor.C, Motor.A);


The two integer arrays are used to hold the commands sent by the PC to the NXT, and the reply is used to hold the NXT's reply to the command.

The Boolean flag is used to indicate if the while loop is to keep looping.

The strings are used to display status messages on the LCD screen.

The streams are used to facilitate communication between the PC and the NXT.

The ultrasonic sensor and pilot objects are used within the while loop to control the NXT.

Step 2: Display a Message and Sound two Beeps

        LCD.drawString(waiting,0,0);
        LCD.refresh();

        Sound.twoBeeps();

These simple commands put the words “Waiting…” on the NXT’s LCD and sounds two beeps.

Step 3: Wait for a Bluetooth Connection

        btc = Bluetooth.waitForConnection();

Remember that this is a blocking method call. The NXT will not go beyond this step until it makes a Bluetooth connection. I’ll explain in future blog posts how the PC initiates the Bluetooth connection. If you try to run this program on your NXT, it will stop at this point and go no further.

Step 4: Set up the Streams

        dis = btc.openDataInputStream();
        dos = btc.openDataOutputStream();

        Sound.beepSequenceUp();

With the Bluetooth connection established the streams can be opened. I make the NXT play a tune as an audible reminder that a Bluetooth connection has been successfully made.

Steps 5 and 6: Go into a while loop

        while (keepItRunning){

        // put your stuff here including code to change the Boolean
        // flag to false once the PC signals to end the connection

        }

Step 7: Close the Streams and the Bluetooth Connection

        dis.close();
        dos.close();
        Thread.sleep(100); // wait for data to drain
        LCD.clear();
        LCD.drawString(closing,0,0);
        LCD.refresh();
        btc.close();
        LCD.clear();
} // end of main

} // end of class Robot

Closing the streams and the Bluetooth connection is very simple. I also have status messages displayed on the NXT’s LCD.

In my next blog entry I’ll elaborate on Steps 5 and 6; that is, the “stuff” you want your NXT to do.

Go questions? Post a comment. I’ll reply.

Still Fixing my Code

2010-07-26T20:34:00.000-04:00

This weekend, I was fixing up my code for my Monte Carlo localization project. Unfortunately, the Lejos API was changed so much so that much of the functionality was broken.

While it didn't take long to change the code over, now I have to test it to ensure that the new libraries that I'm accessing produce similar results.

I'll probably get back to more fixing of my code this weekend coming up.

API Changes have broken my code

2010-07-20T07:49:00.000-04:00

Since I've been away, changes have been made to the Lejos API. Unfortunately, the removal of some classes, and the changes in other classes have broken my code. This code used Bluetooth, and I wanted to illustrate the use of my code to you.

In my next blog entry, I'll demonstrate why changing a public interface in an API can adversely affect users.

Back in Business

2010-07-18T19:28:00.004-04:00

After a long time away from blogging about the NXT and LEJOS programming in the Java programming language, I'm back.

I'm a software developer at Research in Motion, the makers of the BlackBerry smartphone.

My major interest with the NXT is using Bluetooth to use the PC and the NXT together. The PC is dedicated to do complex calculations, while the NXT is used as, essentially, a mobile sensor platform. I am particularly interested in the Monte Carlo localization.

Go questions. Send me a reply to this post.

Stay tuned for new blog posts.

The BlackBerry Simulator

2009-09-07T12:12:00.006-04:00

I have been working with the BlackBerry simulator for the past while, working on arguably simple code to better understand and appreciate how BlackBerry applications are written using J2ME. Here is one example of code that I have had running successfully, based upon an example given in the BlackBerry Developer Zone.

I am intrigued by the BlackBerry. Would I be able to use the BlackBerry as a remote control for the NXT robot? While I can certainly write an application for the PC to control the NXT over Bluetooth, the PC is necessarily stationary. The robot cannot move outside of the Bluetooth broadcast range. On the other hand, if the BlackBerry were used as the remote control device, the remote control is now portable. The user can now move along with the NXT. The NXT's range of movement is theoretically unlimited.

Generating 2D, 3D, and nD Multinomials randomly

2009-04-21T16:42:00.003-04:00

I have been much engrossed with the idea of Gaussian Mixture Models (GMMs) and their applicability to computer vision, especially in segmentation.

However, to be able to test a GMM, it is necessary to generate random vectors from Gaussian distributions. While it is true that Matlab has a function call mvnrnd() for this purpose, I have developed a function that produces a very similar result with great ease. You can click here for a detailed explanation of the code:

%% Function

% Generates normally distributed values

function p = normalMulti(mu, sigma, cases)

[row col] = size(mu);

if(col ~= 1)

mu = mu';

[row col] = size(mu);

end

[V D] = eig(sigma);

A = V*sqrt(D);

p = A*randn(row,cases) + repmat(mu,1,cases);

Controlling the NXT with Matlab

2009-02-20T17:04:00.002-05:00

Here is an interesting link that I found, if you want to try and control the NXT using Matlab. I have not tried this approach myself, but it is something interesting to consider.

Link to an MCL Project that may interest others

2009-02-18T19:41:00.005-05:00

Here is a link to another MCL project that readers may find interesting. Thanks Juan for bringing the link to my attention.

Link to Sample Code

2009-02-15T13:18:00.003-05:00

I have included in the banner of my blog a link to my sample code.

The Reason for not Blogging Lately

2009-02-07T18:03:00.001-05:00

I have been unable to blog lately because I have been away. However, I will soon be back to my normal routine. Once I'm back, I'll be blogging once again on LEJOS: Programming the NXT in Java.

Localization Video

2009-01-12T20:53:00.002-05:00

Here is a video from an unrelated MCL project. I am posting it so that you can see the basic idea.

Added new BoxMuller.java Class

2009-01-12T08:02:00.006-05:00

I have added a new class called BoxMuller.java. Its sole static method randn(float mu, float sigma) returns a random floating-point number drawn from a normal distribution of mean mu and standard deviation sigma. Essentially, it produces the same result as Matlab's randn function.

The BoxMuller.randn method is used in the MCL.java class to add noise to a newly chosen particle in the chooseNewPoseSet method. I have corrected this method so that it now refers to the randn method.

The Box-Muller Algorithm

2009-01-11T14:55:00.010-05:00

Many thanks to Professior Gordon Wyeth at the University of Queensland and his MCL code. I am grateful to him for bringing the Box-Muller algorithm to my attention. It is a way of generating normally distributed random variables with mean zero and standard deviation one, given a set of uniformly generated random variables in the range 0 to 1. This algorithm is important when adding Gaussian noise to a process in the MCL algorithm. Unfortunately, Java's Math class has no method to generate random numbers from a normal distribution. However, Java does have a uniform random number generator function, which can be used to generate a normal distribution by applying the Box-Muller algorithm.

You can download my code to test this algorithm. Here is a frequency histogram of the data that I generated from my program:

This frequency distribution has a mean of 0.0036 and a standard deviation of 1.0008. Of course, your own results will be different, but the mean and standard deviation should come close to 0 and 1, respectively.

The best explanation of why the Box-Muller algorithm works as it does is in this document.

Trying to Find a Java Graphics Package

2009-01-11T10:28:00.004-05:00

In my Monte Carlo Localization project, it would be ideal if I could draw 2D histograms on the Graphics User Interface of the particles, which represent an approximation of the probability distribution function of the location of the robot. At present, I have a 2D plot of the position of the particles.

Because drawing 3D graphs is a bit beyond the scope of my project, I really don't want to implement this functionality from scratch. Does anyone know of any pre-built java code that supports this kind of display?

I have tried google, but there seems to be many choices. I'm looking for a specific recommendation please of one that works better than the others. If you should have a recommendation, please send it to adcaine@gmail.com.

Thank you.

Test my New Program

2009-01-10T13:08:00.003-05:00

Over the next few days, I'm going to be testing my new Monte Carlo Localization (MCL) application on different kinds of enclosures I want to see if the algorithm can handle different situations, localizing the robot correctly within the enclosure. Once I have some results, I will share them with you.

Finding a Pose's Distance to a Boundary

2009-01-09T10:41:00.005-05:00

In the particle localization algorithm, it is crucial for each particle to know its distance to the closest boundary. That distance is compared to the distance measured by the ultrasonic sensor on the robot. If the difference in distances is small, the particle is given a high weight. The particle is seen as a potentially good approximation of the robot's true pose.

On the other hand, if the difference between the particle's computed distance and the distance recorded by the sensor is great, the particle is given a low weight. The particle is seen as being a poor approximation of the robot's true pose.

In this document, I give the details of how the algorithm in the MCL.java class computes the particle distances.

Beta Version of the MCL Released

2009-01-08T21:08:00.013-05:00

Here is the Beta Version of the Monte Carlo Localization project that I have been working on. You can find all of the source code in the table at the end of this post. Here is an example of the output one gets. Further testing is needed in more complicated maps. If you should find any bugs, I will sincerely appreciate it if you report any bugs to adcaine@gmail.com.

NXT Program

Robot.java
This is the program that should be uploaded onto the brick

Ordinary Java Classes

Tester.java
Run this class to initiate the GUI

Localization.java
This class is reponsible for putting the main window on the screen.

BTCommPC.java
Responsible for maintaining communication between the PC and the NXT

CommandPanel.java
Defines the command panel portion of the GUI.

ConnectPanel.java
Defines the connection panel portion of the GUI.

DisplayPanel.java
Defines the display panel portion of the GUI.

EventHandler.java
Handles all action events fired by the GUI's panels.

Gaussian.java
Defines a Gaussian function used in particle weighting.

Line.java
Defines a line, used for boundaries in the map.

MCL.java
Defines the routines used in the Monte Carlo Localization

Point.java
Defines a point as used in the map, lines, and poses.

Pose.java
Defines a pose as used in the particle set

Statistics.java
Defines the statistics panel of the GUI.

A Wii Controlled Robot Arm

2009-01-07T21:45:00.002-05:00

Credit for this project goes to LaFabrick Flash Platform Fan Club Member:

How to Change a Class without Recoding It

2009-01-06T16:17:00.008-05:00

When I was an undergrad studying Java programming, I found the explanation of interfaces to be rather unsatisfying. The instructor’s explanation would often begin like this:

Suppose we have a Java interface defined as


package abstraction;

public interface Formula {
 
 public float getResultOfFormula(float x);

}

Therefore, any class that implements the interface called Formula must provide the method body for getResultOfFormula. It must take a floating point number as a parameter and return a floating point number. An implementing class might take the form:


package abstraction;

public class ImplementedFormula implements Formula {

 public float getResultOfFormula(float x) {
  return (float)Math.exp(x);
 }
}

In this implementation, the method returns the exponential of x. Well, so what? It seems that we have accomplished so very little.

To understand how this idea might be very powerful, let’s consider a concrete example. Let’s suppose that we want to create a class called Computational. It has a method called getValue(float x). By default, the method returns 2*x. However, we also want the user to be able to define the formula without having to rewrite the Computational class. Here is how it is done:


package abstraction;

public class Computational {
 
 private abstraction.Formula userDefinedFormula = null;
 
 public void defineFormula(abstraction.Formula f){
  userDefinedFormula = f;
 }
 
 public float getValue(float y){
  if(userDefinedFormula == null){
   return 2*y;
  }else{
   return userDefinedFormula.getResultOfFormula(y);
  }
 }

}

In the first line, we provide for a variable of the type Formula and initially set it to null. While it is true that interfaces cannot be instantiated, classes that implement interfaces can certainly be instantiated. So, an instantiation of the class ImplementedFormula is of the type Formula.

Indeed, the method defineFormula(Formula f) allows the user to pass a class that has implemented the Formula interface, and the userDefinedFormula variable is set to the parameter.

The method getValue(float y) has an if-statement to check the nullity of the variable userDefinedFormula. If it is null, then the user has not defined the formula and the default of 2*y is used. Otherwise, the method returns the user’s definition. The class called Computational is guaranteed that the method getResultsOfFormula(float y) exists because it must have implemented the Formula interface.

To see how the Computational class might be used in practice, consider this code:


package abstraction;

public class WorkingClass {

 public void runThisClass(){
  Computational comp = new Computational();
  comp.defineFormula(new ImplementedFormula());
  System.out.println(comp.getValue(30.0f));
 }

 public static void main(String[] args) {
  WorkingClass wc = new WorkingClass();
  wc.runThisClass();

 }

}

If this class is run, the output will be 1.06864742E13, which is the exponential of 30. However, if the second line in the runThisClass() method is commented out so that the formula is not user-defined, the output is 60, 2*30. The Computational class has used the default formula of 2*x.

The Computational class could be enhanced by adding another method called removeFormula() such as:


 public void removeFormula(){
  userDefinedFormula = null;
 }

By resetting the variable userDefinedFormula back to null, the method getValue() will use the default.

Finally, the user is not restricted to a single implementation of the Formula interface. Indeed, the user might implement it several times, with the user re-defining the getValue(float x) as the user wishes.

The GUI is Implemented

2009-01-05T12:40:00.003-05:00

The GUI for the Monte Carlo localization is largely implemented. The directory for the files can be found here. The most recently updated files are CommandPanel.java, EventHandler.java, Statistics.java, FileDump.java, ConnectPanel.java, and Localization.java.

I have also added a new class called Robot.java. This class is meant to be run on the NXT brick. Consequently, it should be placed in a separate project, with the project being turned into a Lejos project. I assume here that you are using Eclipse. Link and upload Robot.java to your brick.

To establish the Bluetooth connection, first run Robot.java on the brick. You should hear two beeps, which indicates that the brick is waiting for a Bluetooth connection from the PC.

Next, start Tester.java running. The GUI should open. Recall that the Bluetooth field has been primed with the Bluetooth address of my brick. Your Bluetooth address will be different, and you should enter your brick’s Bluetooth address in the field. Now, press the connect button.

Once the Bluetooth connection is established, you should hear the robot chime, which indicates that it is connected. On the GUI, the connect button should turn red and read disconnect. The status field should read connected. All of the buttons on the GUI should become active.

You can make the robot turn and move using the appropriate buttons and by supplying an angle in degrees or a distance in millimeters, as the case may be. You can request the battery voltage, or manually ping the ultrasonic sensor. After every turn or move, the latest ultrasonic sensor readings should appear on the GUI.

You can dump the statistics of the MCL algorithm into a csv file. However, please note that the file path and file names are hard coded in the Event Handler class. I welcome suggestions on how to make this more flexible.

The Robot class assumes 56 mm diameter wheels with a 121 mm wheel base. The left motor is connected to the C port; the right to the A port. If your robot’s configuration is different than mine, you will have to change the line of code in the Robot class, which instantiates the Pilot class. The Robot class also assumes that the ultrasonic sensor is connected to sensor port S1. Additionally, I assume that the ultrasonic sensor will point directly straight ahead along the robot's line of forward travel. I welcome suggestions on how to make the configuration of the physical parameters more flexible for the user.

Because the MCL algorithm is not yet fully implemented, the particles on the display do not appear to change in response to movements or turns of the robot. This is to be expected. However, if you do want to see the display change, click on the initialize button to restart the MCL algorithm.

To disconnect the PC and the robot, click on the disconnect button, and click on the yes button to confirm. Once the disconnection cycle is complete, the robot should chime. On the GUI, the disconnect button should now read connect in green lettering, and the status field should now read disconnected.

To bring the localization program to an end, click on the close button, and then click on the yes button to confirm.

Happy roboting!

To submit suggestions or report bugs, please email me at adcaine@gmail.com.

Thank you.

The Communication Protocol

2009-01-04T19:24:00.002-05:00

In communicating with the NXT robot, we want to keep our communication protocol as simple as possible. Through simplicity, the PC can direct more of its attention to the problem of Monte Carlo Localization (MCL), and less of its time on Bluetooth connection. The PC sends its command in the form of three integers. The first integer is the command itself, an integer from 0 to 4. The two integers that follow the command are optional parameters.

The robot’s reply is always eight integers. We need eight integers to accommodate the eight echoes that we obtain from the ultrasonic sensor.

There are five cases to consider:


Description:  The user requests the battery voltage
Command          0, 0, 0
Reply   millivoltage, 0, 0, 0, 0, 0, 0, 0


Description:  The user asks for the ultrasonic sensor to ping and report back
Command          1,0,0
Reply   A series of 8 integers being the echo values


Description:  The user requests that the robot move forward/backward
Command          2,distance,speed
Reply     A series of 8 integers being the echo values


Description  The user requests that the robot turn left/right
Command          3,angle,speed
Reply   A series of 8 integers being the echo values


Description  The user requests that the Bluetooth connection be terminated
Command          4,0,0
Reply   255,255,255,255,255,255,255,255

The responsibility for sending the command to the robot and receiving a reply from it is shared between the Event Handler and BTCommPC classes. Upon detecting that an action event has been fired on the GUI, it is the Event Handler’s responsibility to gather the necessary information, compose the three-integer command and pass a reference to the integer array to the BTCommPC class. BTCommPC is responsible for sending the command one integer at a time to the robot and await a reply. Once a reply starts being sent, BTCommPC reads each integer, one at a time, placing them in an eight-integer array. BTCommPC then passes a reference to the array to the Event Handler.

Once the Event Handler receives a reference to the reply array, it is handler’s responsibility to disseminate it. Usually, this involves passing values back to the GUI panels and passing the ultrasonic sensor readings to the MCL class so that the MCL class can update its statistics. This command/reply process continues until the user requests that the Bluetooth connection be terminated either explicitly by pressing the disconnect button, or implicitly by terminating the program.

The robot’s program is complementary to BTCommPC’s algorithm. At first, the robot waits for a Bluetooth connection. Once a connection is made, it waits to receive its three-integer command. Once the command is received, the robot moves or turns, if necessary, and then sends back its eight-integer reply one integer at a time. The robot then waits for its next command. If the robot is commanded to terminate its Bluetooth connection, the robot sends back its acknowledgement, disconnects, and its program terminates on the brick.

More implementation to come.

Putting it all Together: A working GUI

2009-01-04T16:46:00.010-05:00

In this blog, I will explain DisplayPanel.java and then put everything together to make up the final GUI.

To understand and run the Display Panel, you must download Point.java, Line.java, and Pose.java. A point is a location in two-dimensional space; the class is the aggregate of two double precision numbers. In our implementation, a line is defined in terms of two points: a starting point and an ending point. A pose is a particular guess at the location and heading of the robot. It is defined in terms of heading, a point, and three other variables. Those three other variables are the pose’s distance, the shortest distance to the nearest boundary of the enclosure; its weight, the extent to which we believe that the pose is a correct estimate of the robot’s true heading and position; and finally its cumulative weight, the sum of the weights of all of the poses before it and itself.

The display panel looks like this

Admittedly, it’s not very impressive. No pose or boundary line information has been given to it to display. So, the panel simply shows its white background. Once pose and boundary information is given to the display panel, it will show the poses as red dots, and the boundaries and black lines.

Still, we’re now in a position to start putting everything together to get a GUI that looks like this:

The directory to the source code is here. To get the GUI running, execute it from Tester.java.

I have implemented the Connect button. Remember to substitute your own Bluetooth address for your own NXT in the Bluetooth address field of the GUI. Turn on your NXT. Click on the Connect button. You should be connected. The disconnect aspect of the button has not yet been implemented. However, you can achieve a disconnection by pressing on the enter and escape keys of your NXT to reset it. The Bluetooth connection will be lost.

The Localization class is the class responsible for assembling the GUI. It instantiates the panels and, using absolute positioning, places them on the content pane. Because we are using absolute positioning, we make the GUI non-resizable. If we left the GUI resizable, a resizing on the part of the user could cause panels to become hidden from the user.

The Localization class also instantiates the Event Handler, passing a reference to it to all of the panels. The implementation of the Bluetooth connection routine can be found in the Event Handler class.

As well, the Localization class instantiates the BTCommPC class. This class is responsible for opening the Bluetooth connection between the PC and the robot, closing the connection, and negotiating communication between the two. As soon as the Event Handler detects that a connect action event has been fired, the Event Handler calls BTCommPC’s open method to take the necessary steps to open the Bluetooth connection, if it can.

Because problems can arise in establishing a Bluetooth connection, exceptions can be thrown. When these exceptions are thrown, we want to put error messages up on the GUI, such as

To do this, we use the JOptionPane’s static showMessageDialog() method. However, that method requires that it know in which JFrame to draw the dialog box. Consequently, the Localization class passes a reference to itself when it instantiates the BTCommPC class.

Finally, the Localization class is responsible for listening to the user clicking on the close window button in the upper right-hand corner, because the close button is regarded as part of the JFrame and not as part of any of the JPanels. Before terminating the program, the Localization class puts put a confirmation window on the screen.

If the user clicks on the yes button, the Localization class attempts to programmatically click the disconnect button if it currently appears on the GUI. Only when the Bluetooth connection is closed does the program finally terminate.

If you should have any questions, please write to me at adcaine@gmail.com. See you next time when I will show you additional implementations.

Connection, Command, and File Dump Panels

2009-01-04T12:57:00.011-05:00

This blog entry is the continuation of my discussion of my project to demonstrate the idea behind Monte Carlo Localization (MCL) for robots using the NXT Mindstorms robot. The PC will run the MCL algorithm with the robot acting essentially like a mobile sensor platform. The PC and robot will communicate with each other over a Bluetooth connection. Here, I am going to discuss the connection panel, the command panel, and the file dump panel.

We want the Connection Panel to look like this

The single button at the top will act as both the connect and the disconnect button. The text on the button will alternate between the words connect, printed in green, and disconnect, printed in red. The status field, which cannot be edited by the user, will read either disconnected when the PC is not connected to the NXT robot, or connected when a Bluetooth connection has been established between the PC and the robot.

The only method that might need some explanation is this one:


public void clickConnectButton(){
 if(connectButton.getText().equals("Disconnect")){
  connectButton.doClick();
 }
}

This method is needed in case the user attempts to terminate the program before the robot is disconnected from the PC. If this should happen, we need a way to programmatically click the disconnect button before the program terminates. The doClick() method of the connectButton component does just exactly that.

I have primed the Bluetooth address field with my own NXT's Bluetooth address for the sake of personal convenience. You can change this by either commenting out or amending the following line of code.


  addressField.setText("00:16:53:00:57:37");

You can download ConnectPanel.java here.

Another panel that we need is the Command panel. We want it to look like this

It has three buttons: initialize, move, and turn. The initialize button is used to re-start the MCL algorithm from scratch. The move button is used the move the robot a specified number of millimeters forward, if the distance is positive; backwards if the distance is negative. The turn button is used to turn the robot in place a specified number of degrees to the right, if the angle is negative; to the left if the angle is positive. The speed slider is used to adjust the robot’s speed.

We should emphasize that we are not trying to develop a remote control robot. Rather, the robot moves or turns in distinct stages. After each move or turn, students can observe how that move or turn has impacted the MCL algorithm. I am hoping that in this way students can learn how the MCL algorithm works through experimentation and feedback.

The following method may need some additional explanation:


public void setButtonsEnabled(boolean setting){
 initialize.setEnabled(setting);
 move.setEnabled(setting);
 turn.setEnabled(setting);
 speedSlider.setEnabled(setting);
}

A similar method also appears in the Statistics panel. This method is used to enable or disable the input components. The components are disabled whenever there is no connection between the PC and the robot; enabled whenever there is a connection between the PC and the robot. You can download CommandPanel.java here.

Another panel that we need is called the File Dump panel. We want the panel to look like this:

When the user clicks on the File Dump button, all of the statistics currently held by the MCL algorithm are written to a comma-separated value (csv) file using the index number on the spinner as part of the file’s file name. The spinner is automatically incremented to the next integer to avoid an accidental file overwrite. If the auto dump box is checked by the user, then the statistics are written to a csv file automatically after every move or turn of the robot. The spinner is automatically incremented to prevent an accidental file overwrite.

The purpose of the file dump panel is to allow the user to study the MCL algorithm's statistical data in another application such as Matlab or Excel to name two examples.

The File Dump button is always enabled, because there is always statistics to be written out to a file, even when the program is first initialized. You can download FileDump.java here.

The last panel that we will need is the Display Panel. I want to defer discussion of that panel until the next time I blog, because that panel touches very closely upon the MCL algorithm. While the class itself is not complicated as far as Java code goes, some of the things that the panel is doing might be unclear and require some explanation. If you should have any questions, please contact me at adcaine@gmail.com. See you next time.

Monte Carlo Localization of Robots

2009-01-03T20:27:00.014-05:00

In this and future blog entries, I want to discuss my current project with the NXT. I want to develop a program, written in Java, which demonstrates the concept of Monte Carlo robot localization (MCL). For now, I’ll defer any discussion about MCL to another blog entry. However, for those of you who want to read about MCL now, please read the paper by Dellaert, Fox, Burgard, and Thrun.

The problem with demonstrating MCL to a group of students is that the program cannot be run locally on the NXT brick itself. True, the students will be able to see my robot moving around, but they will not be able to understand how the MCL program works.

In my project, the MCL algorithm will be implemented on a PC, which has Bluetooth capability. Using Bluetooth, the PC will send commands to the robot. In response to these commands, the robot, which also has Bluetooth capability, will perform the action -- such as move forward, move backward, turn left, turn right -- and then send back to the PC the reading from the robot’s only sensor, the ultrasonic sensor.

Using the reading from the ultrasonic sensor together with a map of the walls of the enclosure in which the robot is found, the PC will try to localize the robot; i.e. determine its position within the enclosure.

The most challenging part of the project is actually the development of the graphical user interface (GUI). We cannot be content to use the RConsole class as mentioned in my January 1st blog entry. RConsole is too primitive. What we want and need is a professional and attractive display of the current state of the MCL algorithm.

In Java, the main window is called a JFrame. Every JFrame consists of a number of JPanels. Every JPanel consists of a number of Components. Components are such things as buttons, labels, text boxes, sliders, spinners, progress bars, radio buttons, check boxes and other such widgets commonly found on GUIs.

The idea behind this project is that we will define a series of JPanels. Another class, to be defined later, will instantiate each of these JPanels and arrange them in a JFrame, which will act as our project’s GUI.

For our GUI, we’re going to need four panels: a connection panel for handling the Bluetooth connection; a command panel for issuing commands to the robot; a statistics panel for collecting and displaying sensor readings from the robot; and a display panel for displaying the current state of the MCL algorithm. In this blog entry, we’ll focus on the statistics panel.

Before coding the statistics panel, we must consider that every panel can fire what Java calls action events. These action events fire, usually, in response to clicks on buttons, moving a slider, or changing the value of a spinner to name a few examples. We need a mechanism to capture these action events and respond appropriately to them. If we do not listen for these action events, our GUI will be unresponsive and useless to the user.

Unlike my blog entry of January 2nd, we do not want each panel to handle its own events through the use of the code addActionListener(this);. Otherwise, our action event handling code will be spread over several classes making our code very difficult to maintain. Instead, we want a single class to centrally handle all of the action events for all of the panels. The event handler class is:

package local;

import java.awt.event.*;

public class EventHandler implements ActionListener {

public void actionPerformed(ActionEvent e){

}

}

For now, the actionPerformed method has an empty implementation, which we will fill in as we need to code up the steps the EventHandler should take in response to the various action events sent to it by the various panels. Whenever an action event is fired, the actionPerformed method will be called to handle the action event.

However, each panel will need a reference to the EventHandler as instantiated by the class responsible for assembling the JFrame together. This is accomplished by defining each panel’s constructor as public panelName(EventHandler eventH){code}. In other words, for a panel to be instantiated, the panel must be passed a reference to the EventHandler. The class responsible for assembling the JFrame would do something like this:

EventHandler eventHndl = new EventHandler();
Statistics statsPanel = new Statistics(eventHndl);

We want the statistics panel to show the battery voltage and the ultrasonic sensor readings as vertical progress bars. We want the first echo from the ultrasonic sensor to appear in a textbox as a numerical value. We want two buttons: one button to call for the battery voltage and a second button to manually cause the ultrasonic sensor to be pinged and the result displayed on the GUI. The panel should look like this:

The code at the end of this blog entry is meant to be run on your PC. Consequently you should not change your project into a Lejos project. The main method can be called to display the panel, but it must be borne in mind that this main method is strictly for testing and debugging purposes. The real main method will appear in another class. In practice, the main method of the Statistics class will never be run.

Note that the Statistics class extends JPanel --- not JFrame. This is important. Many examples of GUIs given in web tutorials show classes extending JFrame, because these examples typically involve components being added directly to the JFrame. While this approach is fine in simple applications, it is difficult to use when the GUI becomes complex. So, we are not taking that approach here. Components are being added to each JPanel, and each JPanel is being added to a single JFrame.

Finally, the Statistics class owns its components in the sense that its components are private. The state of components can only be changed by users of the panel through the class’s set methods; the values stored in the components can only be accessed through the class's get methods. This approach ensures that the correct data types are used when passing data to and from the JPanel. This approach also prevents the user of the class from changing the state of components that should not be changed; i.e. components like text labels that are purely decorative in nature.

If you wish to see the panel with the two buttons in the active state, add this line of code to the main method.

statsPanel.setButtonsEnabled(true);

The next time I blog, I will discuss the connection panel, the second panel which makes up the four panels of our GUI. If you have any questions, please do not hesitate to contact me. The code for the Statistics class can be found here.

Efficient Programs and Button Listeners

2009-01-02T10:58:00.017-05:00

In my blog on January 1st, I explained how the RConsole class can be used to easily send output from the NXT brick over a Bluetooth connection to your PC. However, the program has a deficiency. The escape button must be pressed and held for the program to stop.

True, the while loop is terminated whenever the brick’s escape button is pressed. But, after reading the raw and processed values of the Electro Optical Proximity Detector (EOPD) and printlning those results to the RConsole, the loop pauses for five seconds. This pausing means that the escape button’s state is checked approximately once every five seconds at the top of the while loop.

So, to stop the program, the user has to press and hold down the escape button until the while loop has an opportunity to read the button’s state. Pressing and releasing the escape button is insufficient. If the escape button is in the released position when the while loop checks the button’s state, the while loop will execute again even though the button might have been pressed and released when the loop was paused.

This problem is best solved by using the ButtonListener interface. The interface has two methods: buttonPressed(Button b) and buttonReleased(Button b). Once a button has been registered with a listener, these methods fire whenever the button is pressed or released, as the case may be. Furthermore, these methods fire regardless of what other things the program might be currently doing.

In the new version of the program, given at the end of this blog entry, the while loop is now controlled by a boolean variable, keepItRunning. The variable is initialized to true. The program implements the buttonPressed method such that it changes the keepItRunning variable to false. Although we are not explicitly using the buttonReleased method, we are obliged to implement it because it is part of the ButtonListener interface. So, we give the buttonReleased method an empty implementation.

The program line Button.ESCAPE.addButtonListener(this); registers the escape button with a listener. The this keyword indicates that the buttonPressed and buttonReleased methods are implemented in this class. (It is possible for the methods to be in another class; so, the addButtonListener method must know where these methods can be found.)

To try this new idea out, upload this program, as given below, to your NXT and start it running. Now, get your nxjconsole running from a command prompt. The program works in exactly the same way as described in my January 1st post. However, if the escape button is pressed and released, the program will end after its five-second pause. The program ends because the pressing of the escape button fires an event, which in turn calls the buttonPressed method. That method changes the keepItRunning boolean variable from true to false. The while loop ends because the while loop will only run as long as keepItRunning is true.

What follows next is the code for EOPDSensor2:

import lejos.nxt.Button;
import lejos.nxt.LCD;
import lejos.nxt.SensorPort;
import lejos.nxt.comm.*;
import lejos.nxt.addon.*;
import lejos.nxt.*;

public class EOPDSensor2 implements ButtonListener{

private boolean keepItRunning = true;

public void buttonPressed(Button b){
keepItRunning = false;
}

public void buttonReleased(Button b){}

public void runProgram(){

Button.ESCAPE.addButtonListener(this);

RConsole.openBluetooth(40000);

LCD.drawString("Connected", 0, 2);

EOPD electroSensor = new EOPD(SensorPort.S1);

while(keepItRunning){
electroSensor.setModeLong();
RConsole.println(Integer.toString(
electroSensor.readRawValue()));
RConsole.println(Integer.toString(
electroSensor.processedValue()));

try{
Thread.sleep(5000);
}catch(InterruptedException e){
System.exit(0);
}
}

RConsole.println("Closing...");

RConsole.close();
}

public static void main(String[] args){
EOPDSensor2 programInstance = new EOPDSensor2();
programInstance.runProgram();
}
}

The Powerful RConsole Class

2009-01-01T16:21:00.018-05:00

The RConsole class is a very useful way of sending output from the nxt brick over a Bluetooth connection to the PC. Consider the following Java code, which runs on the brick. The class EOPDSensor reads the raw and processed values of the Electro Optical Proximity Detector (EOPD) available from HiTechnic and sends those sensor readings back to the console:


import lejos.nxt.Button;
import lejos.nxt.LCD;
import lejos.nxt.SensorPort;
import lejos.nxt.comm.*;
import lejos.nxt.addon.*;

public class EOPDSensor {

 /**
  * @param args
  */
 public static void main(String[] args) {
   
  RConsole.openBluetooth(40000);
  
  LCD.drawString("Connected", 0, 2);
  
  EOPD electroSensor = new EOPD(SensorPort.S1);
  
  while(!Button.ESCAPE.isPressed()){
   electroSensor.setModeLong();
   RConsole.println(Integer.toString(
                       electroSensor.readRawValue()));
   RConsole.println(Integer.toString(
                       electroSensor.processedValue()));
   
   try{
    Thread.sleep(5000);
   }catch(InterruptedException e){
    System.exit(0);
   }
  }
  
  RConsole.println("Closing...");
  
  RConsole.close();
  
  

 }

}

The methods of the RConsole class are all static; the methods belong to the class and not to any instance of the class. So, the RConsole class should not be instantiated with the new keyword. The methods should be called in the form RConsole.method().

In the first line of the main method, RConsole waits 40 seconds (40,000 milliseconds) for the master (i.e the PC) to attempt to open a Bluetooth connection. Once the connection is established, the second and third lines of the while loop println the raw and processed values of the EOPD sensor, respectively. If the user presses the brick’s escape button, the tiny square gray button on the brick, the loop ends. The brick invokes the RConsole's close() method to close the Bluetooth connection.

To establish the connection in practice, do the following things. First, get to a command prompt on your PC. Next, run the Java program on the brick. Remember that the brick will wait only 40 seconds for the PC to attempt a connection. So, you must work quickly once the brick's Java program has started running.

At the PC’s command prompt, enter the command nxjconsole. Your PC will now go out and try to establish a connection with the brick. Once the connection is established, output similar to the following should appear.

To sum it up, the RConsole class is a very quick and easy way of getting output from your brick and onto your PC.

Self-balancing Robots

2008-12-31T21:55:00.002-05:00

Important Links

2008-12-31T17:21:00.000-05:00

Here are some very useful links to get information about programming the NXT in the Java Language:

SourceForge From here, you can download the jar files as well as the brick's firmware to get you started programming in Lejos -- Java for the Mindstorms NXT brick
Setup Tutorial From this website, you can learn how to properly set up your brick with the firmware, and correctly get Eclipse (a Java development environment) working properly
Forum In this forum, you can communicate with other developers who use Lejos to program the NXT brick