Project Euler, problem 1, multiple factors

If we list all the natural numbers below 10 that are multiples of 3 or 5, we get 3, 5, 6 and 9. The sum of these multiples is 23.

Find the sum of all the multiples of 3 or 5 below 1000.

To solve this in a brute force manner, we can count up to 1000, for each number we check to see if it divides into 3 and 5 leaving no remainder. If it does, they we add that number into our running total.

Heres my solution using Java.

public class Problem1 {

    public static void main(String[] args) {
        System.out.println(solve(1000));
    }

    public static int solve(final int subject) {
        int sum = 0;
        for (int i = 0; i < subject; i++) {
            if ((i % 3 == 0) || (i % 5 == 0)) {
                sum += i;
            }
        }
        return sum;
    }
}

The answer is 233168

An intro to Node.js, building a URL shortener

Node has been on my todo list of things to investigate for a little while now, whilst I don’t have much background in javascript, after constantly hearing about it when I was working at O2 Telefonica, I thought I better see what the fuss is all about.

To quote the Node website: “Node.js is a platform built on Chrome’s JavaScript runtime for easily building fast, scalable network applications. Node.js uses an event-driven, non-blocking I/O model that makes it lightweight and efficient, perfect for data-intensive real-time applications that run across distributed devices.”

The best way to learn how to use a new technology is to try and build something with it. I’ve chosen to build a URL shortener because its a relatively straightforward concept; you take a URL and store it against a key, then when that key is requested, you redirect to the original URL. These are commonly used on twitter and other social media where you are limited to sharing small messages, as it enables you to have short URLs (normally on short domains such as bit.ly) refer to a longer URL that you want to share.

I’ve decided to use express, which is a framework for node js for building web applications, I’ve also chosen to use Jade for my views, which is a node template engine for generating HTML views.

Lets get started.

Firstly, you’ll need to setup a development environment if you haven’t already got one. I’m using IntelliJ IDEA and am fortunate that there is node js support that will help you create a new project.

If you’re not using IDEA, or want to create a node project yourself, you’ll need to create a packages.json file, delcare your dependencies, and run npm install, then you can create the app.js and start developing. You can find more info on how to do that on the express documentation.

Creating the short URL

When the user first lands on the site they are present with a form where they can shorten a URL. As we can see from this line in the main.js file the index route is displayed for requests on the base URL:


app.get('/', routes.index);

This refers to the index.js file in the routes directory, which in turn renders a view back to the client, by doing the following:


exports.index = function (req, res) {
 console.log('Displaying index page where users can enter a long url')

res.render('index');
};

Node automatically knows that it needs to render a jade view, because we told it earlier where the views are located, and what view engine to use when we declared these in our main.js:


app.set('views', path.join(__dirname, 'views'));
 app.set('view engine', 'jade');

A quick peak at our index jade file and we can see that we have a very basic form that accepts a single text field and posts this to the /create endpoint


extends layout

block content
 h1= title
 h1.text-center Node.js URL Shortener

div.container
 div.content
 div.well
 form.form-horizontal(name="input", action="/create", method="post")
 div.form-group(align='center')
 p
 input(type="text", name="urlToShorten", class="form-control", placeholder="Enter a URL")
 p
 input(type="submit", value="Shorten", class="btn btn-primary")

Diving back to our main js file, we can see that requests (specifically POST requests) are handled by the createShort function in the shorty.js file:


app.post('/create', shorty.createShort);

This is where the magic happens. Firstly, we need to get hold of the URL that the user submitted in the form, that is relatively easy as we can get it directly off the request body, as shown in the below code snippet. Next we do a quick check to see if its null and display a message to the user if they didn’t provide a URL. If they have provided a URL, we check to see if we need to prepend it with http://., then we create a shortcode for that URL and render a page that will display the link for the user to share.


/*
 * POST creates a short url from a long url
 */
 exports.createShort = function (req, res) {

var urlToShorten = req.body.urlToShorten;
 if (!urlToShorten) {
 console.log('Request did not contain a url to shorten, please provide urlToShorten');
 res.render('short', {message: 'Request did not contain a url to shorten, please provide urlToShorten'});
 } else {

console.log("Request to shorten " + urlToShorten);

urlToShorten = addhttp(urlToShorten);
 var baseUrl = 'http://' + req.app.get('hostname') + '/';

var shortCode = createShortCode(urlToShorten);
 res.setHeader('Content-Type', 'text/html');
 res.statusCode = 200;
 res.render('short', { shortUrl: baseUrl + shortCode });
 }
 };

The reason for prepending the protocol onto the link, is because this is required for the Location header on a redirect which we will serve when the short URL is requested, as mentioned in the HTTP RFC spec, this needs to be an absolute URI. This is a bit of a dirty hack and I’m not that happy with it. Ideally, I would like to create a URL object with the string the user has supplied, regardless of what prefix it has, I was hoping that the node url functions would help with that, but it seems that when parsing and then formatting the URL, it doesn’t retain the protocol. It is also entirely possible that I’m not using the API correctly, perhaps someone can comment, or suggest a better mechanism for this.

To create the short code, we generate a random alphanumeric string of 5 characters, this is quite a simple function. We then add the short code and URL to a collection so we can look them up when the user requests the short code.

Redirecting to the original URL

Looking back at the main js file, we can see another route. This is used when a user requests their short url.


app.get('/:short', shorty.getLong);

Firstly, we grab the short code from the request path. We have to strip the first character as that will be a slash, such as “/ABC123”. Then we find the long URL that the short code refers to by looking in the collection where we stored it earlier. After that, its just a case of writing a 302 redirect response with the Location header set to the long URL, thats it really.


/*
 * GET retrieves long url from short url
 */
 exports.getLong = function (req, res) {

// grab the path and strip the leading slash
 var shortCode = req.path.substring(1);

console.log("Fetching URL indexed by " + shortCode);
 var theLongUrl = shortToLong[shortCode];

console.log('Short code ' + shortCode + " refers to " + theLongUrl);

console.log("redirecting to " + theLongUrl);
 res.writeHead(302, {'Location': theLongUrl});
 res.end();
 };

Conclusion

Whilst I don’t have much of a background in front end web development, my javascript skills are pretty basic, saying that, node js is quite intuitive and the documentation is good enough to get you through the basics. Node has quite a small learning curve (at least for the basics), coupled with StackOverflow for reference, you can build web apps relatively quickly and easily.

Would I use Node again? Absolutely, however I think it’d be far better suited to projects requiring a substantial number of concurrent users, such as web chat applications, or for gaming backends. For building web applications I’d probably choose Grails as I’m more familiar with Java and Groovy, plus you get all the benefits of Spring. Scaffolding is also great.

Node would be a great choice for building development stubs, I’ve built several mobile apps that require backends, for testing, you want a stubbed backend that returns canned responses. You can do this incredibly easy in node by using returning static files served up as JSON responses, plus its very lightweight so you can integrate into your CI nicely.

You can see a live demo of this on my cloudbees account here.

Resources

Robots! Part 2, the android client

Continuing on from my previous post, I’ve created an android client that I can use to send commands to my python server.

Ultimately I want to be able to control the robot remotely, the best way to do this would be to control the robot from a tablet or a phone which communicates wirelessly with the pi via bluetooth or wifi. In my previous post I described setting up a python application that will run on the raspberry pi and listen for commands. All I needed to do was to create a very basic android interface that can send commands to the raspberry pi.

The robot I intend to build will be based on tracks instead of wheels, there are many benefits to this but the most significant is that from an engineering perspective is that its much easier to build. A car needs forward and backwards drive, but also sideways drive for the front axle. In my opinion, it is far simpler to have a tracked vehicle with a motor controlling each side. When both motors are turned in the same direction the vehicle moves forward or backwards, and when the motors run in opposite directions the vehicle will turn on the spot.

My app interface mimics the layout of the vehicle itself, with an up and down button on the left and right hand side of the screen, as shown below.

Arrow buttons for controlling robots tracks
Arrow buttons for controlling robots tracks

You can see the code for this layout on the github repo here.

I want the user to be able to hold a button and the motor will run until they take their finger off. For this I’ve attached listeners on the buttons that will listen for the key up and key down events. It will send separate events for starting and stopping the motors, like so:

        Button leftForward = (Button) findViewById(R.id.leftForward);

        leftForward.setOnTouchListener(new View.OnTouchListener() {
            @Override
            public boolean onTouch(View view, MotionEvent motionEvent) {
                switch (motionEvent.getAction()) {
                    case MotionEvent.ACTION_DOWN: {
                        sendCommand(leftForwardCommand + "-Start");
                        break;
                    }
                    case MotionEvent.ACTION_UP: {
                        sendCommand(leftForwardCommand + "-Stop");
                        break;
                    }
                }
                return false;
            }
        });

As the sendCommand needs send a message over the network, I need to take this off the main UI thread otherwise I’d get an exception such as:

android.os.NetworkOnMainThreadException

To take this off the UI thread, I simply move the sending of the command into an AsyncTask, like so:

private void sendCommand(String command) {
        new SendCommandTask().execute(command);
    }

    class SendCommandTask extends AsyncTask<String, Void, Void> {

        @Override
        protected Void doInBackground(String... commands) {
            String command = commands[0];
            try {
                //TODO: make this configurable inside the app
                Socket socket = new Socket("192.168.0.6", 3033);
                PrintWriter out = new PrintWriter(new BufferedWriter(new OutputStreamWriter(socket.getOutputStream())), true);
                out.println(command);

                Log.d(TAG, "Successfully sent " + command);
            } catch (IOException ioe) {
                Log.d(TAG, "Unable to send command", ioe);
            }
            return null;
        }
    }

Now if I run the python server and then send press the buttons in the app, I see this output from the python application.

Server listening on port 3033...
...L-FORWARD-Start…..L-FORWARD-Stop….

Thats all for now, you can access this code on my github repository. Next post will either be in relation to building the robot, or using the BrickPi APIs.

Robots! (part 1)

Inspired by the android controlled lego robots I saw at DroidCon UK this year, and with difficulty finding a use for my raspberry pi, I’ve decided to have a go at building a robot that I can control via an android app. Having a 24 hour flight home from Australia at the weekend, I’ve had plenty of time to think about how I might approach this task (or challenge as I refer to it as I’ve no prior experience with robotics / socket programming).

Lego robot powered by the BrickPi

My plan is to have a python socket server running on the pi. This will provide a socket that an android client can invoke commands on. This python server will also interact with the python scripts that the BrickPi uses to control lego motors.

Whilst most of my experience revolves around Java, I’ve opted for python for the following reasons

  • Python is supported on the pi out of the box, no need to mess around with installing Java
  • The BrickPi has support for Python (and C)
  • I feel like learning something new

Baby steps…

Starting simple, I thought it best to create a simple script that listens on a socket, and then create a client that sends it some data to print out to the console. Once I have this working I can expand on it and make the client more sophisticated (an android app for example) and also enhance the server so it can handle different types of commands.

Server
There are plenty of example python scripts online, I found a good one here and tweaked it slightly to remove the parts I don’t want. (full credit to pythonadventures!)

#!/usr/bin/env python
# server.py

import socket
import select
import Queue
from threading import Thread
import sys

class ProcessThread(Thread):
    def __init__(self):
        super(ProcessThread, self).__init__()
        self.running = True
        self.q = Queue.Queue()

    def add(self, data):
        self.q.put(data)

    def stop(self):
        self.running = False

    def run(self):
        q = self.q
        while self.running:
            try:
                # block for 1 second only:
                value = q.get(block=True, timeout=1)
                process(value)
            except Queue.Empty:
                sys.stdout.write('.')
                sys.stdout.flush()
        #
        if not q.empty():
            print "Elements left in the queue:"
            while not q.empty():
                print q.get()

t = ProcessThread()
t.start()

def process(value):
    print value

def main():
    s = socket.socket()
    host = socket.gethostname()
    port = 3033
    s.bind((host, port))

    print "Server listening on port {p}...".format(p=port)

    s.listen(5)                 # Now wait for client connection.

    while True:
        try:
            client, addr = s.accept()
            ready = select.select([client,],[], [],2)
            if ready[0]:
                data = client.recv(4096)
                t.add(data)
        except KeyboardInterrupt:
            print
            print "Stopping server."
            break
        except socket.error, msg:
            print "Socket error %s" % msg
            break

    cleanup()

def cleanup():
    t.stop()
    t.join()

if __name__ == "__main__":
    main()

The socket is bound to a port, and then continually listens for incoming data. Once some data is received, it is added onto a queue, which is then sequentially executed.

Client
The client is is fairly straightforward, it involves opening a connection on a socket and writing data to it, a few lines of Java code.

public static void main(String[] args) throws IOException {

        InetAddress address = InetAddress.getLocalHost();
        Socket socket = new Socket(address, 3033);

        BufferedOutputStream bos = new BufferedOutputStream(socket.getOutputStream());

        OutputStreamWriter osw = new OutputStreamWriter(bos, "US-ASCII");

        System.out.println("Sending message...");

        osw.write("Hello!");
        osw.flush();
    }

Thats it for now, you can checkout my code on github (or just copy/paste the above) and run the client and server and see it in action.

I’ll start on the client next..

Selecting partial elements of mongo db documents

By default, when you find documents on mongo it returns the entire document, similar to what SELECT * FROM TABLE; would do in a relational world.

> db.user.find().pretty()
{
	"_id" : ObjectId("5217705cc2e6fc761c0b5371"),
	"userId" : "james",
	"age" : 27,
	"favouriteVideoGame" : "Skyrim"
}

You can retrieve partial document info by using something known as a projection (see mongo docs here). Use find() with an empty query, but specify the fields you want from the document like so:

> db.user.find({}, {userId:true})
{ "_id" : ObjectId("5217705cc2e6fc761c0b5371"), "userId" : "james" }

> db.user.find({}, {userId:true, age:true})
{ "_id" : ObjectId("5217705cc2e6fc761c0b5371"), "userId" : "james", "age" : 27 }

For the Java devs using Spring data, you can do a similar thing using the Query APIs, notably the fields().include() (docs here):

Query q = Query.query(Criteria.where("userId").is(userId));
q.fields().include("age");

User score = mongoTemplate.findOne(q, User.class, "user");

What I thought of “Instant Android Fragmentation Management How-to”


I was recently contacted by the marketing team from Packt Publishing and asked if I could write a review for one of their new ebooks, so here it is!

This book gives a brief yet comprehensive overview of some of the processes you can follow to improve your applications chance of working on the varying versions of android that are currently in existence.

Having developed apps that work perfectly fine on my Galaxy S2 2.3.3, only to find they don’t work on a hardware identical S2 with 4.0, I certainly feel the pain of OS fragmentation!

The book discusses some of the approaches you can take using the android compatibility library, fragments, loaders and action bars that enable you to write against the latest APIs but still have a good chance of having your app work on previous versions.

Things I liked about this book

  • A short and focused read, I got through it in around an hour.
  • Lots of code examples and references to external reading material.
  • The reference to ActionBarSherlock I felt was very much needed, as always if there are open source alternatives that work better, we should adopt them rather than struggle!

Things I didn’t like

  • I did feel that having some more screenshots would have helped emphasised the topic in discussion, whilst there were some screenshots throughout I did constantly find myself thinking “what should this actually look like?”
  • The chapters could have done with a more expanded overview, particularly the loaders section as it jumped right into creating one before explaining what it is, and how it differs from other non-UI asynchronous mechanisms like async tasks.

All in all I rate this as 4/5. The ebook is very reasonably priced and for the price of a few beers it’d certainly worth it, even for the discussions on ActionBarSherlock and Fragments. If your struggling to support multiple android versions, this book is a good starting point for you.

A few little tweaks to automate Android instrumentation testing via Robotium & Maven

Having recently revived an android project I haven’t opened in close to 6 months, I was left scratching my head as to why I couldn’t run any of my integration tests.

Thinking back, I remembered having problems getting robotium to instrument the clicking of a button, as simple as it sounds, theres a few little gotchas involved. Firstly, I needed to modify the test code to click on text, rather than a button, as it has been suggested by various users on StackOverflow that there does tend to be odd side effects when clicking buttons:

public void testClickGetSurveyTakesToSurveyRunner() {
 solo.clickOnText("Get Survey");
 solo.assertCurrentActivity("Current activity was not correct", SurveyRunner.class);
 }

Next, I needed to add this into the test projects AndroidManifest

<uses-sdk android:targetSdkVersion="10" />

And then to add this into the main projects AndroidManifest:

<supports-screens android:anyDensity="true"/> 

Both of these tweaks are mentioned on the Robotium FAQ, so it might be worth glancing there to see if any enhancements get made in the future

After making those changes, I stumbled into another error that reared its ugly head in the logcat:

ERROR/AndroidRuntime(1185): FATAL EXCEPTION: main
 java.lang.IllegalAccessError: Class ref in pre-verified class resolved to unexpected implementation
 at com.roosearch.android.activity.SurveyRunner.onResume(SurveyRunner.java:38)
 at android.app.Instrumentation.callActivityOnResume(Instrumentation.java:1150)
 at android.app.Activity.performResume(Activity.java:3832)
 at android.app.ActivityThread.performResumeActivity(ActivityThread.java:2110)
 at android.app.ActivityThread.handleResumeActivity(ActivityThread.java:2135)
 at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:1668)
 at android.app.ActivityThread.access$1500(ActivityThread.java:117)
 at android.app.ActivityThread$H.handleMessage(ActivityThread.java:931)
 at android.os.Handler.dispatchMessage(Handler.java:99)
 at android.os.Looper.loop(Looper.java:130)
 at android.app.ActivityThread.main(ActivityThread.java:3683)
 at java.lang.reflect.Method.invokeNative(Native Method)
 at java.lang.reflect.Method.invoke(Method.java:507)
 at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:839)
 at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:597)
 at dalvik.system.NativeStart.main(Native Method)

There seems to be a rather peculiar requirement, that any dependencies your using on your main application, also need to be in the test projects pom.xml, but marked with a scope of provided. After making that change as advised here and rebuilding, both the app and instrumentation app build and test successfully.

I was also pleasantly surprised that I could run the instrumentation tests via IntelliJ IDEA 12 Ultimate, last time I tried that (on an older version albeit), it wasn’t so simple…

Bootstrapping a Grails application to prepopulate data on startup

I’ve been dabbling in a little Grails recently, and I found it quite frustrating having to re-enter some sample data everytime I restarted my application, in order to have the views look meaningful.

Fortunately, theres an easy solution, just do all of your setup in the Bootstrapper class like so :

class BootStrap {

def init = { servletContext -&gt;

// Keep references on these as we'll use them for populating sample surveys
Response great = new Response(text: "Great").save()
Response average = new Response(text: "Average").save()
Response poor = new Response(text: "Poor").save()
Response tasty = new Response(text: "Tasty").save()
Response ok = new Response(text: "OK").save()
Response horrible = new Response(text: "Horrible").save()
Response friends = new Response(text: "Friends").save()
Response advert = new Response(text: "Advert").save()
Response radio = new Response(text: "Radio").save()

// These are some other responses that we'll make available in the response bank
["Excellent", "Good", "Satisfactory", "Bad", "Very bad", "true", "false", "Yes", "No", "Maybe"].each {
new Response(text: it).save()
}

Question q1 = new Question(text: "What did you think of the service?", responses: [great, average, poor]).save()
Question q2 = new Question(text: "Was the food nice?", responses: [tasty, ok, horrible]).save()
Question q3 = new Question(text: "How did you hear about our establishment?", responses: [friends, advert, radio]).save()

Survey cosmos = new Survey(title: "COSMOs customer feedback", questions: [q1, q2, q3]).save()
Survey jimmys = new Survey(title: "Jimmys Kitchen customer feedback", questions: [q1, q2, q3]).save()

new User(firstName: 'James',
lastName: 'Elsey',
emailAddress: 'james.elsey.dev@gmail.com',
companyName: 'Jimmys Kitchen',
surveys: [jimmys])
.save()

new User(firstName: 'Manabu',
lastName: 'Takano',
emailAddress: 'manny@hotmail.com',
companyName: 'COSMOs Cardiff',
surveys: [cosmos])
.save()

}
def destroy = {
}
}

Then, every time you start the application, you can be sure that it has the above data populated.

Probably not ideal for production apps, but very useful in development

Deploying a grails application to Cloudbees

I spent much of last weekend experimenting with the grails framework, so I wanted to deploy what I had in the cloud. Theres a big tutorial on the IntelliJ IDEA documentation (the screenshots don’t seem to match my installation of IDEA, even though its the same version number).

There is a much easier way, providing you have the Cloudbees SDK installed, you can just run this one-liner :

grails war; bees app:deploy target/MyWarFile.war -a mycloudbeesusername/applicationcontainernamehere

The above will package the application as a war file, and then deploy it to your Cloudbees instance.

Obviously the IDE will bring some additional features to the deployment, but if you don’t care about that and just want to upload, the one liner wins hands down.

Override the toString on your domain classes so they display as something useful

If you’re finding that your domain objects are not being displayed in a readable manner, chances are its because they haven’t been told to. This is often the case in drop down menus that the grails scaffolding creates. You can easily fix this by overriding the toString method on your domain class, such as the following.

class Question {

    String text

    static hasMany = [responses: Response]

    static constraints = {
        text()
        responses()
    }

    String toString(){
        return text
    }
}