Jrod Blog

Automatically updating appengine-web.xml's version

2011-11-15T17:13:00.001-08:00

If you're using AppEngine in Java land, you're probably using maven with maven-gae-plugin. If you're not, why aren't you?

Anyway, since you already have a project version, why not leverage that in appengine-web.xml's version tag?


<build>
  <plugins>
   <finalName>${project.artifactId}</finalName>
   <!-- Sets the version in appengine-web.xml -->
    <plugin>
      <groupId>com.google.code.maven-replacer-plugin</groupId>
      <artifactId>maven-replacer-plugin</artifactId>
      <version>1.3.9</version>
      <executions>
          <execution>
        <phase>package</phase>
        <goals>
            <goal>replace</goal>
        </goals>
          </execution>
      </executions>
      <configuration>
          <file>target/${project.artifactId}/WEB-INF/appengine-web.xml</file>
          <replacements>
        <replacement>
            <token>%%VERSION%%</token>
            <value>${project.version}</value>
        </replacement>         
          </replacements>
      </configuration>
    </plugin>
  </plugins>
</build>


<appengine-web-app xmlns="http://appengine.google.com/ns/1.0">
  ...
  <version>%%VERSION%%</version>
  ...
</appengine-web-app>

And that's it. When you execute gae:run, gae:deploy, etc, the version will be set automatically.

Call Me CAPTCHA Launched!

2011-11-07T12:33:00.000-08:00

I launched my new service last night, Call Me CAPTCHA. I've been hacking on this project for a little while in my spare time.

Instead of trying to decipher more and more distorted random letters in a CAPTCHA image, why not leverage a system that's already fairly secure, ubiquitous, and easy to use: your phone. The distorted image CAPTCHA is already effectively broken, spammers can either use CPU cycles to break these or they can use humans for it. Yep, you can get paid to break CAPTCHAs; if you're running a bot, that's $2.00 per 1,000 CAPTCHAs.

You've probably seen a phone-based CAPTCHA before. The first instance I can remember is MySpace, then Google, now sites like Bank of America's with "SafePass". Well, I'd call these companies "the big boys". But what if you're a smaller site that can't afford to build a system like this, or even a large site that would rather use an off-the-shelf service? There weren't really any options until Call Me CAPTCHA.

If you've used reCAPTCHA before, you'll notice a lot of similarities; the APIs are very similar. reCAPTCHA is wildly popular, so I used them as a model for my system.

So if you're unhappy with the protection afforded by image-based CAPTCHAs, and need something more robust and easier to use, head on over and check out Call Me CAPTCHA.

Google+ is Killer

2011-07-12T21:38:00.000-07:00

It's very easy to group your friends in different groups. Groups are like the ubiquitous "label", but for your friends. Group friends by degree (best, good, proxy, acquaintance, etc), or by classification (family, extended family, friends, professional), or even by interest (hiking, photography, school)

I have a lot of friends that I would like to keep up with, but I don't have time to. Some of these are old circles of friends from high school, or an old job, or my long distance best friends. I touched base with two different groups today, people that I really like, but haven't talked to or seen for months. Hangouts make this really easy. Hangouts are basically like private chatrooms with video meets social networking. I can foresee hangouts becoming incredible tools for all kinds of things:

Townhalls. I watched host the first townhall on Google+. If Google can really go to town with the Hangout technology, I can see the digital townhall becoming an incredible political tool. Imagine the power of being able to organize an actual town hall (moderation tools and all) for absolutely no cost.

Hangouts could also be chat rooms. A really good example already exists; User Plane. It's basically IRC meets Hangouts.

If posts become rich-text enabled with a WYSIWIG editor, the "Public" visibility option could become a good blog replacement.

With the Android phone application, it's also an excellent microblogging platform.

Google+ is a great new social tool, and if Google can play it's cards right, it could be the next killer social app.

Update: Actually, now that I think about it. Google+ is what Google Wave should have been.

Update 2: Circles are another powerful aspect of Google+. Now I can easily use the same social platform for my professional, and private faces. Google+ is a threat for Facebook and LinkedIn.

RESTful JSON web services with JSOG and Spring

2011-06-03T09:21:00.000-07:00

Before I originally developed JSOG, we were using the json.org library and Restlet. Our needs were pretty basic, we were just GETting and POSTing JSON strings between services. Later we ended up using some JSONP.

My biggest beef with Restlet is there doesn't seem to be a good way to unit test it. From what I can tell, the convention is to keep all your logic outside of Restlet and use it only to expose the REST interface. I'm also a heavy spring user, and it occurred to me very quickly that Restlet's integration with Spring was... complicated.

So, instead of jumping through Restlet's hoops, JSOG now integrates very easily with Spring MVC. Note, there are two ways of integrating JSOG with Spring. We're going to be using the "Flexible" approach, and we'll be using a JSON-specific DispatcherServlet that handles all *.json requests. This basic example should show you how add RESTful services to your application.

First, we'll need the dependencies. I'm using Maven; so here's what a minimal POM looks like.
pom.xml


<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <groupId>jsogws</groupId>
    <artifactId>jsogws</artifactId>
    <packaging>war</packaging>
    <version>1.0-SNAPSHOT</version>
    <name>jsogws</name>
    <url>http://maven.apache.org</url>
    <properties>
        <spring.version>3.0.5.RELEASE</spring.version>
    </properties>
    <dependencies>
        <dependency>
            <groupId>net.sf.jsog</groupId>
            <artifactId>jsog</artifactId>
            <version>0.30</version>
        </dependency>
        <dependency>
            <groupId>commons-io</groupId>
            <artifactId>commons-io</artifactId>
            <version>2.0</version>
        </dependency>
        <dependency>
            <groupId>org.springframework</groupId>
            <artifactId>spring-core</artifactId>
            <version>${spring.version}</version>
        </dependency>
        <dependency>
            <groupId>org.springframework</groupId>
            <artifactId>spring-context</artifactId>
            <version>${spring.version}</version>
        </dependency>
        <dependency>
            <groupId>org.springframework</groupId>
            <artifactId>spring-webmvc</artifactId>
            <version>${spring.version}</version>
        </dependency>
        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
            <version>3.8.2</version>
            <scope>test</scope>
        </dependency>
    </dependencies>
    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>2.0.2</version>
                <configuration>
                    <source>1.6</source>
                    <target>1.6</target>
                </configuration>
            </plugin>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-war-plugin</artifactId>
                <version>2.1-beta-1</version>
                <configuration>
                    <failOnMissingWebXml>false</failOnMissingWebXml>
                </configuration>
            </plugin>
        </plugins>
        <finalName>jsogws</finalName>
    </build>
    <repositories>
        <repository>
            <id>jsog.sourceforge.net</id>
            <url>http://jsog.sourceforge.net/m2-repo</url>
        </repository>
    </repositories>
</project>

Well that's a fair chunk of configuration. Basically, we're importing JSOG from the JSOG maven repository. We also import the relevant Spring artifacts and Commons IO.

Next up, we need to configure the DispatcherServlet.

web.xml


<?xml version="1.0" encoding="UTF-8"?>
<web-app xmlns="http://java.sun.com/xml/ns/javaee"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/web-app_3_0.xsd"
  version="3.0">
    
    <!-- JSOG web services -->
    <servlet>
        <servlet-name>jsog</servlet-name>
        <servlet-class>org.springframework.web.servlet.DispatcherServlet</servlet-class>
        <load-on-startup>0</load-on-startup>
    </servlet>
    <servlet-mapping>
        <servlet-name>jsog</servlet-name>
        <url-pattern>*.json</url-pattern>
    </servlet-mapping>

</web-app>

Here we map every *.json request to the DispatcherServlet named "jsog".

Now we need to configure the dispatcher servlet. We used the default configuration file name, in this case:
jsog-servlet.xml


<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:context="http://www.springframework.org/schema/context"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xsi:schemaLocation="
       http://www.springframework.org/schema/beans
       http://www.springframework.org/schema/beans/spring-beans-3.0.xsd

       http://www.springframework.org/schema/context
       http://www.springframework.org/schema/context/spring-context-3.0.xsd">

    <!-- Look for controllers -->
    <context:component-scan base-package="jsogws.services.json"/>

    <!-- All controllers managed by this servlet should use the JsogViewResolver -->
    <bean id="viewResolver" class="net.sf.jsog.spring.JsogViewResolver"/>

    <!-- We need this to consume JSOG that has been sent in the request body -->
    <bean class="org.springframework.web.servlet.mvc.annotation.AnnotationMethodHandlerAdapter">
        <property name="messageConverters">
            <list>
                <bean class="net.sf.jsog.spring.StringJsogHttpMessageConverter"/>
            </list>
        </property>
    </bean>

</beans>

In this configuration file, we tell the DispatcherServlet to look for "@Controller" annotated classes in jsogws.services.json. Then we setup the JsogViewResolver, which tells Spring to create and use a JsogView whenever we return a JSOG object in a controller. (JsogView does the lion's share of the work). We also setup a MessageConverter so we can consume JSON.

Note that at this point, most of this is pretty standard Spring stuff. The MessageConverter and ViewResolver are the most "abnormal" things in here: about 8 lines of boilerplate configuration.

From this point, creating a JSON-based RESTful service is just about like creating any other RESTful service.

Let's take a look at a Hello World example:

HelloJsog.java


package jsogws.services.json;

import net.sf.jsog.JSOG;
import org.springframework.stereotype.Controller;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestMethod;

@Controller
public class HelloJsog {

    @RequestMapping(value="/hello.json", method=RequestMethod.GET)
    public JSOG hello() {
        return JSOG.object("Hello", "World!");
    }

    @RequestMapping(value="/hello.json", method=RequestMethod.POST)
    public JSOG hello(@RequestBody JSOG request) {
        String name = request.get("name").getStringValue();
        return JSOG.object("Hello", name);
    }

}

This class defines two methods:
One accepts a GET request and responds with an object: {"Hello", "World!"}.

The other accepts a object via POST, that contains a "name" property. It returns an object with the name: {"Hello", "foo"}.

That's all there is to creating RESTful JSON web services with JSOG and Spring. If you're wondering "what's RESTful about this?" Check out Spring's @PathVariable annotation. Wiring the rest together is elementary.

A quick gotcha: When you send data to your services, you need to use the content type "application/json" and the encoding ISO-8859-1. These are modifiable through properties on the JsogViewResolver and StringJsogHttpMessageConverter.

You also get JSONP for free. Append ?callback=foo and "foo" becomes your callback method. The name of this parameter is settable in JsogViewResolver, but will work with jQuery by default.

Now your web services use the same framework style as everything else in your application, and can easily be unit tested.

Download
jsogws.tgz

A trip down the number line

2011-05-26T11:14:00.001-07:00

This was written by the Fark.com user Hector_Lemans, 2009-09-12 09:43:17 PM

I've tried to track it down online, perhaps it was an article from somewhere else, but it looks like it's original. I'm cleaning out my Google notebook, and wanted to keep it here for posterity.

I've always been fascinated by large numbers. As a child, I would occasionally play the "one-up" game with my friends. Perhaps you've played this game. Two kids disagree about something and one says "bet you a hundred bucks it's not!" The other one retorts "bet you a million bucks it is!" And so on into infinity...which we usually got to rather quickly. But there are quite a few numbers between 100 and infinity. I'd like to take you, dear reader, on a trip to visit a handful of those numbers in the hope that you too can share in my wonderment of big numbers. So grab a power-bar, put on some running shoes, and let's begin.

Our journey will be entirely on the number line. As you may recall from grade school, the number line is a straight line with the counting numbers ticked off at regular intervals.

We start at point zero and begin moving straight ahead. We haven't worked out in a while so we're kind of slow off the starting blocks, but fairly quickly we pass 1, then 2, then 10, then 20. Now we're picking up some speed! As we pass 100, we notice that about a quarter of the numbers we just passed by were bold. As it turns out, we are on a special number line that has all the prime numbers in bold. Prime numbers are numbers that can only be evenly divided by 1 and themselves. So, for instance, 37 is a prime, because there is no other number we can divide it by (besides 1 and 37) that will give us another whole number. The fact that our number line bolds all of these primes is merely a curiosity...for now.

Fairly quickly, we pass 1,000; then 10,000; then 100,000. Now we pass the smallest of our big numbers: one million. A million dollars is what most people define as rich. Even a million pennies add up to $10,000 - not exactly chump change. You would need to flip a coin a million times before having a fair chance of getting 20 tails in a row. In other ways, though, one million isn't that big at all. One million seconds is just over 11 days. A million feet is only 190 miles - a few hours trip in the car. Ten good-sized novels contain a million words. Let's continue on.

We pass 5 million, 50 million, 500 million, and still we are gathering speed. By the time we hit one billion, barely 5% of the numbers we pass are bold prime numbers. A billion seconds is 32 years. A billion feet is 190 thousand miles - that's over three-quarters of the way to the moon! On the other hand, a large drop of blood has about a billion red blood cells in it. Several thousand public companies in the US have a market capitalization of over a billion dollars. So let's continue on in our search for big numbers.

50 billion, 500 billion, one trillion. Now we're approaching the high end of any numbers that come up in everyday conversation. The US debt stands at almost five trillion dollars. One trillion seconds is over 31,000 years - far older than recorded history and deep in the middle of the last ice age. One trillion feet is approximately the diameter of the earth's orbit around the sun. One light-year (the distance light travels in a year) is about six trillion miles. Alpha Centauri, our nearest neighboring star, is about 4.4 light-years distant. Going there in the space shuttle would take about 38,000 years. You would need to flip a coin at least a trillion times to have a fair chance of getting 40 tails in a row...but at 3 seconds a flip, that would take about 95,000 years. Think 3 seconds a flip is too fast? Take 5 seconds a flip and you're up to 160,000 years. On the other hand, one trillion drops of water is only a few hundred thousand gallons - a big pond. Most laptop computers can perform a trillion calculations in under half an hour. The average person is made up of about 50 trillion cells. Onward we go.

We pass 100 trillion and on to a quadrillion. It's hard to make out the numbers as they whiz past us, but a bit less than 3% - every 30th number or so - is a prime. That's interesting. Going from zero to a million, the concentration of primes dropped by over 95%, but going from a million to a quadrillion drops the concentration down only two more percent. Obviously, the primes are thinning out, but at a greatly decreasing rate. Will we soon pass by the last one? Let's find out.

We pass 100 quadrillion, then make our way into the quintillions. Our nearest neighboring galaxy, Andromeda, is about 15 quintillion miles away. We speed up a bit and find ourselves in the sextillions. At least, that is their official name. Another, more helpful way of visualizing these numbers is by scientific notation. Using this notation, one sextillion is 10^21. The little superscript 21 to the right of the 10 means we multiply the 10 by itself, 21 times. So 1,000 would be 10^3 (10 x 10 x 10), and 1,000,000 would be 10^6. Notice that the superscript also tells us how many zeros are in the number if written out in full.

We pass the sextillions by and move on into the septillions (10^24). From high school chemistry class, you may recall a unit of measurement called the mole. One mole of any element has the same number of atoms: 6 x 10^23, or just under one septillion. One mole of gold is a bit less than half a pound. One mole of hydrogen gas weighs only a gram. Coincidently, the visible universe is also about 6 x 10^23 miles across, or 93 billion light-years. The earth weighs (if you could weigh the earth!) 10 septillion pounds.

Fast and faster we go. The number line is a blur of gigantic numbers, with about every 100th one in bold. I say "about" because there doesn't seem to be any real pattern to how the prime numbers are spaced out - sometimes a couple primes will be within two or three numbers of each other, and other times there will be thousands of numbers between two consecutive primes. No matter, onward we must go.

In no time at all, we are deep in the octillions (10^27). The human body contains approximately 7 x 10^27 atoms. If we divide that by the 50 trillion cells we estimated earlier, we find the average human cell contains over 100 trillion atoms. Think about that - each cell in your body is constructed of more atoms than there are bricks in the Great Wall of China!

We put on a burst of speed and race past the next several large numbers. Nonillion (10^30), decillion (10^33)...the names don't mean much anymore. It's easier to just go by the scientific notation. We move on past 10^50 - approximately how many times you'd have to flip a coin to have a good shot at getting 150 tails in a row. As we move up to 10^70 and beyond, we start approaching the limits of anything physical these gigantic numbers can be compared to. The known universe has 10^80 atoms (give or take). If we assume the universe is a sphere, 93 billion light-years in diameter, that amounts to less than one atom per cubic foot! We live in a universe of mostly empty space.

Huffing and puffing, we finally make it to a truly, incredible, gi-normous number: a googol. A one followed by a hundred zeros - 10^100. The name "googol" was coined by a professor's nine-year-old nephew when asked to think up a name for a really big number. Have we run out of primes yet? Not hardly - less than half a percent of the numbers we run across are primes, but that still works out to 1 out of every 230, on average. Now, take a deep breath and scarf down that power bar. To reach our last few big numbers, we're going to have to sprint.

We blow past a googol and make our way up to 10^124. This is the number of particles you'd have if the universe were nothing but a solid mass of electrons - no empty space at all. We run past 10^155 and bump into some numbers used every day by millions of people! Notice how random those bold prime numbers always appear to be, no matter how far across the number line we go? Turns out, that randomness makes them perfect for encoding sensitive data - like a credit card number - and transmitting that information over the Internet. The actual process is quite complicated (as you might expect), but the essential steps are as follows: Two very large prime numbers are chosen and kept secret by the bank. These two primes are then multiplied together to form a huge composite number. This composite number is given to the person with the credit card, and is used to encode the credit card information, which is then sent back to the bank. The bank, having the original two primes the composite number came from, factors them out and retrieves the data. The beauty of the system is that the encoding using the huge composite number is a one-way process if you don't know the primes it is made of - factoring composite numbers with upwards of 155 digits is next to impossible, even with the fastest computers. So it doesn't matter if someone figures out the composite number you're using, they still can't decrypt an encoded message.

From here on out, most of the interesting numbers we'll run across will be prime numbers. Mathematicians like to find and collect huge primes, like some sort of super nerdy baseball card collector. The late Samuel Yates collected what he called Titanic primes - prime numbers with more than 1,000 digits. Chris Caldwell, of the University of Tennessee, took up the torch after Yates' death in 1991 and stores a list of these large primes on the Internet, with a new category for prime numbers with more than 5,000 digits called Gigantic primes. Several of these primes are quite strange. One, that has 5,114 digits, is composed of only 1s and 0s. Another, with 6,400 digits, is all 9s except for one 8. Currently, the largest known prime, discovered on September 4th, 2006 is just shy of ten million digits long. As we reach it on the number line, we marvel at its size. Written in exponential notation (similar to scientific notation but the base number doesn't have to be ten), the current largest prime number is 2^32,582,657 - 1.

And now, let us accelerate to a speed only possible in our mind's eye. Numbers blur together, numbers so big that just writing them down in full would be a problem in the real world...not enough paper! On and on we go. There it is, just up ahead - the number that will end our journey. We slow down from our impossibly fast pace and stop on the number line directly over this stupendous number. A googolplex - a one followed by a googol zeros. If the whole universe were nothing but paper and ink, you still couldn't even write down the number in full (never mind trying to imagine how big it is). This is the end of our journey, but we can look ahead and see even bigger numbers...a tiny select few are even in bold. We still haven't run out of primes! Do the primes ever end? Does this number line ever end? Does infinity really exist? Just because we can say a thing doesn't make it real or even possible - I can describe a universe in which 2 + 2 = 5 or a square is round, but that doesn't mean a place like that could exist...does it? If we squint our eyes and look really hard, we can see a number so large that no number appears to follow it. It's a number so large we can't even describe it in words. Is that the last number? Only one way to find out. Let's go...last one there is a rotten egg!

Nexus One Debugging

2011-03-25T11:47:00.000-07:00

I was having trouble getting ADB from the Android SDK to recognize my phone under Windows 7, here's how I fixed it.

1. Start the Android SDK and AVD Manager.
2. Select "Available packages" on the left.
3. Navigate to: Third party Add-ons > Google Inc. > Google USB Driver package
4. Click "Install Selected" and proceed through the dialogs.
5. Connect your phone
6. Start the "Device Manager" management application, and locate your phone.
7. It should say it's not recognized, right click on your phone and click "Update Driver Software".
8. Click "Browse my computer for driver software"
9. Navigate to the android SDK directory ...\extras\google\usb_driver
10. Proceed through the dialogs.

For me, the driver is located in:
C:\Program Files (x86)\Android\android-sdk\extras\google\usb_driver

I then ran adb devices, and my phone was listed.

Netflix's Phone-only support

2011-03-15T13:51:00.001-07:00

Until recently, I had the 6-at-a-time Netflix account. I've become rather busy and just don't have the time to watch those DVDs anymore; they just sit around my place collecting dust. I didn't even watch the last batch I had, which sat around for probably a month.

So I dropped my Netflix account down to the online-only service. Well, now you've gotta send those DVDs back with 7 days after your previous subscription expires. I've just been sitting on them anyway, so I opened and resealed the DVDs I had, and put them in the mailbox.

Netflix kindly sent me the next DVDs in my queue, since I still have time on the previous month's subscription. Since I don't have time to watch them anyway, I'll just send them right back without ever removing the DVDs from their envelope.

I don't have a huge complaint about that whole process. It would've been convenient if they had a way for me to say "don't ship me anymore DVDs". I'm not aware of a way to do so, but that's OK... minor annoyance.

One way for me to stop them from shipping me new DVDs is to clear out my queue. Now, I have a full DVD queue plus a handful of saved movies; nearly 500 in total. Problem: There is no "Delete All" button, you have to click 500 little delete buttons. (This also seems to cause the queue to throw JS errors occasionally.)

Now it won't help me right now, but maybe someone (me?) could benefit from a Delete All button in the future if it were implemented. So I hit the "contact us" button looking for a place to fire off a suggestion. That's when I found out there's no way to email Netflix, they only have phone support.

I don't actually want to talk to anyone. I want to send a suggestion. A quick missive "Can you please add a 'delete all' button to the queue?"

An email can get forwarded around; hopefully to a UI team that could implement the feature at some point in the future.

So, Netflix: Sometimes email is better. I'd rather have a choice.

In the meantime, I guess @netflixhelps will have to do. Just keep suggestions to less than 140 characters.

SCP: I've been doing it wrong all along

2011-03-13T13:01:00.000-07:00

It's not something I have to do a lot, so it's never occurred to me that it might be a problem. If you're transferring a lot of small files with SCP, you're doing it wrong too.

Here, I'm copying an old source code repository to my laptop from my desktop using SCP:


jrodriguez@laptop:~/foo$ time scp -Cqr desktop:/tank/Secure/vcs scp

real 1m49.188s
user 0m0.328s
sys 0m0.392s

Here I'm running the directory through tar and piping it's output through SSH, and back through tar on my side:


jrodriguez@laptop:~/foo$ time ssh desktop tar zcC /tank/Secure/vcs . | tar zxC tar

real 0m7.185s
user 0m0.200s
sys 0m0.144s

That's 789 files totaling 6.6 MB.

Lesson: If you're transferring a lot of small files, don't let SCP do it for you.

JavaScript Object Graph (JSOG) 0.31 Released!

2011-01-18T14:14:00.000-08:00

JSOG has grown a lot since I first decided to create it. It's been growing quietly, adding features without much documentation available.

Today, that's changed. New features that have been added, and documented:

* JSOGPath expressions - Navigate an object graph by XPath-like expressions.
* XmlJsogFactory - Convert an XML document to a JSOG.
* BeanJsogFactory - Convert a JavaBean into a JSOG.
* Spring support - Namespace extension for UrlBuilder, MVC integration.
* Dynamic methods - Dynamically inject object graph values into a method.

Head on over to the JSOG site and check out the new and improved User Guide.

JSOG - JavaScript Object Graph

2010-06-13T01:26:00.000-07:00

So I had a project where I was creating a user preferences service. Because the environment in which the preferences service was to be operating is so chaotic (lots of changes are expected), I decided to use JSON as a way to store the user preferences. The service is a glorified document database, if you will.

Now, working with JavaScript objects and arrays in JavaScript is pretty easy. There's a lot of implicit coercion taking place, it's a dynamic language after all. That doesn't translate to Java very well, however. I looked at JSON-lib, FlexJSON, Jackson, and the json.org library. None of them provided the functionality I was looking for. I want to easily navigate a JSON object graph, and manipulate the values within.

Let's look at my particular use case for example. I had a JSON object for a user. Within that JSON object, I basically had the following: (Formatted for readability)


{
  "services": {
    "xyz": {
      // XYZ specific preferences
    }
  }
}

Except since things are so chaotic, I don't know if the "services" node exists, or if "xyz" exists.

I'll demonstrate with Jackson here, but the process is basically the same for the other libraries I looked at.

In this example, we create the object from scratch:


ObjectNode prefs = nf.objectNode();

ObjectNode services = nf.objectNode();
prefs.put("services", services);

ObjectNode xyz = nf.objectNode();
services.put("xyz", xyz);

xyz.put("foo", "bar");

What does that same code look like in JSOG?


JSOG jsog = JSOG.createObjectNode();
jsog
   .get("services")
       .get("xyz")
           .put("foo", "bar");

A little more straight forward, but Jackson wasn't that bad. What about a more complex example?

Instead of creating everything from scratch, what if we need to parse the existing preferences and add a value? If the JSON object doesn't contain "services", or "xyz" (which we're not sure of), we need to create those nodes.

Let's parse a string, empty in this case, and build up the same object, and add a value to "xyz" called "baz" with value 42.

Here's what you're facing with Jackson:


ObjectMapper om = new ObjectMapper();
JsonNodeFactory nf = om.getNodeFactory();

// Parse the JSON string
JsonNode json = om.readTree("");

// Get or create the prefs container object
ObjectNode prefs;
if (json.isObject()) {
   prefs = (ObjectNode) json;
} else {
   prefs = nf.objectNode();
   json = prefs;
}

// Get or create the services object
JsonNode servicesNode = prefs.path("services");
ObjectNode services;
if (servicesNode.isObject()) {
   services = (ObjectNode) servicesNode;
} else {
   services = nf.objectNode();
   prefs.put("services", services);
}

// Get or create the xyz object
JsonNode xyzNode = services.path("xyz");
ObjectNode xyz;
if (xyzNode.isObject()) {
   xyz = (ObjectNode) xyzNode;
} else {
   xyz = nf.objectNode();
   prefs.put("xyz", xyz);
}

// Set the value
xyz.put("baz", 42);

And here's JSOG doing the same thing:


JSOG jsog = JSOG.parse("");
jsog
   .get("services")
       .get("xyz")
         .put("baz", 42);

Now we're talking! JSOG pulls off in 5 clean, clear, concise lines of code what took Jackson 38 LOC to do. JSOG does the implicit object creation or coercion for you, and to make things even easier, you get builder-style syntax!

I want to point out that I'm not picking on Jackson here, this is a failing of all the libraries I looked at. I think Jackson is actually one of the easier libraries to use, which is why I chose to use it for JSOG's serialization and deserialization.

JSOG is Unlicensed. I've committed it to the public domain. I would love to see JSOG or an equivalent find it's way into each JSON library.

Visit the JSOG site, hosted on SourceForge.net and give it a whirl. I suspect people working with CouchDB will *really* appreciate JSOG.

Basic Motorcycle Safety

2010-04-14T19:26:00.000-07:00

So I've been going through the work to get a motorcycle, and I was reviewing the Hurt Report findings. I'd recommend going over them yourself, but here's the short-short version for my own reference and yours:

Cornering and intersections are deadly.
At intersections, expect oncoming traffic to turn in front of you.
Under/overbraking are common in the moments preceding an accident - get a bike with ABS/practice emergency stop drills.
Be as conspicuous as possible. Orange, yellow, and red colored clothing, helmets, and bike colors are a Good Thing. Keep your headlights on.
Wear a full face or modular helmet. Leathers will help you keep your skin where it belongs.
Take a motorcycle safety class.
Pay attention!

These basic rules cover such an overwhelming percentage of accidents, and they're so simple to follow. Unless you're trying to give your organs away, follow them.

Shooting with the Lensbaby Composer

2010-03-08T18:51:00.000-08:00

The following two images are shot using the Lensbaby Composer. They were taken at the Phoenix Art Museum on the March First Friday.

These are using my 18-105mm lens, on the same night.

Ubuntu's NetworkManager is Flaky

2010-01-21T08:07:00.000-08:00

The debate rages on as to what component is actually causing the problem: NetworkManager, wpa_supplicant, drivers, or something else. But the problem is clear: Frequent disconnects, particularly in an environment where multiple access points provide access over a single SSID.

Either way, replacing NetworkManager with wicd resolves the problem.

sudo apt-get install wicd

After you've done that, wicd-client will start in your tray on the next login.

If you want, you can also configure it through the wicd-curses interface. One oddity, though, is that the curses interface is case-sensitive.

Fall harvest

2009-12-05T09:47:00.000-08:00

My pumpkin vine is looking pretty bad after a really cold night, so I decided I'd better pick the pumpkins. I also harvested the butternut squash while I was at it.

I'll include the pics of some of the corn I harvested earlier this year. The corn, by the way, is called Mirai sweet corn. It's a variety that produces extremely sweet ears, there was much rejoicing about it's taste. I actually harvested 6 ears of corn. I would've had more, but a lot of the seeds didn't germinate. I think that's because the soil was still a little 'raw' and the seeds didn't get enough water to germinate.

Four of the six:

These three went to friends. I ate another recently.

Today's harvest:

I still have a few pumpkins on the vine that aren't quite ready yet, we'll see if they survive the weather.

Here are the current stats from the garden:
Pumpkins - 89lbs
Butternut Squash - 20.5lbs
Corn - 6 ears

Update 2010-01-01:
The weather killed off the rest of the vines, so I had 5 pumpkins and two butternut squash that didn't fully develop. I did manage to harvest 1 more 9.5lb pumpkin, bringing the total to 98.5lbs.

Ah well, the rabbits and chickens won't mind the leftovers.

My Steak Method

2009-10-26T22:52:00.001-07:00

I posted this on Fark a bit ago, and figured since I typed it all out I might as well save it for posterity's sake.

1. The cut and quality of the meat are of significant importance. You really can't go wrong with a choice or prime T-bone. I like mine about 3/4 to 1" thick. Any bigger and it doesn't grill well.

2. It has to be room temperature before it's put on the grill. Let it sit out for 15 minutes or so before you put it on the grill.

3. Cooking time and temperature are important. I crank my grill up to high and do 3-4 minutes, flip, 3-4 minutes. That gives me a good rare-medium rare. Don't worry about any flames, it'll char the outside but won't be going for long enough to cause it to burn.

4. It has to sit for a few minutes after it's taken off the grill, the inside of the steak will cook from the heat of the outer parts. Take it off the grill, THEN get out any plates/flatware. Get your sides together, take your time, I give it 3-5 mins.

For seasoning, I use McCormic Grill Mates® Montreal Steak Seasoning. Work it on there with your hands so it sticks well, be generous but don't go overboard.

Raw steak is shitty, rare-medium rare is fucking amazing.

S'Mars

2009-09-25T23:20:00.000-07:00

The Smore meets the Rice Krispy Treat.

1 jar of jet puffed marshmallow creme
1 brick of graham crackers
1 cup of chocolate chips, room temp
1 stick of butter

Crush the graham crackers (a plastic bag and rolling pin work wonderfully). Mix the marshmallow creme, graham cracker crumbs, and chocolate chips. It'll be somewhat stiff, but get the mixture to be fairly homogeneous. Melt the stick of butter in a separate bowl, I nuked mine for 1 minute and it was boiling a bit when I pulled it out. Pour the hot butter into the mixture and stir, it should start to stir more easily. At this point, the result should have a texture a lot like Floam.

Put it in the fridge to solidify.

The whole process takes, literally, 5 minutes or so.

Modifications:
Try different kinds of chips, peanut butter is particularly good.

Video

Java XML validation, "Cannot find the declaration of element"?

2008-12-17T12:47:00.000-08:00

Many thanks to Santiago Pericas-Geertsen, who blogged about this exact problem earlier this year. If you can't figure out for the life of you why your document won't validate, even though you know it's valid, try setting setNamespaceAware(true) on your DocumentBuilderFactory:

DocumentBuilderFactory factory = DocumentBuilderFactory.newInstance();
factory.setNamespaceAware(true);

Chances are that's all you need and you'll be up and running.

Programming is easy, it's all the gotchas that make it hard.

Netbeans-Apache Commons library importer

2008-08-01T02:30:00.001-07:00

I threw together a script to download the latest version of all the Apache Commons components, and create the library files used for Netbeans. It's a little rough and short on docs, but easy enough to use. Some components can't be downloaded and processed easily (those with -e) and the signature of betwixt can't be verified because it's key is out of date, so we force it's acceptance (-f)

cd ~
mkdir JavaLibraries
wget http://www.apache.org/dist/commons/KEYS -qO - | gpg --import
./asfcommons2nb.pl --sig --dir JavaLibraries/Commons \
-e daemon \
-e httpclient \
-e jci \
-e latka \
-e attributes \
-f betwixt

When you're done, you'll want to copy the xml files from JavaLibraries/Commons/Libraries/
to ~/.netbeans/6.1/config/org-netbeans-api-project-libraries/Libraries/
Replace the bold part with the appropriate version.

It's pretty smart. It'll check to see that you have the current version and only update if you don't.

I like Netbeans' library feature, but keeping up to date on things sucks. If you don't care about docs, Ivy would suffice, but I do want instant javadoc access. That's why I'm using an IDE!

Anywho, it's public domain, feel free to send me enhancements or bugfixes.

CA signed SSL certs in glassfish, for lazy admins

2008-03-19T17:03:00.000-07:00

So setting up CA signed SSL in Glassfish sucked, until I finally found the process for doing so. John Yeary outlines the process, but it's still manual and annoying.

Never do more than twice anything you can script, and so here we go. The script below is designed to work with CACert.org certificates. Save it to your local system and run it as a user who has access to write to the Glassfish installation.


#!/bin/sh
# Copyright 2008 Jeff Rodriguez [jeffrodriguez.com]
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

/bin/echo "This script was designed to work with CACert.org."
/bin/echo "It may work with other sites, YMMV."
/bin/echo "USE AT YOUR OWN RISK."
/bin/echo -e "\n\n"

read -rp "SSL domain name [`hostname`]: " DOMAIN
if [ -z "$DOMAIN" ]; then DOMAIN="`hostname`"; fi

read -rp "AppServer domain directory [/home/`whoami`/SUNWappserver/domains/domain1]: " DIR
if [ -z "$DIR" ]; then DIR="/home/`whoami`/SUNWappserver/domains/domain1"; fi

read -rp "Keystore password [changeit]: " KSPASS
if [ -z "$KSPASS" ]; then KSPASS="changeit"; fi

# Make sure the domain directory is correct
KS="$DIR/config/keystore.jks"
if [ ! -e "$KS" -a ! -w "$KS" ]; then
/bin/echo "Your keystore ($KS) is not writable."
exit 1
fi

# Handy alias
KSCMD="keytool -keystore $KS -storepass $KSPASS"

# Generate the key
if ( ! $KSCMD -genkey -keyalg RSA -alias $DOMAIN -keypass $KSPASS -noprompt -validity 365 -dname "cn=$DOMAIN"); then
/bin/echo "Failed to generate key."
exit 1
fi
/bin/echo -e "\n\nA private key has been created in your app server's keystore."
/bin/echo "If this script fails at any point, and you wish to start over, you must delete this private key with the command: "
/bin/echo "$KSCMD -delete -alias $DOMAIN"
read -sp "Press enter to continue." REPLY
/bin/echo -e "\n\n"

# Create the CSR
if ( ! $KSCMD -certreq -v -alias $DOMAIN -file /tmp/$DOMAIN.csr); then
/bin/echo "Failed to create certificate signing request."
exit 1
fi

/bin/echo "Below is your Certificate Signing Request (CSR)."
/bin/echo "On the CACert.org website, go to Server Certificates -> New. Paste the text below into the form and submit it."
cat /tmp/$DOMAIN.csr
rm -f /tmp/$DOMAIN.csr

# Get the certificate
/bin/echo -e "\n\nCACert will generate a signed certificate. Paste it here, finish with EOF (CTRL+D)."
cat > /tmp/$DOMAIN.crt

# Fetch the CACert certificates
/bin/echo "Before you can use your certificate, it has to be imported to the app server keystore. To do that, we first have to import the CACert.org certificates."
read -rp "Would you like the script to automatically download the certificates? [y|N]: " DLCERTS

if [ $DLCERTS = 'Y' -o $DLCERTS = 'y' ]; then
wget -q --no-check-certificate -O /tmp/cacert_root.crt "http://www.cacert.org/certs/root.crt"
wget -q --no-check-certificate -O /tmp/cacert_class3.crt "http://www.cacert.org/certs/class3.crt"
else
/bin/echo "You have chosen not to automatically download the certificates. To do this manually,"
/bin/echo "Go to http://www.cacert.org/index.php?id=3"
/bin/echo "Save the Class 1 PKI Key Root Certificate (PEM Format) as /tmp/cacert_root.crt"
/bin/echo "Save the Class 3 PKI Key Root Certificate (PEM Format) as /tmp/cacert_class3.crt"
read -sp "Press enter to continue." REPLY
fi

if [ ! -e /tmp/cacert_root.crt -o ! -e /tmp/cacert_class3.crt ]; then
/bin/echo "CACert certificates weren't found in /tmp. Either automatic download failed or you have not manually downloaded the certificates properly."
exit 1
fi

# Import the certificates
if ( ! $KSCMD -import -v -alias cacert -trustcacerts -noprompt -file /tmp/cacert_root.crt); then
/bin/echo "Failed to import CACert root certificate."
exit 1
fi
if ( ! $KSCMD -import -v -alias cacert3 -trustcacerts -noprompt -file /tmp/cacert_class3.crt); then
/bin/echo "Failed to import CACert class 3 certificate."
exit 1
fi
if ( ! $KSCMD -import -v -alias $DOMAIN -trustcacerts -noprompt -file /tmp/$DOMAIN.crt); then
/bin/echo "Failed to import domain certificate."
exit 1
fi

# Cleanup
rm -f /tmp/cacert_root.crt
rm -f /tmp/cacert_class3.crt
rm -f /tmp/$DOMAIN.crt
$KSCMD -delete -alias cacert
$KSCMD -delete -alias cacert3

# Final instructions for the user
/bin/echo -e "\n\n"
/bin/echo "The certificate has been imported into your app server keystore."
/bin/echo "To use the certificate follow these last steps:"
/bin/echo "  1. Go to your Admin Console (http://$DOMAIN:4848), Configuration, HTTP Service, HTTP Listeners, http-listener-2 (or whatever your SSL listener is)."
/bin/echo "  2. On the SSL tab, set your Certificate Nickname to $DOMAIN, and click save."
/bin/echo "  3. Restart the app server."
/bin/echo "NOTE: Your certificate will expire in 1 year, you must update the certificate before then or browsers will receive SSL warnings; other programs may fail entirely."
/bin/echo "Mark your calendar. No, really."
/bin/echo -e "\n\n"
/bin/echo "If something went, or goes wrong, it could prevent your app server from starting. One possible remedy is to locate your SSL http listener in $DIR/config/domain.xml and disable it."

Though supposedly, it's technically possible to use an existing private key and certificate, you'll shoot yourself in the face before you finish. Cut your losses and just start over with a new private key and cert. You'll have to pardon me if the script is a bit rough around the edges. Send patches my way if you like.