Tuesday, November 15, 2011

Automatically updating appengine-web.xml's version

If you're using AppEngine in Java land, you're probably using maven with maven-gae-plugin. If you're not, why aren't you?

Anyway, since you already have a project version, why not leverage that in appengine-web.xml's version tag?

<build>
  <plugins>
   <finalName>${project.artifactId}</finalName>
   <!-- Sets the version in appengine-web.xml -->
    <plugin>
      <groupId>com.google.code.maven-replacer-plugin</groupId>
      <artifactId>maven-replacer-plugin</artifactId>
      <version>1.3.9</version>
      <executions>
          <execution>
        <phase>package</phase>
        <goals>
            <goal>replace</goal>
        </goals>
          </execution>
      </executions>
      <configuration>
          <file>target/${project.artifactId}/WEB-INF/appengine-web.xml</file>
          <replacements>
        <replacement>
            <token>%%VERSION%%</token>
            <value>${project.version}</value>
        </replacement>         
          </replacements>
      </configuration>
    </plugin>
  </plugins>
</build>

<appengine-web-app xmlns="http://appengine.google.com/ns/1.0">
  ...
  <version>%%VERSION%%</version>
  ...
</appengine-web-app>

And that's it. When you execute gae:run, gae:deploy, etc, the version will be set automatically.

Monday, November 7, 2011

Call Me CAPTCHA Launched!

I launched my new service last night, Call Me CAPTCHA. I've been hacking on this project for a little while in my spare time.

Instead of trying to decipher more and more distorted random letters in a CAPTCHA image, why not leverage a system that's already fairly secure, ubiquitous, and easy to use: your phone. The distorted image CAPTCHA is already effectively broken, spammers can either use CPU cycles to break these or they can use humans for it. Yep, you can get paid to break CAPTCHAs; if you're running a bot, that's $2.00 per 1,000 CAPTCHAs.

You've probably seen a phone-based CAPTCHA before. The first instance I can remember is MySpace, then Google, now sites like Bank of America's with "SafePass". Well, I'd call these companies "the big boys". But what if you're a smaller site that can't afford to build a system like this, or even a large site that would rather use an off-the-shelf service? There weren't really any options until Call Me CAPTCHA.

If you've used reCAPTCHA before, you'll notice a lot of similarities; the APIs are very similar. reCAPTCHA is wildly popular, so I used them as a model for my system.

So if you're unhappy with the protection afforded by image-based CAPTCHAs, and need something more robust and easier to use, head on over and check out Call Me CAPTCHA.

Tuesday, July 12, 2011

Google+ is Killer

It's very easy to group your friends in different groups. Groups are like the ubiquitous "label", but for your friends. Group friends by degree (best, good, proxy, acquaintance, etc), or by classification (family, extended family, friends, professional), or even by interest (hiking, photography, school)


I have a lot of friends that I would like to keep up with, but I don't have time to. Some of these are old circles of friends from high school, or an old job, or my long distance best friends. I touched base with two different groups today, people that I really like, but haven't talked to or seen for months. Hangouts make this really easy. Hangouts are basically like private chatrooms with video meets social networking. I can foresee hangouts becoming incredible tools for all kinds of things:

Townhalls. I watched host the first townhall on Google+. If Google can really go to town with the Hangout technology, I can see the digital townhall becoming an incredible political tool. Imagine the power of being able to organize an actual town hall (moderation tools and all) for absolutely no cost.

Hangouts could also be chat rooms. A really good example already exists; User Plane. It's basically IRC meets Hangouts.

If posts become rich-text enabled with a WYSIWIG editor, the "Public" visibility option could become a good blog replacement.

With the Android phone application, it's also an excellent microblogging platform.

Google+ is a great new social tool, and if Google can play it's cards right, it could be the next killer social app.

Update: Actually, now that I think about it. Google+ is what Google Wave should have been.

Update 2: Circles are another powerful aspect of Google+. Now I can easily use the same social platform for my professional, and private faces. Google+ is a threat for Facebook and LinkedIn.

Friday, June 3, 2011

RESTful JSON web services with JSOG and Spring

Before I originally developed JSOG, we were using the json.org library and Restlet. Our needs were pretty basic, we were just GETting and POSTing JSON strings between services. Later we ended up using some JSONP.

My biggest beef with Restlet is there doesn't seem to be a good way to unit test it. From what I can tell, the convention is to keep all your logic outside of Restlet and use it only to expose the REST interface. I'm also a heavy spring user, and it occurred to me very quickly that Restlet's integration with Spring was... complicated.

So, instead of jumping through Restlet's hoops, JSOG now integrates very easily with Spring MVC. Note, there are two ways of integrating JSOG with Spring. We're going to be using the "Flexible" approach, and we'll be using a JSON-specific DispatcherServlet that handles all *.json requests. This basic example should show you how add RESTful services to your application.

First, we'll need the dependencies. I'm using Maven; so here's what a minimal POM looks like.
pom.xml

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>jsogws</groupId>
<artifactId>jsogws</artifactId>
<packaging>war</packaging>
<version>1.0-SNAPSHOT</version>
<name>jsogws</name>
<url>http://maven.apache.org</url>
<properties>
<spring.version>3.0.5.RELEASE</spring.version>
</properties>
<dependencies>
<dependency>
<groupId>net.sf.jsog</groupId>
<artifactId>jsog</artifactId>
<version>0.30</version>
</dependency>
<dependency>
<groupId>commons-io</groupId>
<artifactId>commons-io</artifactId>
<version>2.0</version>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-core</artifactId>
<version>${spring.version}</version>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-context</artifactId>
<version>${spring.version}</version>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-webmvc</artifactId>
<version>${spring.version}</version>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>3.8.2</version>
<scope>test</scope>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>2.0.2</version>
<configuration>
<source>1.6</source>
<target>1.6</target>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-war-plugin</artifactId>
<version>2.1-beta-1</version>
<configuration>
<failOnMissingWebXml>false</failOnMissingWebXml>
</configuration>
</plugin>
</plugins>
<finalName>jsogws</finalName>
</build>
<repositories>
<repository>
<id>jsog.sourceforge.net</id>
<url>http://jsog.sourceforge.net/m2-repo</url>
</repository>
</repositories>
</project>


Well that's a fair chunk of configuration. Basically, we're importing JSOG from the JSOG maven repository. We also import the relevant Spring artifacts and Commons IO.

Next up, we need to configure the DispatcherServlet.

web.xml

<?xml version="1.0" encoding="UTF-8"?>
<web-app xmlns="http://java.sun.com/xml/ns/javaee"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/web-app_3_0.xsd"
version="3.0">

<!-- JSOG web services -->
<servlet>
<servlet-name>jsog</servlet-name>
<servlet-class>org.springframework.web.servlet.DispatcherServlet</servlet-class>
<load-on-startup>0</load-on-startup>
</servlet>
<servlet-mapping>
<servlet-name>jsog</servlet-name>
<url-pattern>*.json</url-pattern>
</servlet-mapping>

</web-app>


Here we map every *.json request to the DispatcherServlet named "jsog".

Now we need to configure the dispatcher servlet. We used the default configuration file name, in this case:
jsog-servlet.xml

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:context="http://www.springframework.org/schema/context"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="
http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans-3.0.xsd

http://www.springframework.org/schema/context
http://www.springframework.org/schema/context/spring-context-3.0.xsd">

<!-- Look for controllers -->
<context:component-scan base-package="jsogws.services.json"/>

<!-- All controllers managed by this servlet should use the JsogViewResolver -->
<bean id="viewResolver" class="net.sf.jsog.spring.JsogViewResolver"/>

<!-- We need this to consume JSOG that has been sent in the request body -->
<bean class="org.springframework.web.servlet.mvc.annotation.AnnotationMethodHandlerAdapter">
<property name="messageConverters">
<list>
<bean class="net.sf.jsog.spring.StringJsogHttpMessageConverter"/>
</list>
</property>
</bean>

</beans>


In this configuration file, we tell the DispatcherServlet to look for "@Controller" annotated classes in jsogws.services.json. Then we setup the JsogViewResolver, which tells Spring to create and use a JsogView whenever we return a JSOG object in a controller. (JsogView does the lion's share of the work). We also setup a MessageConverter so we can consume JSON.

Note that at this point, most of this is pretty standard Spring stuff. The MessageConverter and ViewResolver are the most "abnormal" things in here: about 8 lines of boilerplate configuration.

From this point, creating a JSON-based RESTful service is just about like creating any other RESTful service.

Let's take a look at a Hello World example:

HelloJsog.java

package jsogws.services.json;

import net.sf.jsog.JSOG;
import org.springframework.stereotype.Controller;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestMethod;

@Controller
public class HelloJsog {

@RequestMapping(value="/hello.json", method=RequestMethod.GET)
public JSOG hello() {
return JSOG.object("Hello", "World!");
}

@RequestMapping(value="/hello.json", method=RequestMethod.POST)
public JSOG hello(@RequestBody JSOG request) {
String name = request.get("name").getStringValue();
return JSOG.object("Hello", name);
}

}


This class defines two methods:
One accepts a GET request and responds with an object: {"Hello", "World!"}.

The other accepts a object via POST, that contains a "name" property. It returns an object with the name: {"Hello", "foo"}.

That's all there is to creating RESTful JSON web services with JSOG and Spring. If you're wondering "what's RESTful about this?" Check out Spring's @PathVariable annotation. Wiring the rest together is elementary.

A quick gotcha: When you send data to your services, you need to use the content type "application/json" and the encoding ISO-8859-1. These are modifiable through properties on the JsogViewResolver and StringJsogHttpMessageConverter.

You also get JSONP for free. Append ?callback=foo and "foo" becomes your callback method. The name of this parameter is settable in JsogViewResolver, but will work with jQuery by default.

Now your web services use the same framework style as everything else in your application, and can easily be unit tested.

Download
jsogws.tgz

Thursday, May 26, 2011

A trip down the number line

This was written by the Fark.com user Hector_Lemans, 2009-09-12 09:43:17 PM

I've tried to track it down online, perhaps it was an article from somewhere else, but it looks like it's original. I'm cleaning out my Google notebook, and wanted to keep it here for posterity.


I've always been fascinated by large numbers. As a child, I would occasionally play the "one-up" game with my friends. Perhaps you've played this game. Two kids disagree about something and one says "bet you a hundred bucks it's not!" The other one retorts "bet you a million bucks it is!" And so on into infinity...which we usually got to rather quickly. But there are quite a few numbers between 100 and infinity. I'd like to take you, dear reader, on a trip to visit a handful of those numbers in the hope that you too can share in my wonderment of big numbers. So grab a power-bar, put on some running shoes, and let's begin.

Our journey will be entirely on the number line. As you may recall from grade school, the number line is a straight line with the counting numbers ticked off at regular intervals.

We start at point zero and begin moving straight ahead. We haven't worked out in a while so we're kind of slow off the starting blocks, but fairly quickly we pass 1, then 2, then 10, then 20. Now we're picking up some speed! As we pass 100, we notice that about a quarter of the numbers we just passed by were bold. As it turns out, we are on a special number line that has all the prime numbers in bold. Prime numbers are numbers that can only be evenly divided by 1 and themselves. So, for instance, 37 is a prime, because there is no other number we can divide it by (besides 1 and 37) that will give us another whole number. The fact that our number line bolds all of these primes is merely a curiosity...for now.

Fairly quickly, we pass 1,000; then 10,000; then 100,000. Now we pass the smallest of our big numbers: one million. A million dollars is what most people define as rich. Even a million pennies add up to $10,000 - not exactly chump change. You would need to flip a coin a million times before having a fair chance of getting 20 tails in a row. In other ways, though, one million isn't that big at all. One million seconds is just over 11 days. A million feet is only 190 miles - a few hours trip in the car. Ten good-sized novels contain a million words. Let's continue on.

We pass 5 million, 50 million, 500 million, and still we are gathering speed. By the time we hit one billion, barely 5% of the numbers we pass are bold prime numbers. A billion seconds is 32 years. A billion feet is 190 thousand miles - that's over three-quarters of the way to the moon! On the other hand, a large drop of blood has about a billion red blood cells in it. Several thousand public companies in the US have a market capitalization of over a billion dollars. So let's continue on in our search for big numbers.

50 billion, 500 billion, one trillion. Now we're approaching the high end of any numbers that come up in everyday conversation. The US debt stands at almost five trillion dollars. One trillion seconds is over 31,000 years - far older than recorded history and deep in the middle of the last ice age. One trillion feet is approximately the diameter of the earth's orbit around the sun. One light-year (the distance light travels in a year) is about six trillion miles. Alpha Centauri, our nearest neighboring star, is about 4.4 light-years distant. Going there in the space shuttle would take about 38,000 years. You would need to flip a coin at least a trillion times to have a fair chance of getting 40 tails in a row...but at 3 seconds a flip, that would take about 95,000 years. Think 3 seconds a flip is too fast? Take 5 seconds a flip and you're up to 160,000 years. On the other hand, one trillion drops of water is only a few hundred thousand gallons - a big pond. Most laptop computers can perform a trillion calculations in under half an hour. The average person is made up of about 50 trillion cells. Onward we go.

We pass 100 trillion and on to a quadrillion. It's hard to make out the numbers as they whiz past us, but a bit less than 3% - every 30th number or so - is a prime. That's interesting. Going from zero to a million, the concentration of primes dropped by over 95%, but going from a million to a quadrillion drops the concentration down only two more percent. Obviously, the primes are thinning out, but at a greatly decreasing rate. Will we soon pass by the last one? Let's find out.

We pass 100 quadrillion, then make our way into the quintillions. Our nearest neighboring galaxy, Andromeda, is about 15 quintillion miles away. We speed up a bit and find ourselves in the sextillions. At least, that is their official name. Another, more helpful way of visualizing these numbers is by scientific notation. Using this notation, one sextillion is 10^21. The little superscript 21 to the right of the 10 means we multiply the 10 by itself, 21 times. So 1,000 would be 10^3 (10 x 10 x 10), and 1,000,000 would be 10^6. Notice that the superscript also tells us how many zeros are in the number if written out in full.

We pass the sextillions by and move on into the septillions (10^24). From high school chemistry class, you may recall a unit of measurement called the mole. One mole of any element has the same number of atoms: 6 x 10^23, or just under one septillion. One mole of gold is a bit less than half a pound. One mole of hydrogen gas weighs only a gram. Coincidently, the visible universe is also about 6 x 10^23 miles across, or 93 billion light-years. The earth weighs (if you could weigh the earth!) 10 septillion pounds.

Fast and faster we go. The number line is a blur of gigantic numbers, with about every 100th one in bold. I say "about" because there doesn't seem to be any real pattern to how the prime numbers are spaced out - sometimes a couple primes will be within two or three numbers of each other, and other times there will be thousands of numbers between two consecutive primes. No matter, onward we must go.

In no time at all, we are deep in the octillions (10^27). The human body contains approximately 7 x 10^27 atoms. If we divide that by the 50 trillion cells we estimated earlier, we find the average human cell contains over 100 trillion atoms. Think about that - each cell in your body is constructed of more atoms than there are bricks in the Great Wall of China!

We put on a burst of speed and race past the next several large numbers. Nonillion (10^30), decillion (10^33)...the names don't mean much anymore. It's easier to just go by the scientific notation. We move on past 10^50 - approximately how many times you'd have to flip a coin to have a good shot at getting 150 tails in a row. As we move up to 10^70 and beyond, we start approaching the limits of anything physical these gigantic numbers can be compared to. The known universe has 10^80 atoms (give or take). If we assume the universe is a sphere, 93 billion light-years in diameter, that amounts to less than one atom per cubic foot! We live in a universe of mostly empty space.

Huffing and puffing, we finally make it to a truly, incredible, gi-normous number: a googol. A one followed by a hundred zeros - 10^100. The name "googol" was coined by a professor's nine-year-old nephew when asked to think up a name for a really big number. Have we run out of primes yet? Not hardly - less than half a percent of the numbers we run across are primes, but that still works out to 1 out of every 230, on average. Now, take a deep breath and scarf down that power bar. To reach our last few big numbers, we're going to have to sprint.

We blow past a googol and make our way up to 10^124. This is the number of particles you'd have if the universe were nothing but a solid mass of electrons - no empty space at all. We run past 10^155 and bump into some numbers used every day by millions of people! Notice how random those bold prime numbers always appear to be, no matter how far across the number line we go? Turns out, that randomness makes them perfect for encoding sensitive data - like a credit card number - and transmitting that information over the Internet. The actual process is quite complicated (as you might expect), but the essential steps are as follows: Two very large prime numbers are chosen and kept secret by the bank. These two primes are then multiplied together to form a huge composite number. This composite number is given to the person with the credit card, and is used to encode the credit card information, which is then sent back to the bank. The bank, having the original two primes the composite number came from, factors them out and retrieves the data. The beauty of the system is that the encoding using the huge composite number is a one-way process if you don't know the primes it is made of - factoring composite numbers with upwards of 155 digits is next to impossible, even with the fastest computers. So it doesn't matter if someone figures out the composite number you're using, they still can't decrypt an encoded message.

From here on out, most of the interesting numbers we'll run across will be prime numbers. Mathematicians like to find and collect huge primes, like some sort of super nerdy baseball card collector. The late Samuel Yates collected what he called Titanic primes - prime numbers with more than 1,000 digits. Chris Caldwell, of the University of Tennessee, took up the torch after Yates' death in 1991 and stores a list of these large primes on the Internet, with a new category for prime numbers with more than 5,000 digits called Gigantic primes. Several of these primes are quite strange. One, that has 5,114 digits, is composed of only 1s and 0s. Another, with 6,400 digits, is all 9s except for one 8. Currently, the largest known prime, discovered on September 4th, 2006 is just shy of ten million digits long. As we reach it on the number line, we marvel at its size. Written in exponential notation (similar to scientific notation but the base number doesn't have to be ten), the current largest prime number is 2^32,582,657 - 1.

And now, let us accelerate to a speed only possible in our mind's eye. Numbers blur together, numbers so big that just writing them down in full would be a problem in the real world...not enough paper! On and on we go. There it is, just up ahead - the number that will end our journey. We slow down from our impossibly fast pace and stop on the number line directly over this stupendous number. A googolplex - a one followed by a googol zeros. If the whole universe were nothing but paper and ink, you still couldn't even write down the number in full (never mind trying to imagine how big it is). This is the end of our journey, but we can look ahead and see even bigger numbers...a tiny select few are even in bold. We still haven't run out of primes! Do the primes ever end? Does this number line ever end? Does infinity really exist? Just because we can say a thing doesn't make it real or even possible - I can describe a universe in which 2 + 2 = 5 or a square is round, but that doesn't mean a place like that could exist...does it? If we squint our eyes and look really hard, we can see a number so large that no number appears to follow it. It's a number so large we can't even describe it in words. Is that the last number? Only one way to find out. Let's go...last one there is a rotten egg!

Friday, March 25, 2011

Nexus One Debugging

I was having trouble getting ADB from the Android SDK to recognize my phone under Windows 7, here's how I fixed it.

1. Start the Android SDK and AVD Manager.
2. Select "Available packages" on the left.
3. Navigate to: Third party Add-ons > Google Inc. > Google USB Driver package
4. Click "Install Selected" and proceed through the dialogs.
5. Connect your phone
6. Start the "Device Manager" management application, and locate your phone.
7. It should say it's not recognized, right click on your phone and click "Update Driver Software".
8. Click "Browse my computer for driver software"
9. Navigate to the android SDK directory ...\extras\google\usb_driver
10. Proceed through the dialogs.

For me, the driver is located in:
C:\Program Files (x86)\Android\android-sdk\extras\google\usb_driver

I then ran adb devices, and my phone was listed.

Tuesday, March 15, 2011

Netflix's Phone-only support

Until recently, I had the 6-at-a-time Netflix account. I've become rather busy and just don't have the time to watch those DVDs anymore; they just sit around my place collecting dust. I didn't even watch the last batch I had, which sat around for probably a month.

So I dropped my Netflix account down to the online-only service. Well, now you've gotta send those DVDs back with 7 days after your previous subscription expires. I've just been sitting on them anyway, so I opened and resealed the DVDs I had, and put them in the mailbox.

Netflix kindly sent me the next DVDs in my queue, since I still have time on the previous month's subscription. Since I don't have time to watch them anyway, I'll just send them right back without ever removing the DVDs from their envelope.

I don't have a huge complaint about that whole process. It would've been convenient if they had a way for me to say "don't ship me anymore DVDs". I'm not aware of a way to do so, but that's OK... minor annoyance.

One way for me to stop them from shipping me new DVDs is to clear out my queue. Now, I have a full DVD queue plus a handful of saved movies; nearly 500 in total. Problem: There is no "Delete All" button, you have to click 500 little delete buttons. (This also seems to cause the queue to throw JS errors occasionally.)

Now it won't help me right now, but maybe someone (me?) could benefit from a Delete All button in the future if it were implemented. So I hit the "contact us" button looking for a place to fire off a suggestion. That's when I found out there's no way to email Netflix, they only have phone support.

I don't actually want to talk to anyone. I want to send a suggestion. A quick missive "Can you please add a 'delete all' button to the queue?"

An email can get forwarded around; hopefully to a UI team that could implement the feature at some point in the future.

So, Netflix: Sometimes email is better. I'd rather have a choice.

In the meantime, I guess @netflixhelps will have to do. Just keep suggestions to less than 140 characters.

Sunday, March 13, 2011

SCP: I've been doing it wrong all along

It's not something I have to do a lot, so it's never occurred to me that it might be a problem. If you're transferring a lot of small files with SCP, you're doing it wrong too.

Here, I'm copying an old source code repository to my laptop from my desktop using SCP:

jrodriguez@laptop:~/foo$ time scp -Cqr desktop:/tank/Secure/vcs scp

real 1m49.188s
user 0m0.328s
sys 0m0.392s


Here I'm running the directory through tar and piping it's output through SSH, and back through tar on my side:

jrodriguez@laptop:~/foo$ time ssh desktop tar zcC /tank/Secure/vcs . | tar zxC tar

real 0m7.185s
user 0m0.200s
sys 0m0.144s


That's 789 files totaling 6.6 MB.

Lesson: If you're transferring a lot of small files, don't let SCP do it for you.

Tuesday, January 18, 2011

JavaScript Object Graph (JSOG) 0.31 Released!

JSOG has grown a lot since I first decided to create it. It's been growing quietly, adding features without much documentation available.

Today, that's changed. New features that have been added, and documented:

* JSOGPath expressions - Navigate an object graph by XPath-like expressions.
* XmlJsogFactory - Convert an XML document to a JSOG.
* BeanJsogFactory - Convert a JavaBean into a JSOG.
* Spring support - Namespace extension for UrlBuilder, MVC integration.
* Dynamic methods - Dynamically inject object graph values into a method.

Head on over to the JSOG site and check out the new and improved User Guide.