CSSR (Curvature Scale Space Recognizer)

Get the Android app from Google Play.

I wish there is something like CSSR outside of academia to describe it. So that you would know by now if the app is right for you. But other than broad or hyped terms like Computer Vision or Augmented Reality, there is nothing more I can say unless you follow a little tutorial.

The Tutorial

What You Need:

The Camera View

This is the first view you encounter once you start CSSR. It gets right to work trying to select objects. Point the cross marking center of the camera at an object and see how well an outline of the object is selected. You can name the object it selected and it will recall that name whenever you point it to the same object again.

Lets do an example we can all get our hands on. A Game of Rock Paper and Scissors. Make a paper in one hand stretched in front of you and point the cross at it using the other hand, with one finger free to tab on screen. Move the camera back until the whole hand is in view. Your sleeve should cover your arm and be secured. Since arms are bigger than fingers, you don't want CSSR to learn to differentiate by the length of exposed arm over the number of fingers. The outline should now select your hand. Otherwise, move around and try different backdrops. Look for a simple backdrop of a color unlike skin. Once you have a good consistent selection, without moving either hand, tab with the free finger. A pop-up will appear with your selection, and you are free to move as you like now.

The pop-up shows an outline that it will learn. If it shows a hand with five fingers then type "paper" in the text field and press "Learn New Shape", otherwise press "Cancel" and try again. Now that CSSR knows one sign named paper lets test it out. Return to your previous positions and reproduce the paper selection. Does your screen say "paper???" (less ? marks means a closer match, and a ! means an almost exact match). To avoid fluctuations, the center of the screen is an aggregated result. On the left upper side is the frame by frame result. On the second line, you may observe something like "paper:42%" meaning it is 42/100 like paper. As you correctly believes, that is not a very good result. Your hand might be held slightly differently, or the stars are not aligned. But we can do better by reinforcing the learning process. Tab the screen will bring back the learn popup. This time, with "paper" in the text field, the "Learn New Shape" button have turned into "Reinforce". Confirm that the outline captured is right and press "Reinforce". Repeat a few times untill it says "paper?" or better. Now we have one sign learned and two more to go.

Make a scissors. Since it only knows paper so far, it may falsely report paper. We correct this by teaching it scissors the same way as paper. Once CSSR can consistently recognize scissors, it should still remember what paper looks like. Switch your hand between paper and scissors. The screen will report paper or scissors accordingly. When there is a bad match, you can always use reinforce.

At last, rock. We are finishing with rock because it is the hardest of the three for CSSR. You will really have to reinforce this sign many time. It has an outline that is most sensitive to pose and the least amount of distinguishing features.

After playing a game of Rock Paper and Scissors, you can also try to make any other hand sign for CSSR to learn, or anything else you can find around you.

The Library

Now that we have many shapes remembered, they are saved in the library. Select the “Menu” button on the lower left corner of screen and select “Library” to view and edit them.

The options on an instances are: Move, Delete, and Cancel.

You should now see a list of names of the shape you have saved. select one of them takes you to a grid of all instances of that shape. Select one instance to manage it. You can also manage the whole group with the Menu key or the Action Bar.

Go back to the list of names with the Back key. The Menu in this view features:

Settings

Feedback

contact: george@xiegeo.com