Debatably Beta

Sphinx For Full-text Indexing

Tue, 27 May 2008 11:21:00 -0500

For a few projects at work, I’ve been using Sphinx as a separate full-text engine. The configuration to get going is about as easy as I could ask for, and indexing is lightning-fast:

collected 28851 docs, 6.1 MB sorted 6.0 Mhits, 100.0% done total 28851 docs, 6134126 bytestotal 7.432 sec, 825404.12 bytes/sec, 3882.17 docs/sec rotating indices: succesfully sent SIGHUP to searchd (pid=27222).

For an example of usage, check out a project I made for aggregating news from a few select sources based on keyword sets: http://feedwords.net/

I realize the similarity to Google’s blog notification system, but this was built to index with less noise.

Address Parsing with Geocoder.us

Thu, 24 Apr 2008 15:18:00 -0500

Address Parsing with Geocoder.us:

Geocoder.us has added functionality to parse an address into components.

Given

http://rpc.geocoder.us/service/namedcsv? address=1600+Pennsylvania+Ave,+Washington+DC &parse_address=1

it will return:

number=1600, prefix=, street=Pennsylvania, type=Ave, suffix=, city=Washington, state=DC, zip=, original address lat=38.898748, long=77.037684, number=1600, prefix=, street=Pennsylvania, type=Ave, suffix=NW, city=Washington, state=DC, zip=20502, geocoder modified

The service, however, does not conform to any standardization model.

New ElasticFox Release

Thu, 24 Apr 2008 13:19:32 -0500

Amazon has announced a new version (1.4) of ElasticFox, it’s EC2 management console for FireFox 2, and a version for 3 on the way.

Included in this version is support for the (awesome) elastic IP functionality, a few security updates, and user-selectable kernals. The project has also been open-sourced and moved to Source Forge so anyone with a few minutes and an idea can hack out whatever improvements they have in mind.

Amazon is on a roll!

Google AJAX Translation API

Mon, 24 Mar 2008 21:00:36 -0500

Google AJAX Translation API:

Finally!

Google Visualization Kit API

Mon, 24 Mar 2008 20:56:28 -0500

Google Visualization Kit API:

Google’s introduced a fantastic new API to help us mortals visualize information! Looks like its tightly integrated with Google Docs, which means you can pull information from your spreadsheets and visualize them in your documents or Sites setup.

Call me crazy, but it almost looks like Google is introducing the concept of schema-free information storage by allowing people to use a spreadsheet-style layout (akin to, say, Amazon’s SimpleDB or CouchDB) to define data sources.

Multi-step Forms with Django and FormWizard

Mon, 24 Mar 2008 20:48:19 -0500

Multi-step Forms with Django and FormWizard:

Discusses using multi-step forms with Django’s new FormWizard contribution

Reddit Submit Screen Usability Issue

Sat, 09 Feb 2008 16:28:26 -0600

Reddit recently introduced a cool new feature that allows people to create their own sub-reddits. Unfortunately, they’ve opted to employ a drop-down widget for sub-reddit selection.

How about an auto-completer, Reddit? Or, how about keeping the drop-down for the site’s most common sections and having an auto-completer widget for finding the sub-reddit best fit to the content.

TSA Complaints - The First Day

Fri, 08 Feb 2008 10:57:00 -0600

As of this around 9:30 PM CST, here are the stats from Google Analytics:

3,550+ uniques
18,000+ pageviews
Less than 40% bounce rate (should go down once more interactive features are added, I’d imagine)
30 usable contributions to the site thus far
- ~1% interaction rate. Virginia is for lovers, and the web is for voyuers.
Major referers are:
1. Fark (landed on the front page!)
2. Stumbleupon
3. Reddit
4. Flyertalk

I was expecting the opposite. While I’m glad to see that a majority of people stuck around to check out the site, I plan on developing a few more features to attract return visitors. See below for that. But first:

A BIG “Thank You” to everyone that has emailed me suggestions, comments, bug reports (incorrect linking in the RSS feeds based on an older version of the site, and a problem with San Antonio), and words of encouragement!

And, of course, a REALLY BIG “Thank You” to everyone that has submitted (usable) content so far

Heres the rundown of what I’ll be adding soon to make this site even more interactive and up-front with the information:

Keyword extraction courtesy of the Yahoo! Keyword Extraction API as a form of automatic tagging
Potentially user tagging
Voting on quality of articles

Contrary to what I thought out loud about earlier, I don’t think complaint comments will be happening. Yet.

If you’d like to cover TSA Complaints, I’d be much appreciative. I’m planning on sending out a small batch of press releases to more than a few sensible outlets, but this site needs your contributions and complaints to be at all useful in terms of its greater goal: statistical mining to find what’s going wrong.

Feel free to email me at: tsa -dot- complaints -at- gmail -dot- com

Again, thanks to everyone making this possible. I can’t wait to see where the site goes from here.

TSA Complaints Is Up and Running

Thu, 07 Feb 2008 16:36:00 -0600

I’ve had a ton of page views to TSA Complaints in its first few hours, hit the front page of Fark, gotten some interesting feedback from a TSA agent, and am excited to see content materialize.

Im working on dropping in django-threadedcomments and testing it locally to launch tomorrow or later tonite, so that people can comment on individual complaints. The trick to this, though, is the complaint pages are static media generated by StaticGenerator, so that policy may be reversed. We’ll see.

Announcing TSA Complaints

Tue, 05 Feb 2008 11:21:00 -0600

Im just about ready to launch my first big web project in a while: TSA Complaints.

TSA Complaints a forum for travellers suffering from TSA-burn. Use it to describe your incidents with the TSA, and find which airports are the worst at what times.

Im really excited for the statistics that will be rolling in from the site, and can’t wait to see how the whole thing goes. This project has been equal parts reaction to the TSA blog and excitement over aggregating some data directly from the source (you!) and analyzing it.

Obligatory name droppings: This project was done with Django, and makes use of StaticGenerator, Mootools (including Aeron Glemann’s Calendar and a modified version of Harald Kirschner’s Autocompleter), memcached, nginx and Apache, and MySQL. It’s hosted on a Slicehost slice.

Big thanks to Andy, Paul and Travis at work for feedback, and Jason for both feedback and untangling some (painfully obvious) brain-teasers.

"Software Morphs Rapper Prodigy Into Global Cipher"

Mon, 21 Jan 2008 09:48:00 -0600

Rapper Prodigy’s new album could conceptually be delivered any any of almost 1500 languages IN HIS OWN VOICE thanks to speech conversion software.

“The prospect of having fans understand what I’m saying and repeat it in their language (drew me to) the company,” said Prodigy in a phone interview just before he began a jail term for illegal gun possession. “Now, fans will like more than just the beat or the rhythm. They’ll understand what I’m saying and relate to it.

(excellent paragraph)

Here’s how the Voxonic translation process works. After translating the lyrics by hand, the text is rerecorded by a professional speaker in the selected language. Proprietary software is used to extract phonemes, or basic sounds, from Prodigy’s original recording to create a voice model. The model is then applied to the spoken translation to produce the new lyrics in Prodigy’s voice.

What?!

“A 10-minute sample is all we need to imprint his voice in Spanish, Italian or any language,” said Deutsch.

Now thats just scary-cool. I wonder what famous speeches were culled and parsed for phoneme samples, and what the developers had the orators saying.

Conceivably, you could arrange phoneme samples like beats for a song and re-articulate anything. A video artist in Germany has already done a similar project using video samples from Michael Jackson interviews. The end result was an eerily realistic digital Michael Jackson who would accurately say anything the artist “asked” him to, gestures and all.

Voxonic’s software is able to convert any bit of recorded text into 1,468 different languages with 99 percent accuracy, according to the company.

One audiobook, translated into several languages; one album translated hundreds of times over. A contemporary point-and-click babelfish. Goodbye to overdubs.

Click here to read the rest of the article at Wired.

Extracting Data From Google Street View

Mon, 21 Jan 2008 10:08:34 -0600

Diddling Web 2.0 gives a neat look at how to manually manipulate and extract information from Google’s Street View via query string. Very cool, albeit that Street View is fairly limited (fairly limited is a relative term when used in Google-sized context) in terms of its reach.

Wouldn’t be hard to wrap this into a quick-and-dirty API.

Macbook Air and Ubiquitous Wifi

Mon, 21 Jan 2008 08:10:04 -0600

So, what are the odds that Apple will introduce in the new firmware for the EVDO/3G/next-gen-capable iPhone the ability to (perhaps even simultaneously) use it as an iPod and as an always-on wifi connection? It’d be a smart move that wouldn’t necessarily require Apple to stick a communications chipset into the Macbook Air, and could be the tipping point for those who aren’t yet entirely sold on the iPhone. I know I’d take the plunge, at least on the phone.

Of course, always-on access built into the oh-so-thin notebook would be a breath of fresh Air. Even though the machine itself it best used as a secondary device, it’d finally be the first true out-of-box “ultra-portable.”