HamsterSoup - Recipes for Data Geeks

Our short caving expedition is already finished. The Canto system is now almost 80km long!

There will be no updates for a week and a half as I’m caving in Spain.

I’m working on a BBCode generating bookmarklet for Freebase data. I didn’t want to pollute online forums with lots of test posts so I whipped up my own BBCode live editor.

Type BBCode into the top textarea and the rendered HTML will appear after a second of inactivity.

There are other live editors on the web but since I’m running my own I can easily configure it to support the different dialects out there.

Will’s live editor in action:

1. bolded text
2. italicized text
3. underlined text
4. http://wikipedia.org
5. Wikipedia
6. 

When I show developers Freebase and the query language MQL, the first thing they often try to do is extract all the info they can out of the database for a generic topic.

This is actually quite a good learning experience, as you have to learn lots of MQL concepts:

• How “*” expands (default values v objects, type specified or not)
• Multiple queries v nested queries
• Reflection

I had been keeping my own notes, but I thought they might be worth sharing so I whipped up a nice stylesheet and here they are:

Step by step MQL with Jimi Hendrix

Still lots of uncompleted TODO points but I’d be interested in your feedback. Is this something worth working on?

Nick’s demo

The latest version of MJT supports auto-complete. See Nick’s simple example.

Will’s demo

Here’s my example (currently specific to films) Notice how the thumbnail updates as you navigate the menu I had to add an extra ‘onchange’ hook, see source code for details.

I made some updates to my Map of Freebase Users (see old post and Freebase topic)

• Lots more user photos (using /common/topic/image rather than picture_s)
• Does not randomize if map location is unqiue (Thanks Dannyman!)
• Disabled clustering - it seemed to be a performance hit. It’s easy to make maps with 1000’s of markers so I may have to reconsider this.
• Markers are added to the to map in a separate thread. GUI is more responsive
• Built-in error reporting
• Code is now valid xhtml

TODO:

• Use cursors to make data-loading more responsive. This should also fix the timeouts that Dan reported.
• IE support (but only if somebody bugs me)

I’m a big believer in built-in error reporting. When your app goes wrong, most users just give up and the ones who do report a bug tend to write things like “It doesn’t work”.

By building error reports into your app, your users can send detailed bug reports with just two clicks.

Here’s a self-destructing MJT app with error reporting.

Is there any interest in having this included in the MJT library? It could be My demo shows how it could be added to the template compiler and the the MQL task runner. (Updated: 21 Aug)

If you don’t use Emacs to write MJT code then stop reading now.

nXhtmlMode will dramatically improve your productivity if you write MJT code in Emacs. It highlights invalid XHTML as you type so you never have strange MJT bugs due to unclosed DIVs. You can also add a MJT schema to avoid unnecessary warnings about proprietary attributes.

Step by step guide to installing nXhtml mode

Install nXhtml:

• Download the latest version of nXhtml.zip
• Unzip into a directory, for example: ~/.emacs.d
• Edit your .emacs and add
  (load "~/.emacs.d/nxml/autostart.el")
• Run emacs, open an HTML file and type M-x nxhtml-mode

Accept the default customization:

• Click “remove this message”
• Set Nxhtml Skip Welcome: “toggle” to on
• Click “Save for future sessions”
• Click “Finish”

Install the MJT schema
(At this stage, any errors in your HTML should be highlighted. Unfortunately all the special MJT attrbitues will be highlighted as errors too. We can fix this by overriding the XHTML schema with a MJT schema)

1. Download mjt.rnc
2. Move mjt.rnc to ~/.emacs.d/nxml/nxml-mode-20041004/schema/
3. Edit mjt.rnc and change the first line from
   include "emacs/lisp/nxml-mode/schema/xhtml.rnc"
to this: include "xhtml.rnc"
4. Load a MJT html file and type M-x nxhtml-mode. Check the mjt.task attributes are now accepted

Make nxhtml-mode load automatically on opening .html files
Add the following to your .emacs

;; nXhtml-mode
(load “~/.emacs.d/nxml/autostart.el”)
;;WILL: there should be a cleaner way to do this
(fset 'html-helper-mode 'nxhtml-mode)
(fset ‘html-mode ‘nxhtml-mode)

Conclusion

Now whenever you open an HTML file with Emacs you will automatically see any markup errors. This can save a lot of debug time!

TODO: make multiple modes work so that mjt.script appears as formatted javascript inside an HTML file.

Thanks to Nick for the pointer to nxml-mode and to Jessica for prompting me to write this doc.

Please leave a comment if you try these steps. I’d like to know if it works for other people (or if you know a better way)

Jon promises to keep the list del.icio.us/judell/publicdata updated. I wonder if some of the sources are suitable for dumping into Freebase.

I’ve created two bookmarklets that are very useful when working with the Query Editor.

• Drag the 2 bookmarklets to your browser toolbar.
• Select a Freebase ID, for example: #9202a8c04000641f8000000003ec6ca4
• Click the View ID on Freebase bookmarklet in your toolbar

The topic The Meadows Restaurant will appear. Cool eh?

Comments and suggestions welcome.