« October 2006 | Main | December 2006 »

November 02, 2006

TheLadders.Com does Hack Day

Here at TheLadders.Com we strive to give our employees opportunities for growth, experimentation and innovation. We want people's creativity and passion to be fully engaged. With these goals in mind, we decided to host our own Hack Day in the office. I must give thanks to Yahoo for the idea. Maybe we'll take a group of developers to participate in Yahoo's Hack Day next year.

To start, we canvassed the whole company for ideas. We wanted to include as many people as possible, rather than just technology. In the end, eleven people from technology, business intelligence and product participated individually or in teams. The experiment was a success, as evidenced by the fact that we had over 30 project ideas submitted and 13 projects completed and demoed the next morning.

One lesson for me... I realized that prototyping is a skill that not everyone knows. The difference between creating a full fledged application and a simple prototype that will get the point across is a skill that needs to be developed. So, another benefit of Hack Day was practice for those skills and raising awareness through-out the company of the value of producing prototypes.

I don't want to get into a discussion on rapid application development here, but I think the point was made that with just a day's work you can actually have something interesting to look at. Something that will help the organization gauge the value of the idea and will inform the work estimation process and technical design phase when it comes time for actual implementation. A few caveats here: 1) the prototype being produced was already visualized in the mind of the developer, so no time or value lost in translation; 2) there is an extra level of motivation since the developers choose their own projects and were excited for the event; and 3) free beer and pizza always helps (actually, it was pasta and salad instead of pizza, but you get the point).

You could take that in two ways. That prototypes for "real" projects wouldn't move as quickly without the befits mentioned above. Or, the more optimistic spin, prototype development can be accelerated by understanding how developers work. A free-for-all probably won't get your very far as a day-to-day process, but an organization can still apply this knowledge to their advantage. Developers are generally interested in projects for one of two reasons: 1) They want to work on something that has value; and 2) they want to work on something that involves a new challenge (maybe a new technology)-- something new. So, align your real projects with these motivations even better if your project addresses both criteria. Start with a master list of project ideas, including ideas submitted by the developers themselves and ideas from other departments. Let them choose their own technology and architecture. After all, it's just a prototype. If it proves out, it can go through a proper life-cycle and technology and architecture can be rethought. Also, sell them on the business value. This is always a good idea. Explain why you want to do this thing, what are the goals and benefits for the company and its customers.

I'll admit, one of my concerns going in to this hack day was that people would bite off more than they could chew. We'd wind up with only a couple of completed projects in the given (1 day) time frame and lots of good intentions that didn't make it to fruition. I was happy to learn that this fear was unjustified. As stated above we had over a dozen completed projects.

The Schedule and Thoughts On Each Step

Thursday, Oct 19th - deadline for anyone in the company to submit ideas to hackday@theladders.com

I had to walk around and encourage people to enter their ideas. Generally, people are busy and focused on other things. And some people were still confused about what sorts of things would be appropriate. I tried to provide as much feedback as I could. Should be easier in the future, now that people have seen it once.

Friday, Oct 20th - Developers (including designers, product, etc) meet to review the submitted ideas (create teams, avoid unwanted project overlap)

This went well. We used an internal wiki page to coordinate the submissions and to run the meeting and take notes. The wiki page was distributed before hand with the initial suggestions so people had a chance to think about some of the stuff coming up. Some interesting extensions and modifications of the original ideas were surfaced at this time.

Thursday, Oct 26th - HACK DAY (lunch and dinner supplied by TheLadders.Com)

The hardest part was coordinating lunch for the whole company.

Mostly, people worked away. Helping each other and forming teams as needed. A lot of great team building happened that day. Everyone learned a lot. The other challenge was to prevent everyday responsibilities from interfering with the day's activities. Luckily we had cooperation from the rest of the company.

Friday, Oct 27th 10am to noon - Company wide demos (5 minutes for each demo, followed by 5 minutes of Q&A).

A little rough here. Switch over from one demo to the next was time consuming. Should have had a better system for setting up the next demo as soon as the Q&A started on the previous one. Time limits were not a problem. We got through the whole set in about an hour and fifteen minutes. Might consider providing a suggested format or just tips for the presentations to help people who are less comfortable with public speaking. The idea would be to suggest points or topics that should be touched on when presenting the idea and prototype. Matt McAlister, who wrote about the Yahoo Hack Day, has some good guidelines.

The Projects

Job Wars - Similar to the some what addictive http://kittenwar.com/ but with Job data.

Desktop plants to visualize key company metrics - The Tamagatchi idea.

Customizable Home Page - Like a portal page, or Google's custom homepage. Customers can add and remove modules from the pages, slide them around the page, etc.

Dynamic Quick Help - Ability to easily add pop-up (like quick search) text to any field or UI element on any ladders page.

Smarter Jobsite Fill - Tweaking some of the internal algorithms that fill queues of job sites for our harvesting team.

Firefox/IE search plugin for JIRA issues

Firefox/IE search plugin for subscriber information

Recruiter Blog Widget - displays job listings from TheLadders.Com on the side of your blog.

Harvested Job Quality Numbers - Some metrics about the quality of jobs that were collected for TheLadders.com

Job Feed Import Enhancements - Validation and other tools for accepting XML feeds with job listings

Google Home Page module for Ladders jobs - Display job listings in your field on your Google homepage

The REAL Intranet - Sharepoint didn't work for us, so we constructed a workable model in one day using Ruby on Rails. This is the only Ruby on Rails application at this company.


Prizes

To stimulate competition and to just make the day more fun, we awarded prizes for the following categories:

  • Audience Favorite (Wow factor!)-- done by company wide email survey.
  • Most Business Value
  • Most original

We provided prizes for the winning teams and runner ups.


The Future

I'm hoping we can do Hack Days often. The expense is low (a day of developer time) compared with the reward (morale, idea generation, professional development, actual prototypes and in some cases releasable components). Once a quarter might be appropriate. I'd like to see every one of the ideas submitted throughout the company, implemented as a prototype. Why not?

Finally, I want to thank our CEO and the Executive Team at TheLadders.Com for giving approval and supporting this effort!

Some photos from Hack Day

These "plants" grow over time, based on how many jobs we have collected. A thriving plant, indicates a robust in flow of job postings.
garden.png

Below, each block represents a person. The size of the block is how many jobs were collected and the color indicates the quality score for the jobs harvested.
HQNcoloredblocks.jpg


Posted by Marc Limotte at 03:37 PM | | Comments (3) | TrackBacks (0)

Introducing Javascript full-text search

Here at TheLadders.com, we're big fans of Lucene -- it's fast, flexible, and easy to use. There's also plenty of documentation and lots of helpful folks on the mailing lists.  We use it in our job search and resume search, as well as a few internal applications.

For our "MyJobs" project, we were faced with a requirement to display a list of jobs that a user had applied to or saved, and in ajax-style (without reloading the page), narrow that list down as they were typing in a search term.  Since we had all this data on the client side for display anyway, we thought, hey, can we do the searching there, too?  Pretty quickly we were able to prototype a simple client-side only text index and search.  We ended up being to reuse this code in a few different areas of the website.

API Use

creating an index: 
// creating an index, adding documents
var index = new LADDERS.search.index();
var d = new LADDERS.search.document();
d.add("id", 1);
d.add("text", "one two three");
index.addDocument(d);
d = new LADDERS.search.document();
d.add("id", 2);
d.add("text", "four five six");
d.add("moretext", "ten");
index.addDocument(d);
// etc...
searching: 
// searching
var searchTerms =
["one", "one two", "two one", "four six", "f*", "t*"];

for (var w in searchTerms)
{
  print(searchTerms[w] + " => " + index.search(searchTerms[w]));
}
output: 

one => 1
one two => 1
two one => 1
four six => 2
f* =&g\t; 2
t* => 1,2
Indexing

You can create as many indices as you like in your application.  Each document must have a field labeled "id".  The value of this field will be returned to you if the document matches your search.  Any other fields you add will be searchable once the document is added to the index.  Multi-field search is not currently supported; all fields (except for id) are treated identically.

Searching

Only "AND" search is supported.  That is, only documents that match all the words of your search will be returned, but the order in which those words appear is not important.  Simple wildcarding (*) is supported.

Implementation

Despite the limitations in the current implementation (detailed below), we've found this search functionality to be surprisingly useful.  Additionally, moving this logic directly to the front-end greatly simplifies the backend work, as well as client-server communication and also the number of requests that are necessary.
We're releasing this as open source under the Apache License, available here.

It's a really simple inverted index implementation, and nothing more.  Internally we've experimented a bit with more complex structures (a naive javascript trie implementation), and while this dramatically sped up wildcard searching as one would expect, it also dramatically reduced the indexing speed due to the large number of additional javascript objects created.  In fact, adding any additional information to the index (e.g. offset, for doing phrase matching) slows down indexing speed greatly because of the large number of javascript objects required to hold that information.

Missing Features / TODOs

  • Field-specific searching.
  • Field weighting.
  • Boolean support (i.e. "NOT", "OR").
  • Phrase queries.
  • More complex query expressions, building on above ("a AND b NOT (c OR d)").

Files / Contact

Our javascript search source code is available here under the Apache License

Please report any issues through the javascript search project page.

You can also send any comments, corrections, updates, suggestions to js-search [at] theladders [dot] com.

Posted by Larry Ogrodnek at 12:20 PM | | Comments (0) | TrackBacks (0)

November 01, 2006

Product Requirement Document Samples

The Ladders has created several specs in the annotated, visual format I described in the previous entry. I would like to share some of those, but since they are proprietary, I can only do so upon request. If you would like to see what we've done, please email me at mlimotte@theladders.com.

Posted by Marc Limotte at 02:35 PM | | Comments (1) | TrackBacks (0)