We're very excited to announce that you can now run custom code on 80legs.  We have just released version 0.8, which gives users the ability to write their own content analysis logic using processDocument() and their own link extraction logic using parseLinks().  For more information on how to write and run code on 80legs, please visit http://80legs.pbworks.com/Custom-Code. The total list of changes in this release include:

• Custom code initial release (first IWebAnalysisConnector release with parseLinks() and processDocument())
• Option to analyze specific MIME types
• Option to preserve query strings when crawling
• Resulting crawl list shows status codes and other reasons for failing to crawl (e.g. robots.txt, DNS, etc)
• Better handling of failed URLs
• Sandbox server for testing custom code on your own machine using the 80legs framework.
• Stop problem jobs automatically

We've also granted access to several more users on our private beta list.  If you haven't received access yet, but would really like to get access soon, please let us know, and we'll try and include you in the next set of beta users. We're already working on the new features, such as:

• A web service for programmatically submitting and managing jobs
• An "app store" that will allow users to run pre-built applications developed by trusted third-parties
• Our payment system, which will be released first as a "demo", allowing users to get used to the system before actually requiring payment