[rdfweb-dev] [ANN] Java RDF crawler

Mon Apr 21 13:57:33 UTC 2003

Just released: version 0.1 of an RDF crawler (aka scutter) using Java
and Jena that spiders the web (following rdfs:seeAlso) gathering up RDF
data and storing it in any of Jena's backend stores (in-memory, Berkeley
DB, mysql, etc). It does multithreaded downloading, and retains
provenance information which it uses to maintain consistency over
multiple runs.

The code's only had me as a user so far; comments and corrections highly
appreciated.

More information at http://www.hackdiary.com/archives/000030.html

Cheers,
Matt.