Metadata

School: 

St. Lawrence University

Language: 

English

Language: 

English

Record Type: 

Student paper

Identifier:

oai:dspace.lasrworks.org:10349/931

Creator: 

Denu, Doug

Contributor: 

Harcourt, Ed

Contributor: 

St. Lawrence University. Department of Math, Computer Science and Statistics

A web crawler is a program that scours the Internet moving from website to website. Web crawlers have many different purposes, such as sending out junk mail, finding dead links within a domain, and searching websites and databases for relevant information like that of Google. This project focuses on using a web crawler to map the hierarchy of links within a particular domain. Starting at the St. Lawrence University’s home page, the web crawler gathers all the links that are found while crawling the St. Lawrence domain. The software is written in the Python programming language and uses another piece of software called Graphviz to graph the links once the links have been gathered.