Guide to Investigative Web Research

This introductory guide to investigative web research (IWR) was created by The Engine Room, an international organisation working to help activists, organisations, and other social change agents make the most of data and technology to increase their impact. As part of their aim to encourage shared, open documentation about the use of technology in social change, they are publishing it so that other people can use it, too. The guide is designed for researchers, activists, and journalists who need to collect online information about people, entities, or events and use it for investigative research or advocacy.

The Engine Room explains that the internet can be an effective and efficient tool for investigative research - helpful for those who are, for example, tracking corporate ownership and corruption or web page activity, or who are mapping political influence. However, web research can also turn up information that is messy, inaccurate, outdated, or incomplete. IWR tools can help to mitigate against some of these risks. IWR tools can be used, for example, to discover otherwise hidden information, collect and store large amounts of data, or refine search engine results. The guide is designed to be practical and straightforward, pointing the reader to more detailed resources and providing the context to decide what tools he or she might need. The key areas of focus are:

• How to document and store web pages
• How to acquire information on who owns websites
• How to extract structured information from web pages
• Key investigative databases
• Advanced search

In addition to describing tools and techniques for IWR, it explores pros and cons, challenges, and implementation, also providing links to additional IWR resources. The guide's source code is available on Github.