Searching in the Dark

Using the dark net for OSINT investigations. What, where, and how?

While this is an open-source intelligence (OSINT) focused blog, this was not intended to be my first post on OSINT techniques. I would have preferred to start with something more foundational such as how to set up a "sock puppet" or a basic OSINT workflow. But, through my work as a researcher and OSINT analyst at the International Institute for Counter-Terrorism last week, I was given the opportunity to lecture to a group of research interns on the dark net and how it can be leveraged to support OSINT investigations. As I had already created the presentation, sharing it here also felt appropriate. While I will not release the original presentation, many of the points are covered in this blog post.

What is the "Dark Net," and how is it different from other parts of the internet?

A common model circulated in the OSINT community views the internet as an iceberg. As with all models, its intention is to simplify complex concepts to be easily understood. As a result, the model is somewhat simplistic and does not include all of the nuances of the real situation; but, as a tool to understand the basic outline of the internet, it can be very useful.

According to the iceberg model, the internet is divided into three sections: the surface net, the deep net, and the dark net.

Model Outlining the Relationship between the Surface, Deep, and Darknet

Surface Net

The surface net, sometimes referred to as the surface web, is the iceberg area above the water and is thus visible to anyone on the internet. This is comprised of content that is meant for public viewing. It is Wikipedia pages, public business sites, Youtube uploads, and other material for public consumption. To help people find these pages, search engines such as Google, Bing, Yahoo, and DuckDuckGo. These search engines use "crawlers" to read through sites collecting their URLs, metadata, and more daily. Using this collected data, the search engine is able to pull up relevant results when you enter a search query. Put simply, when you search for information about something, the search engine uses its crawled data to identify which web pages are relevant to your search.

Deep Net

While many pages on the internet are intended for public consumption, a substantial amount of pages are not. These pages are protected by paywalls, registration portals, and other means of identity authentication to ensure that only authorized users are able to access the content. Given their restricted nature, these pages are not indexed by search engines, as they are not accessible to surface net crawlers. This network of restricted pages is cumulatively called the deep net, and sometimes referred to as the deep web.

While deep net pages themselves are not publicly accessible, they are often linked to publically accessible pages. A good example of this is your online bank account. While your bank account and internal information are not for public consumption, the bank's home page is. As such, while your bank's home page can be found through search engines, the page hosting your bank's balance cannot.

In addition to personal private information, the deep net is also composed of private business records, medical records, private photo libraries (e.g., Google Photos, iCloud), and content providers (e.g., Netflix).

Dark Net

The dark net, sometimes referred to as the dark web, is a subset of the deep net (in that it is hidden from search engine crawlers and thus not indexed by major search engines). People often confuse the deep net with the dark net or think the terms are interchangeable; they are not. The major difference between the two is how you can access the material. While you can access deep net pages, such as your private banking page, via any web browser (e.g., Chrome, Safari, or Firefox), you cannot access the dark net from these browsers. Web pages on the darknet require specialized browsers to access them and require users to enter the exact URL of the site. While there are a few dark net browsers that exist (I2P, Freenet, TOR), the one most often used is TOR.

Accessing the Dark Net: TOR

What is TOR?

TOR, an acronym for The Onion Router, is a special browser that routes your online traffic (requests, responses, downloads, etc.) through a series of virtual tunnels to hide your traffic from anyone that may be interested in what you are accessing. TOR is able to achieve this through its unique use of relays.

To understand how TOR functions, it is helpful to understand your interactions on the surface net. When you go to a site on the surface web, your computer sends a request to the site for whatever data you are interested in. The webpage receives this request and then issues a response. During this process, there is a direct connection between you and the website that anyone (with the right skills or access) can see.

You asking WiserGuidance.com for a new blog post

While this is fine for normal traffic, sometimes you want more anonymity between you and the site. TOR helps you achieve this through relays. Relays are machines throughout the world that receive traffic through TOR and then route it to other relays through the network until it reaches its final destination. Each of these relays serves as an additional layer that covers your original request (much like an onion has layers. Get why it is called The Onion Router now?).

You asking WiserGuidance.com for a new blog post through TOR

While you can access regular websites through TOR, such as WiserGuidance.com, I wouldn't recommend it. Although these additional relays work to protect your anonymity, they also substantially slow down your requests. Given that each request has to be routed through multiple relays before it reaches its final destination and is processed, the process is much slower than a direct connection to the web page.

Unless you have some need for anonymous internet activity, the primary reason for using TOR should be to access TOR-specific sites. These pages are similar to surface net pages in that they have a URL and host content, but unlike their surface net counterparts, TOR-specific sites can only be accessed through the TOR browser, and their domain names end in ".onion." An example of this is the Central Intelligence Agency TOR address: ciadotgov4sjwlzihbbgxnqg3xiyrg7so2r2o3lt5wz5ypk4sxyjstad.onion

How do I access TOR?

To access TOR, you will need to download the TOR browser from https://www.torproject.org/download/. While other methods exist for accessing TOR, this is the simplest and most secure method for accessing the browser.

Click the image or the link above to be redirected to The Tor Project to download TOR

Once you have downloaded and installed TOR on your machine, you should feel comfortable navigating the platform. Like other browsers, it has a URL address bar, tabs, and bookmarks. Given these features, you could use it as a regular browser to access sites like WiserGuidance.com; I wouldn't recommend it as the connection will be very slow.

Navigating TOR and Using it for Dark Net Investigations

One of the primary reasons you may use TOR, especially if you are reading this post, is for OSINT investigations. During these investigations, you may have to seek out new sources or sites to collect data. How do you find these onions and hidden sites?

Libraries, Repositories, and Wikis

On TOR and the surface web, certain individuals have developed repositories or wikis of different onion links. Like libraries, the founders usually categorize each onion by type of content or use. While many such wikis exist, I will only share a few here.

The Hidden Wiki

The Hidden Wiki is one repository of onion links available online. Each onion is categorized by topic and use. One unique feature of the Hidden Wiki is that it is available both through TOR and regular browsers. The Hidden Wiki is updated regularly, so if you are looking for up-to-date onions, be sure to check it out.

Regular link: https://thehiddenwiki.org/

TOR link: http://paavlaytlfsqyvkg3yqj7hflfg5jw2jdg2fgkza5ruf6lplwseeqtvyd.onion/

The Hidden Wiki on TOR

Finding New Onion URLs

Dark Net Search Engines

While the dark net is not indexed by traditional search engines such as Google or Bing, specialized search engines have been developed for finding onions. Below are a few search engines that are specifically for onion URLs.

Torch: http://torch4st4l57l2u2vr5wqwvwyueucvnrao4xajqr2klmcmicrv7ccaad.onion

Haystak: http://haystak5njsmn2hqkewecpaxetahtwhsbsa64jom2k22z5afxhnpxfid.onion

Tor66: http://tor66sewebgixwhcqfnp5inzp5x5uohhdy3kvtnyfxc2e5mxiuh34iid.onion/

Surface Net Search Engines

In addition to the dark net search engines, some surface net sites have attempted to crawl and index onion URLs. While there are several, one of the most popular is Ahmia.fi. Remember that while you can search for onion URLs, these can only be accessed through TOR.

Dark Net Monitors and Aggregators

Certain sites offer services that periodically crawl the un-indexed onion pages to create an up-to-date listing of active, down, and primary language sites. One of the most popular ones is the Hunchly Daily Darkweb Report. After signing up with your email, you will receive a daily spreadsheet with up-to-date information on link.

Example of the Hunchly Darkweb Report

Disclaimer:

Operating on the dark net has inherent risks, including, but not limited to, unintended disclosure of personal information and exposure to malware. The author of this article does not assert that this piece is exhaustive or provides the necessary knowledge to protect from potential dangers. Any action you take based on this blog's information is strictly at your own risk. Under no circumstances will WiserGuidance, Alexander Pack, any affiliates, or partners be held liable for any losses or damages resulting from your use of the information contained on this website.*