Free cybersecurity APIs for looking up Indicators of Compromise (IOCs).

Article

Top cyber APIs

APIs can provide access to threat intelligence feeds and other cybersecurity-related data. In this article, we will cover the following great APIs, which you can access for free.

• Shodan
• Google safe browsing
• PhishTank
• VirusTotal
• GreyNoise
• RiskIQ / PassiveTotal
• URLScan
• AlienVault Open Threat Exchange
• SecurityTrails
• IPinfo
• WhoisXMLAPI
• Spamhaus
• mnemonic
• ThreatFox (from abuse.ch)

Read on to learn more about APIs and how to use them before diving into each of the different infosec APIs covered. Whether you are a bug bounty hunter, a SOC analyst, or something in between, we've explained what data is provided for each API, tips on using it, and any limitations. You can find links to each one in the references section for further information.

What is an API?

API stands for 'Application Programming Interface'. For a long time, software developers have exposed an API to allow third-party programs to reuse functionality. For example, operating systems (OS) provide a network socket API that developers can use to access the networking functionality of the OS. To use an API, developers must first read the API documentation! Then, they must write code to make the necessary function calls to the originating software.

It is now common practice for web services to use APIs to provide access to data programmatically. API access allows consumers of the service to write code that talks directly to the web service rather than using web pages that make up the graphical front end. There are two key benefits to offering API access to web services:

• It allows the integration of multiple services and data sources.
• It supports power users who want to access and manipulate services programmatically.

Web APIs are typically accessed with HTTP requests and return data in JSON format.

What can I use APIs for in cybersecurity

A wide range of cybersecurity-focused APIs is available which can support both red and blue team members. You can use these API's to learn more about assets and to access threat feeds. API's can provide enrichment and context to support your engagements and investigations. You might choose to use APIs directly or integrate them with your SIEM (Security Information and Event Management system) or threat intelligence platform (TIP).

We can loosely group security-focused APIs into two categories:

• Services that provide an aperture onto the internet. These APIs will give you information about the public IP space, 'whois' information and more. They can help you understand your own assets and any other infrastructure that your network interacts with.

• Services that focus on threat. These threat intelligence APIs can help understand whether a specific file or IP address is malicious. Typically, you can use them to lookup an IOC (Indicator of Compromise) you have observed to learn if it is associated with a known threat.

How can I access APIs

First and foremost, read the API documentation. This will tell you what data is provided by the API and how to access it. Good documentation will usually give you code snippets that you can copy or examples that can be run directly from the documentation pages.

Some APIs are entirely open and don't require authentication, but many require you to sign up and obtain an API key (even if they're free to use). You will then include the key with requests which you make to the API. Your unique key/token will allow you (and the service provider) to keep track of usage quotas and control which resources you can access.

You may already have access to a SIEM (security information and event management platform) or TIP (threat intelligence platform), which can integrate directly with third-party API's. Integration may be as simple as a one-click operation, or you may need to create a separate account with the API provider to obtain an API key.

If you aren't looking to integrate the API into another platform, you can access APIs directly with dedicated tools or programmatically by writing code. When interacting with APIs directly, you have to create HTTP requests which typically include your API key for authentication and tell the service what data you would like to access. Some API calls will be made using 'GET' HTTP requests, whilst others use 'POST' requests.

One common approach is to use curl, a command-line tool available on GNU/Linux (and other platforms) which can make HTTP requests. The example below shows how you can use curl to access data through the GreyNoise API.

curl --request GET --url https://api.greynoise.io/v3/community/8.8.8.8 \
     --header 'Accept: application/json' --header 'key: <INSERT API KEY HERE>'

In the example above, we make a 'GET' request to the GreyNoise community API URL. We specify the IP address we want to look up (8.8.8.8) directly in the URL. We use the --header option to tell curl what to put in the header, including the type of data we will accept back (JSON) and our API key for authentication.

If you prefer, you could use a graphical API client such as Postman.

Finally, you can make API calls using the programming language of your choice. Python is a common approach, and many developers use the 'requests' library to do this. However, all popular languages support making HTTP requests. Many mature APIs will have a client library or SDK (Software Development Kit) in one or more languages, making it even easier to work with. The example below shows how we can use Python to make a request to the GreyNoise API equivalent to the curl example above (taken from the GreyNoise documentation).

import requests

url = "https://api.greynoise.io/v3/community/8.8.8.8"

headers = {
    "Accept": "application/json",
    "key": "<INSERT API KEY HERE>"
}

response = requests.request("GET", url, headers=headers)

print(response.text)

Are they really free?

Yes, but you will need to read the terms for each service. Some infosec APIs provide free access solely for personal use, whilst others offer a heavily constrained version of their commercial services for free.

The APIs listed below should give you a good idea of what's out there and provide the opportunity to start interacting with them. Becoming familiar with security and threat APIs will help you understand what kind of data is available and how you might use them in the future.

Shodan

Shodan describes itself as:

"The world's first search engine for Internet-connected devices."

The site contains data from regular internet-wide scans across a range of ports. This information can show you what ports are open for a given IP address and provide some insights into the products/services it's running.

Access Requirements: to use the Shodan API, you must create a free account.

Free Account Limitations: using a free account will limit the number of results you can retrieve and the types of queries you can run. For more flexibility, you can pay a one-off fee for a lifetime membership.

What data is available: At its most basic, you can search based on a keyword (such as Apache) or IP address (such as 8.8.8.8). The data returned will include metadata about the IP address (including estimated location and owner) and information about open ports and, where available, the services and operating systems that are running.

How to use the API: Shodan provides an official Python client with unofficial clients in other languages. You can also interact with the API directly.

Examples

The example below shows how to use Python to access Shodan data for an IP address. The sample code is taken from the official repository README.

from shodan import Shodan

api = Shodan('MY API KEY')

# Lookup an IP
ipinfo = api.host('8.8.8.8')
print(ipinfo)

You can also access Shodan from the command line using curl:

curl -X GET "https://api.shodan.io/shodan/host/8.8.8.8?key={YOUR_API_KEY}"

Google Safe Browsing

Safe Browsing is a service provided by Google which can be used to check a URL against Google's list of dangerous websites. The Safe Browsing list includes websites that Google has identified as hosting phishing pages or malicious software downloads.

Access Requirements: To get started, you will need a Google account. Then you can create a Google Developer Console project which will allow you to generate an API key and activate the Safe Browsing APIs.

Free Account Limitations: A free account will provide full access to the API, but this is for non-commercial use only. Google defines commercial use a 'for sale or revenue-generating purposes. They provide a separate 'Web Risk' Google Cloud service for commercial use.

What data is available: The API has two primary endpoints, the 'Lookup API' and the 'Update API'. The Lookup API allows you to query for a URL and receive a response back about whether that URL is on a list (i.e. it's malicious) along with metadata about what type of threat Google associates with that site. The Update API is intended for users who wish to host some Safe Browsing data locally to allow local queries which don't all have to be sent to Google's servers.

How to use the API: The Lookup API uses HTTP POST requests to look up one or more URLs. You need to include your API key as a URL parameter and then provide information about the query in the POST body.

PhishTank

Similar to Google Safe Browsing, PhishTank maintains a list of suspected and verified website addresses that are used for phishing. You can browse the list directly on their website or access the data through the API. Data on the site is provided and maintained by the community.

Access Requirements: anyone can use the PhishTank API. You can use it with or without an API key.

Free Account Limitations: there are no limits placed on free API access with PhishTank. If you don't register for an API key, you will face heavier rate-limiting when making requests.

What data is available: Their simple API lets you check whether a particular URL has been submitted to PhishTank.

How to use the API: you can access the 'simple API' at http://checkurl.phishtank.com/checkurl/. You must make an HTTP POST request to that URL with the following parameters:

• url - the URL you want to check
• format - the data format (XML, PHP or JSON) you want the response to use
• app_key - an optional API key you can get by registering on the site.

VirusTotal

VirusTotal (VT) is an online repository of suspicious files, including a large amount of malware. Anyone can upload a file or submit a URL to be analysed and shared with the community. This vast sea of data can then be queried by anyone looking to check if a file or URL has previously been submitted. To search for a file without sharing it with the community, you should search based on the file's hash (such as the MD5 or SHA256 hash) of the file rather than uploading it. You can also use VirusTotal's powerful features to learn more about malware and 'pivot' to potentially related samples.

Access Requirements: To use VirusTotal's Public API, you will need to sign up for a VirusTotal Community account. Once you have an account, you can obtain an API key and start making requests.

Free Account Limitations: The number of API requests you can make with the Public API is restricted to 500 per day and no more than 4 per minute. It is for non-commercial use only - see the website for full terms and conditions.

What data is available: There are a large number of endpoints that you can call, allowing a range of queries and enrichments. As well as submitting new files (don't forget that you may prefer to search based on hash rather than uploading the file), you can run queries based on a wide range of features that VirusTotal generates for each upload. You can also query for network indicates such as domains, IP addresses, and URLs, which malware might use for command and control (C2) communications.

How to use the API: The VT API requires any requests to have an 'x-apikey' header which includes your API key. Successful responses will generally return a 200 OK with a JSON body.

GreyNoise

GreyNoise data is based on an 'internet-wide sensor network passively collects packets from hundreds of thousands of IPs seen scanning the internet every day.' This represents the general, untargeted background noise of the internet. Some of those IPs will be malicious, trying to exploit unprotected devices on the internet. However, many of the IPs represent benign scanning by security researchers or services such as Shodan. Therefore, GreyNoise can help security analysts understand whether IP addresses scanning their assets are targeted or have malicious intent.

Access Requirements: To get started with the GreyNoise API, you will need to sign up for a free Community account. Registering will allow you to obtain an API key.

Free Account Limitations: GreyNoise has a dedicated Community API which provides a subset of the contextual information GreyNoise has available. Additional advanced features are available with a subscription.

What data is available: For IP addresses that are present in the dataset, the GreyNoise Community API will provide basic information on the IP, including any associated name (such as the owner organisation), classification (is it benign or malicious) and some additional context.

How to use the API: To query the GreyNoise API for a specific IP address, you can use a GET request including the IP address in the URL path. You will also need to include your API key in the header of the request.

Examples

See below for an example of how to query the Community API using Python.

import requests

url = "https://api.greynoise.io/v3/community/8.8.8.8"

headers = {
    "Accept": "application/json",
    "key": "<INSERT API KEY HERE>"
}

response = requests.request("GET", url, headers=headers)

print(response.text)

RiskIQ community?

RiskIQ have several products designed to help organisations and security researchers understand assets exposed to the internet. We will be focusing on PassiveTotal, which provides intelligence and context related to IP addresses, domains, certificates and more. This rich dataset allows you to pivot from one indicator to another to understand internet infrastructure better.

Access Requirements: To access the RiskIQ Community dataset and API, you'll need to register for a free account. You can then obtain an API key from your account page.

Free Account Limitations: The number of searches you can run and elements such as how many days history for a given record you can see will be limited with a free account. Enterprise products are available for commercial use.

What data is available: As well as historical 'whois' data and passive DNS records, RiskIQ provides additional enrichment about what services have been seen on a given IP address. Researchers and analysts can also use tracker IDs and JARM hashes to pivot to related infrastructure. The PassiveTotal dataset also provides infrastructure classification such as newly observed domains/hosts, Tor nodes and malware/phishing analytics.

How to use the API: There are client applications available in Python and Rust which provide an easy to use wrapper around the API. You can also use HTTP requests directly with your API key in the header. You will need to Base64 encode your API key and use it as follows in the HTTP headers - Authorization: Basic <Base64 Encoded API Key>.

URLScan

URLScan is a free service to scan and analyse public websites - it is used to scan thousands of websites every day. You give URLScan a URL (Uniform Resource Locator - the address of a web page), and then it will visit the page like a user and return the data it receives and an analysis of it. This is helpful for obtaining information about a suspicious website without having to access it directly. The data presented can help you make an informed decision as a security analyst as to whether or not a URL is suspicious. Furthermore, you can use the information returned to pivot to related websites and infrastructure.

You can choose whether scans you run are Public (the scan shows up publicly for others to see and search), Unlisted (the scan is visible to vetted security researchers and organisations) or Private (no one else will see the URL you submitted).

Access Requirements: Access to URLScan is free, and you can choose whether or not to register.

Free Account Limitations: Unauthenticated users receive only a small quota for running queries. Registered users receive more generous allocations.

What data is available: URLScan return a mix of raw information and analysis. The raw data returned includes:

• Domains and IP addresses contacted
• Resources requested (such as CSS, javascript and images)
• Cookies

Enrichments and analysis provided includes:

• IP address geolocation
• IP address ASN (Autonomous System Number)
• Maliciousness score
• Maliciousness category (if applicable)

How to use the API: to use the API with a free API key, you need to include that as an 'API-Key' header and submit a POST request including a 'url' parameter and optionally a 'visibility' parameter (public, unlisted or private).

Examples

An example of querying the URLScan API is given below (taken from the URLScan website).

import requests
import JSON

API_KEY = <YOUR API KEY>

headers = {'API-Key': API_KEY,'Content-Type': 'application/json'}
data = {"url": "https://example.com/path/", "visibility": "unlisted"}

response = requests.post('https://urlscan.io/api/v1/scan/',
    headers=headers, data=json.dumps(data))

print(response)
print(response.json())

AlienVault

AlienVault Open Threat Exchange (OTX) is a threat intelligence community platform with thousands of users and millions of threat indicators. Security analysts can use the AlienVault OTX API to help determine if activity seen on a network is malicious.

Access Requirements: To access the API, you must register for a free account.

Free Account Limitations: OTX is free; AlienVault also offer an open-source endpoint product, 'OSSIM'. For additional functionality, they offer AlienVault Unified Security Management - a commercial, cloud-based security monitoring platform.

What data is available: OTX has a range of information available to query. The simplest use case is to look up an indicator seen in your environment (IP address, domain, file hash etc.) for additional context (such as geodata) and any information AlienVault has on reputation and related malware.

How to use the API: To use the API, you will need to include your AlienVault API key as an "X-OTX-API-KEY" header. You can use CURL to query the API directly or use one of the available software development kits (SDKs) for Python, Java or Golang.

Examples

For example, the following code (taken for the OTX GitHub) demonstrates how to use their Python library to query the API.

from OTXv2 import OTXv2
from OTXv2 import IndicatorTypes

otx = OTXv2("API_KEY")

# Get everything OTX knows about example.com
otx.get_indicator_details_full(IndicatorTypes.DOMAIN, "example.com")

SecurityTrails

SecurityTrails provide a 'total internet inventory' to help organisations understand threats and their own attack surface. With the SecurityTrails API, you have access to over 3 trillion historical DNS lookups as well as a range over other IP and domain data.

Access Requirements: To access the SecurityTrails API, you will need to register for a free account.

Free Account Limitations: Free accounts only have access to a subset of data and a limit of 50 queries per month.

What data is available: On the free plan, you can use the SecurityTrails API to access current and historical DNS records and subdomains as well as IP address research. You don't get access to WHOIS data or more advanced features.

How to use the API: You need to set up an account to get an API key. Your key should then be included in requests as an 'apikey' header.

Examples

An example request to lookup DNS information using a GET request with CURL is given below:

curl --request GET \
  --URL https://api.securitytrails.com/v1/history/example.com/dns/a \
  --header 'apikey: <YOUR API KEY>'

IPinfo

As the name suggests, IPinfo is a great source for information about an IP address. They handle billions of API requests a month, providing contextual information such as geolocation and ownership data for Ips.

Access Requirements: To access the API, you will need to register for a free account.

Free Account Limitations: The free plan only provides access to geolocation data, with a limit of 50,000 requests per month.

What data is available: As mentioned above, the free plan only provides geolocation data. Paid subscriptions also get access to information such as associated Autonomous System Number (ASN) and additional classification information (such as if the IP is a known VPN node).

How to use the API: you can access the IPinfo using HTTP GET requests and presenting your API token as an HTTP Basic Auth username, a bearer token, or a token URL parameter.

Examples

Querying an IP address with IPinfo is simple:

curl ipinfo.io/1.1.1.1?token=<YOUR API TOKEN>

whoisxmlapi

WhoisXMLAPI offers a range of services related to domain and IP data intelligence. Here we will look at their WHOIS API. The API lets you query by IP address, domain or email address to get related WHOIS data. Every domain has 'WHOIS' data related to who has registered it, who the registrar is and more. Despite having XML in their name, the API does offer data in JSON format too.

Access Requirements: to use the WHOIS API, you will need to register for a free account.

Free Account Limitations: free accounts can make up to 500 API requests per month.

What data is available: data available for domains includes: registrant name, organisation, email address, registration address, registrar information, creation date, expiration date, updated date, domain availability, and domain age.

How to use the API: to use the API, you can simply perform an HTTP GET request, including your API key and the domain to lookup as URL query components.

Example

The following example shows how you would look up WHOIS data for 'example.com'. You would need to insert your own WhoisXMLAPI API key.

curl -X GET "https://www.whoisxmlapi.com/whoisserver/WhoisService?apiKey=<YOUR API KEY>&domainName=example.com"

Spamhaus - Passive DNS

Spamhaus focus on protecting mailboxes and provide another source of IP and domain intelligence. Passive DNS data is a collection of historical DNS queries that can help understand how DNS records and related infrastructure have changed over time. Spamhaus provide access to their passive DNS data through a web interface or API.

Access Requirements: to access the Spamhaus Passive DNS API, you will need to register for a free account.

Free Account Limitations: free Spamhaus accounts are limited to 20 queries per day and 200 queries per month.

What data is available: their passive DNS database holds a wide range of historic DNS records, including A, AAAA, MX, CNAME, TXT, SPF and more. You can use this information to understand the connections between related infrastructure.

How to use the API: To get data from the API, you need to obtain an AUTH TOKEN to include as an authorisation bearer with requests by using the login API. You can then run searches on different fields such as forward and reverse DNS lookups. Consult their technical documentation for further information and examples.

mnemonic

Another source of passive DNS data is mnemonic. They are a Norwegian company offering a range of cybersecurity-focused products and services. You can access their passive DNS data through a web interface or using the API.

Access Requirements: you can access the mnemonic passive DNS data for free, without an account.

Free Account Limitations: unauthenticated users are limited to 100 requests per minute and 1000 requests per day. If you exceed the limit, you will receive an HTTP 402 response.

What data is available: querying for a domain will return information about related DNS records, including the DNS answer (e.g. An IP address), the number of times the record has been seen, as well as the first and last seen dates.

How to use the API: you just need to give the domain that you want to query in an HTTP GET request. You can also specify additional query parameters, such as which types of records to return.

Examples

A simple lookup of example.com would look like this.

curl https://api.mnemonic.no/pdns/v3/example.com

ThreatFox API - abuse.ch

This project, run by a university in Switzerland, offers a range of open data to support security researchers. Projects run by abuse.ch include:

• Malware Bazaar
• Feodo Tracker (botnet tracker)
• I Got Phished
• SSL Blacklist
• URL Haus
• Threat Fox

It is worth visiting their website to learn more about their work. Here we'll focus on just ThreatFox, which provides an API for querying interesting datasets about malicious activity.

Access Requirements: To submit data through abuse.ch APIs, you'll need to obtain an API key by logging in with a Twitter account. Queries don't require an account.

Free Account Limitations: There are no limitations on free accounts. Some abuse.ch projects offer additional functionality for vetted security researchers.

What data is available: The ThreatFox dataset contains thousands of indicators of compromise (IOCs) submitted by security researchers. The ThreatFox API allows you to lookup an indicator of compromise (such as a file hash, domain, URL or IP address) to see whether someone has reported it as malicious.

How to use the API: to search for an IOC, you can use a simple POST request.

Examples

The following API example is taken from the ThreatFox documentation.

curl -X POST https://threatfox-api.abuse.ch/api/v1/ \
     -d '{ "query": "search_ioc", "search_term": "139.180.203.104" }'

---

References

Learn more about this topic by checking out these references.

---