SERP and On-Page Data API (Automate your SEO)

Looking for a SERP and On-Page API returning 

• The top 100 results on Google Search for any keyword, country, location or device
• Statistics about word count, image count, content update
• Headlines structure and content body for any pages
• One page data such as metadata, schema and links
• Keywords density and N-grams
• And more… 

You are at the right place. 

With thruuu, you have an

• awesome SERP analyser and
• an API to access all the data from Google Search programmatically and
• scrape individual web pages to get On-Page data.

You can now automate your SERP Analysis with an awesome SERP API.

Your first step with an awesome SERP and On-Page API

The first step is to create an account with thruuu, a free SERP Analyzer to access the SERP API.

Once you are logged in, open the menu and visit the API view. 

To use the API, you need two things:

• An API Key
• A webhook endpoint

The API Key allows you to access the API programmatically. You can generate a new API key by clicking on “Generate Key” from the interface. This key will have to be present in every request made to the API. 

The second parameter is the webhook endpoint. 

The thruuu API uses webhooks to notify your application when a SERP has been successfully scraped. The webhook endpoint is an URL configured on your application (or server) that receives data from the thruuu API. 

You should enter a valid endpoint and click on “Update endpoint”. 

We will detail later how to configure your server to get notified by thruuu, or just watch the video below.

Scrape the SERP or scrape web pages by API

With thruuu SERP Scraper, you can access the following services or endpoints from the API:

• Scraping the SERP: you get a complete overview of the Google Search Result Page, every single result is scraped and key information extracted.
• Scraping a web page: you get a lot of on-page information such as headlines, word count, keyword density and more from a single URL.

In the following sections, we will see in detail how to access each of the services provided by the thruuu API with examples in NodeJS.

Endpoint: Scrape the SERP

The endpoint returns a complete Google SERP, statistics and on-page data for any result. To receive the data back from thruuu, you will need to have a webhook endpoint setup on your application.

To make a call to the endpoint, you need to make the following HTTP request:

GET  https://api.thruuu.com/api/v1/scrape-serp

With the following parameters:

q	The search query (mandatory)
apikey	The api key generated by thruuu (mandatory)
search_engine	The google search engine (optional, default: google.com)
location	The geolocalisation (optional)
country	The country code (optional, default: US)
language	The language code (optional, default: en)
device	The device – desktop or mobile (optional, default: mobile)
num	The number of search results. Values can be 1 – 100 (optional, default: 10)

Here is an example to call the endpoint from a Browser:

https://api.thruuu.com/api/v1/scrape-serp?apikey=YOUR_API_KEY&q=what is thruuu&search_engine=google.com&location=&country=US&language=en&device=desktop&num=10

As the process can take more than 30 seconds, thruuu will notify the webhook endpoint defined on your application and return a JSON object containing the result in a data object.

Follow this example in NodeJS to set up an HTTP endpoint on your server to accept a request with a POST method:

const express = require('express');
const app = express();
 
// Match the raw body to content type application/json
// If you are using Express v4 - v4.16 you need to use body-parser, not express, to retrieve the request body

app.post('/test-webhook', express.json({ type: 'application/json', limit: '50MB' }), async (req, res) => {
 console.log('request from thruuu is coming');
 if (req.body !== undefined && req.body.data !== undefined) {
   console.log('request from thruuu is valid');
   console.log(`query -> ${req.body.data.query}`);
   console.log(`event type -> ${req.body.data.eventType}`);
 }
 // Return a response to acknowledge receipt of the event
 res.json({ received: true });
});
 
app.listen(8000, () => console.log('Running on port 8000'));

From there, you can manipulate a JSON object containing the following main (but not only) properties:

query	The search query
result	The search engine result page containing all scraped pages
related _searches	The related searches displayed on Google
similar_questions	Similar questions found across the result page
Keywords_*	Keywords frequency for different elements such as title, description, etc…
num	The number of search results. Values can be 1 – 100 (optional, default: 10)

Endpoint: Scrape a web page

The endpoint returns a scrape of a web page, including on-page and NLP information. You don’t need to have a webhook endpoint setup on your application.

To make a call to the endpoint, you need to make the following HTTP request:

GET  https://api.thruuu.com/api/v1/scrape-url

With the following parameters:

url	The url of the page to scrape (mandatory)
apikey	The api key generated by thruuu (mandatory)
language	The language of the content

Here is an example to call the endpoint from a Browser:

https://api.thruuu.com/api/v1/scrape-url?apikey=YOUR_API_KEY&url=https://samuelschmitt.com/what-is-a-topic-cluster&language=en

From there, you can manipulate a JSON object containing the following main (but not only) properties:

title	The title of the page
h1, h2, h3	The headlines of the page
wordCount	The word count
toc	The table of content of the page
comment_questions	All the questions found in the comment section
body	The body of the page
kwBody	The most frequent terms found in the body.