Link ScraperLink Scraper

Token Usage:10 per call

Link Scraper is a simple tool for scraping web page links. It returns all the links on a web page.

This API provides reliable and fast access to link scraper data through a simple REST interface. Built for developers who need consistent, high-quality results with minimal setup time.

To use this API, you need an API key. You can get one by creating a free account and visiting your dashboard.

View API in Directory

Endpoint

POST Request
POST https://api.apiverve.com/v1/linkscraper

Code Examples

Here are examples of how to call this API in different programming languages:

cURL Request
curl -X POST \
  "https://api.apiverve.com/v1/linkscraper" \
  -H "X-API-Key: your_api_key_here" \
  -H "Content-Type: application/json" \
  -d '{
  "url": "https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/concepts.html",
  "maxlinks": 20,
  "includequery": false
}'
JavaScript (Fetch API)
const response = await fetch('https://api.apiverve.com/v1/linkscraper', {
  method: 'POST',
  headers: {
    'X-API-Key': 'your_api_key_here',
    'Content-Type': 'application/json'
  },
  body: JSON.stringify({
    "url": "https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/concepts.html",
    "maxlinks": 20,
    "includequery": false
})
});

const data = await response.json();
console.log(data);
Python (Requests)
import requests

headers = {
    'X-API-Key': 'your_api_key_here',
    'Content-Type': 'application/json'
}

payload = {
    "url": "https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/concepts.html",
    "maxlinks": 20,
    "includequery": false
}

response = requests.post('https://api.apiverve.com/v1/linkscraper', headers=headers, json=payload)

data = response.json()
print(data)
Node.js (Native HTTPS)
const https = require('https');
const url = require('url');

const options = {
  method: 'POST',
  headers: {
    'X-API-Key': 'your_api_key_here',
    'Content-Type': 'application/json'
  }
};

const postData = JSON.stringify({
  "url": "https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/concepts.html",
  "maxlinks": 20,
  "includequery": false
});

const req = https.request('https://api.apiverve.com/v1/linkscraper', options, (res) => {
  let data = '';
  res.on('data', (chunk) => data += chunk);
  res.on('end', () => console.log(JSON.parse(data)));
});

req.write(postData);
req.end();
PHP (cURL)
<?php

$ch = curl_init();

curl_setopt($ch, CURLOPT_URL, 'https://api.apiverve.com/v1/linkscraper');
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_CUSTOMREQUEST, 'POST');
curl_setopt($ch, CURLOPT_HTTPHEADER, [
    'X-API-Key: your_api_key_here',
    'Content-Type: application/json'
]);

curl_setopt($ch, CURLOPT_POSTFIELDS, json_encode({
    'url': 'https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/concepts.html',
    'maxlinks': 20,
    'includequery': false
}));

$response = curl_exec($ch);
curl_close($ch);

$data = json_decode($response, true);
print_r($data);

?>
Go (net/http)
package main

import (
    "fmt"
    "io"
    "net/http"
    "bytes"
    "encoding/json"
)

func main() {
    payload := map[string]interface{}{
        "url": "https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/concepts.html",
        "maxlinks": "20",
        "includequery": "false"
    }

    jsonPayload, _ := json.Marshal(payload)
    req, _ := http.NewRequest("POST", "https://api.apiverve.com/v1/linkscraper", bytes.NewBuffer(jsonPayload))

    req.Header.Set("X-API-Key", "your_api_key_here")
    req.Header.Set("Content-Type", "application/json")

    client := &http.Client{}
    resp, err := client.Do(req)
    if err != nil {
        panic(err)
    }
    defer resp.Body.Close()

    body, _ := io.ReadAll(resp.Body)
    fmt.Println(string(body))
}
Ruby (Net::HTTP)
require 'net/http'
require 'json'

uri = URI('https://api.apiverve.com/v1/linkscraper')
http = Net::HTTP.new(uri.host, uri.port)
http.use_ssl = true

payload = {
  "url": "https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/concepts.html",
  "maxlinks": 20,
  "includequery": false
}

request = Net::HTTP::Post.new(uri)
request['X-API-Key'] = 'your_api_key_here'
request['Content-Type'] = 'application/json'

request.body = payload.to_json

response = http.request(request)
puts JSON.pretty_generate(JSON.parse(response.body))

Authentication

This API requires authentication via API key. Include your API key in the request header:

Required Header
X-API-Key: your_api_key_here

Parameters

The following parameters are available for this API endpoint:

ParameterTypeRequiredLocationDescriptionExample
urlstringYesbodyThe URL of the web page to scrape links fromhttps://docs.aws.amazon.com/AWSEC2/latest/UserGuide/concepts.html

Response

The API returns responses in JSON, XML, and YAML formats:

Example Responses

JSON Response
200 OK
{
  "status": "ok",
  "error": null,
  "data": {
    "url": "http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/concepts.html",
    "linkCount": 16,
    "links": [
      {
        "text": "",
        "href": "http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/concepts.html/pdfs/AWSEC2/latest/UserGuide/ec2-ug.pdf#concepts",
        "external": false
      },
      {
        "text": "Documentation",
        "href": "http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/concepts.html/index.html",
        "external": false
      },
      {
        "text": "Amazon EC2",
        "href": "http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/concepts.html/ec2/index.html",
        "external": false
      },
      {
        "text": "User Guide",
        "href": "concepts.html",
        "external": true
      },
      {
        "text": "Amazon EC2 Instance Types Guide",
        "href": "https://docs.aws.amazon.com/ec2/latest/instancetypes/instance-types.html",
        "external": true
      },
      {
        "text": "PCI DSS Level 1",
        "href": "https://aws.amazon.com/compliance/pci-dss-level-1-faqs/",
        "external": true
      },
      {
        "text": "Amazon EC2 Auto Scaling",
        "href": "https://docs.aws.amazon.com/autoscaling/",
        "external": true
      },
      {
        "text": "AWS Backup",
        "href": "https://docs.aws.amazon.com/aws-backup/",
        "external": true
      },
      {
        "text": "Amazon CloudWatch",
        "href": "https://docs.aws.amazon.com/cloudwatch/",
        "external": true
      },
      {
        "text": "Elastic Load Balancing",
        "href": "https://docs.aws.amazon.com/elasticloadbalancing/",
        "external": true
      },
      {
        "text": "Amazon GuardDuty",
        "href": "https://docs.aws.amazon.com/guardduty/",
        "external": true
      },
      {
        "text": "EC2 Image Builder",
        "href": "https://docs.aws.amazon.com/imagebuilder/",
        "external": true
      },
      {
        "text": "AWS Launch Wizard",
        "href": "https://docs.aws.amazon.com/launchwizard/",
        "external": true
      },
      {
        "text": "AWS Systems Manager",
        "href": "https://docs.aws.amazon.com/systems-manager/",
        "external": true
      },
      {
        "text": "Amazon Lightsail",
        "href": "https://docs.aws.amazon.com/lightsail/",
        "external": true
      },
      {
        "text": "Amazon Lightsail or Amazon EC2",
        "href": "https://docs.aws.amazon.com/decision-guides/latest/lightsail-or-ec2/lightsail-or-ec2.html",
        "external": true
      }
    ],
    "maxLinksReached": false
  }
}
XML Response
200 OK
<Root>
  <status>ok</status>
  <error />
  <data>
    <url>http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/concepts.html</url>
    <linkCount>16</linkCount>
    <links>
      <text></text>
      <href>http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/concepts.html/pdfs/AWSEC2/latest/UserGuide/ec2-ug.pdf#concepts</href>
      <external>false</external>
    </links>
    <links>
      <text>Documentation</text>
      <href>http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/concepts.html/index.html</href>
      <external>false</external>
    </links>
    <links>
      <text>Amazon EC2</text>
      <href>http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/concepts.html/ec2/index.html</href>
      <external>false</external>
    </links>
    <links>
      <text>User Guide</text>
      <href>concepts.html</href>
      <external>true</external>
    </links>
    <links>
      <text>Amazon EC2 Instance Types Guide</text>
      <href>https://docs.aws.amazon.com/ec2/latest/instancetypes/instance-types.html</href>
      <external>true</external>
    </links>
    <links>
      <text>PCI DSS Level 1</text>
      <href>https://aws.amazon.com/compliance/pci-dss-level-1-faqs/</href>
      <external>true</external>
    </links>
    <links>
      <text>Amazon EC2 Auto Scaling</text>
      <href>https://docs.aws.amazon.com/autoscaling/</href>
      <external>true</external>
    </links>
    <links>
      <text>AWS Backup</text>
      <href>https://docs.aws.amazon.com/aws-backup/</href>
      <external>true</external>
    </links>
    <links>
      <text>Amazon CloudWatch</text>
      <href>https://docs.aws.amazon.com/cloudwatch/</href>
      <external>true</external>
    </links>
    <links>
      <text>Elastic Load Balancing</text>
      <href>https://docs.aws.amazon.com/elasticloadbalancing/</href>
      <external>true</external>
    </links>
    <links>
      <text>Amazon GuardDuty</text>
      <href>https://docs.aws.amazon.com/guardduty/</href>
      <external>true</external>
    </links>
    <links>
      <text>EC2 Image Builder</text>
      <href>https://docs.aws.amazon.com/imagebuilder/</href>
      <external>true</external>
    </links>
    <links>
      <text>AWS Launch Wizard</text>
      <href>https://docs.aws.amazon.com/launchwizard/</href>
      <external>true</external>
    </links>
    <links>
      <text>AWS Systems Manager</text>
      <href>https://docs.aws.amazon.com/systems-manager/</href>
      <external>true</external>
    </links>
    <links>
      <text>Amazon Lightsail</text>
      <href>https://docs.aws.amazon.com/lightsail/</href>
      <external>true</external>
    </links>
    <links>
      <text>Amazon Lightsail or Amazon EC2</text>
      <href>https://docs.aws.amazon.com/decision-guides/latest/lightsail-or-ec2/lightsail-or-ec2.html</href>
      <external>true</external>
    </links>
    <maxLinksReached>false</maxLinksReached>
  </data>
</Root>
YAML Response
200 OK
status: ok
error: 
data:
  url: http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/concepts.html
  linkCount: 16
  links:
  - text: ''
    href: http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/concepts.html/pdfs/AWSEC2/latest/UserGuide/ec2-ug.pdf#concepts
    external: false
  - text: Documentation
    href: http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/concepts.html/index.html
    external: false
  - text: Amazon EC2
    href: http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/concepts.html/ec2/index.html
    external: false
  - text: User Guide
    href: concepts.html
    external: true
  - text: Amazon EC2 Instance Types Guide
    href: https://docs.aws.amazon.com/ec2/latest/instancetypes/instance-types.html
    external: true
  - text: PCI DSS Level 1
    href: https://aws.amazon.com/compliance/pci-dss-level-1-faqs/
    external: true
  - text: Amazon EC2 Auto Scaling
    href: https://docs.aws.amazon.com/autoscaling/
    external: true
  - text: AWS Backup
    href: https://docs.aws.amazon.com/aws-backup/
    external: true
  - text: Amazon CloudWatch
    href: https://docs.aws.amazon.com/cloudwatch/
    external: true
  - text: Elastic Load Balancing
    href: https://docs.aws.amazon.com/elasticloadbalancing/
    external: true
  - text: Amazon GuardDuty
    href: https://docs.aws.amazon.com/guardduty/
    external: true
  - text: EC2 Image Builder
    href: https://docs.aws.amazon.com/imagebuilder/
    external: true
  - text: AWS Launch Wizard
    href: https://docs.aws.amazon.com/launchwizard/
    external: true
  - text: AWS Systems Manager
    href: https://docs.aws.amazon.com/systems-manager/
    external: true
  - text: Amazon Lightsail
    href: https://docs.aws.amazon.com/lightsail/
    external: true
  - text: Amazon Lightsail or Amazon EC2
    href: https://docs.aws.amazon.com/decision-guides/latest/lightsail-or-ec2/lightsail-or-ec2.html
    external: true
  maxLinksReached: false

Response Structure

All API responses follow a consistent structure with the following fields:

FieldTypeDescriptionExample
statusstringIndicates whether the request was successful ("ok") or failed ("error")ok
errorstring | nullContains error message if status is "error", otherwise nullnull
dataobject | nullContains the API response data if successful, otherwise null{...}

Response Data Fields

When the request is successful, the data object contains the following fields:

FieldTypeSample Value
urlstring"http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/concepts.html"
linkCountnumber16
linksarray[{"text":"","href":"http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/concepts.html/pdfs/AWSEC2/latest/UserGuide/ec2-ug.pdf#concepts","external":false}, ...]
maxLinksReachedbooleanfalse

Headers

Required and optional headers for API requests:

Header NameRequiredExample ValueDescription
X-API-KeyYesyour_api_key_hereYour APIVerve API key. Found in your dashboard under API Keys.
AcceptNoapplication/jsonSpecify response format: application/json (default), application/xml, or application/yaml
User-AgentNoMyApp/1.0Identifies your application for analytics and debugging purposes
X-Request-IDNoreq_123456789Custom request identifier for tracking and debugging requests
Cache-ControlNono-cacheControl caching behavior for the request and response

Client Libraries

To get started with minimal code, most of our APIs are available through client libraries and clients:

Run in PostmanNPMPyPINuGetSwagger

Error Codes

The API uses standard HTTP status codes to indicate success or failure:

Status CodeMessageDescription
200OKRequest successful, data returned
400Bad RequestInvalid request parameters or malformed request
401UnauthorizedMissing or invalid API key
403ForbiddenAPI key does not have permission for this endpoint
429Too Many RequestsRate limit exceeded, please slow down requests
500Internal Server ErrorServer error, please try again later

What's Next?

Continue your journey with these recommended resources

Was this page helpful?

Help us improve our documentation