Link Scraper

Token Usage:10 per call

Link Scraper is a simple tool for scraping web page links. It returns all the links on a web page.

This API provides reliable and fast access to link scraper data through a simple REST interface. Built for developers who need consistent, high-quality results with minimal setup time.

To use this API, you need an API key. You can get one by creating a free account and visiting your dashboard.

View API in Directory

Endpoint

POST Request

POST https://api.apiverve.com/v1/linkscraper

Code Examples

Here are examples of how to call this API in different programming languages:

cURL Request

curl -X POST \
  "https://api.apiverve.com/v1/linkscraper" \
  -H "X-API-Key: your_api_key_here" \
  -H "Content-Type: application/json" \
  -d '{
  "url": "https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/concepts.html",
  "maxlinks": 20,
  "includequery": false
}'

JavaScript (Fetch API)

const response = await fetch('https://api.apiverve.com/v1/linkscraper', {
  method: 'POST',
  headers: {
    'X-API-Key': 'your_api_key_here',
    'Content-Type': 'application/json'
  },
  body: JSON.stringify({
    "url": "https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/concepts.html",
    "maxlinks": 20,
    "includequery": false
})
});

const data = await response.json();
console.log(data);

Python (Requests)

import requests

headers = {
    'X-API-Key': 'your_api_key_here',
    'Content-Type': 'application/json'
}

payload = {
    "url": "https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/concepts.html",
    "maxlinks": 20,
    "includequery": false
}

response = requests.post('https://api.apiverve.com/v1/linkscraper', headers=headers, json=payload)

data = response.json()
print(data)

Node.js (Native HTTPS)

const https = require('https');
const url = require('url');

const options = {
  method: 'POST',
  headers: {
    'X-API-Key': 'your_api_key_here',
    'Content-Type': 'application/json'
  }
};

const postData = JSON.stringify({
  "url": "https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/concepts.html",
  "maxlinks": 20,
  "includequery": false
});

const req = https.request('https://api.apiverve.com/v1/linkscraper', options, (res) => {
  let data = '';
  res.on('data', (chunk) => data += chunk);
  res.on('end', () => console.log(JSON.parse(data)));
});

req.write(postData);
req.end();

PHP (cURL)

<?php

$ch = curl_init();

curl_setopt($ch, CURLOPT_URL, 'https://api.apiverve.com/v1/linkscraper');
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_CUSTOMREQUEST, 'POST');
curl_setopt($ch, CURLOPT_HTTPHEADER, [
    'X-API-Key: your_api_key_here',
    'Content-Type: application/json'
]);

curl_setopt($ch, CURLOPT_POSTFIELDS, json_encode({
    'url': 'https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/concepts.html',
    'maxlinks': 20,
    'includequery': false
}));

$response = curl_exec($ch);
curl_close($ch);

$data = json_decode($response, true);
print_r($data);

?>

Go (net/http)

package main

import (
    "fmt"
    "io"
    "net/http"
    "bytes"
    "encoding/json"
)

func main() {
    payload := map[string]interface{}{
        "url": "https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/concepts.html",
        "maxlinks": "20",
        "includequery": "false"
    }

    jsonPayload, _ := json.Marshal(payload)
    req, _ := http.NewRequest("POST", "https://api.apiverve.com/v1/linkscraper", bytes.NewBuffer(jsonPayload))

    req.Header.Set("X-API-Key", "your_api_key_here")
    req.Header.Set("Content-Type", "application/json")

    client := &http.Client{}
    resp, err := client.Do(req)
    if err != nil {
        panic(err)
    }
    defer resp.Body.Close()

    body, _ := io.ReadAll(resp.Body)
    fmt.Println(string(body))
}

Ruby (Net::HTTP)

require 'net/http'
require 'json'

uri = URI('https://api.apiverve.com/v1/linkscraper')
http = Net::HTTP.new(uri.host, uri.port)
http.use_ssl = true

payload = {
  "url": "https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/concepts.html",
  "maxlinks": 20,
  "includequery": false
}

request = Net::HTTP::Post.new(uri)
request['X-API-Key'] = 'your_api_key_here'
request['Content-Type'] = 'application/json'

request.body = payload.to_json

response = http.request(request)
puts JSON.pretty_generate(JSON.parse(response.body))

Authentication

This API requires authentication via API key. Include your API key in the request header:

Required Header

X-API-Key: your_api_key_here

GraphQL AccessBETA

Most APIVerve APIs support GraphQL queries, allowing you to combine multiple API calls into a single request and retrieve only the data you need. This powerful feature enables you to orchestrate complex data fetching with precise field selection.

This API does not currently support GraphQL as it requires a POST request. GraphQL support for POST APIs is coming soon.

Parameters

The following parameters are available for this API endpoint:

Parameter	Type	Required	Location	Description	Example
url	string	Yes	body	The URL of the web page to scrape links from	https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/concepts.html

Response

The API returns responses in JSON, XML, and YAML formats:

Example Responses

JSON Response

200 OK

{
  "status": "ok",
  "error": null,
  "data": {
    "url": "http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/concepts.html",
    "linkCount": 16,
    "links": [
      {
        "text": "",
        "href": "http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/concepts.html/pdfs/AWSEC2/latest/UserGuide/ec2-ug.pdf#concepts",
        "external": false
      },
      {
        "text": "Documentation",
        "href": "http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/concepts.html/index.html",
        "external": false
      },
      {
        "text": "Amazon EC2",
        "href": "http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/concepts.html/ec2/index.html",
        "external": false
      },
      {
        "text": "User Guide",
        "href": "concepts.html",
        "external": true
      },
      {
        "text": "Amazon EC2 Instance Types Guide",
        "href": "https://docs.aws.amazon.com/ec2/latest/instancetypes/instance-types.html",
        "external": true
      },
      {
        "text": "PCI DSS Level 1",
        "href": "https://aws.amazon.com/compliance/pci-dss-level-1-faqs/",
        "external": true
      },
      {
        "text": "Amazon EC2 Auto Scaling",
        "href": "https://docs.aws.amazon.com/autoscaling/",
        "external": true
      },
      {
        "text": "AWS Backup",
        "href": "https://docs.aws.amazon.com/aws-backup/",
        "external": true
      },
      {
        "text": "Amazon CloudWatch",
        "href": "https://docs.aws.amazon.com/cloudwatch/",
        "external": true
      },
      {
        "text": "Elastic Load Balancing",
        "href": "https://docs.aws.amazon.com/elasticloadbalancing/",
        "external": true
      },
      {
        "text": "Amazon GuardDuty",
        "href": "https://docs.aws.amazon.com/guardduty/",
        "external": true
      },
      {
        "text": "EC2 Image Builder",
        "href": "https://docs.aws.amazon.com/imagebuilder/",
        "external": true
      },
      {
        "text": "AWS Launch Wizard",
        "href": "https://docs.aws.amazon.com/launchwizard/",
        "external": true
      },
      {
        "text": "AWS Systems Manager",
        "href": "https://docs.aws.amazon.com/systems-manager/",
        "external": true
      },
      {
        "text": "Amazon Lightsail",
        "href": "https://docs.aws.amazon.com/lightsail/",
        "external": true
      },
      {
        "text": "Amazon Lightsail or Amazon EC2",
        "href": "https://docs.aws.amazon.com/decision-guides/latest/lightsail-or-ec2/lightsail-or-ec2.html",
        "external": true
      }
    ],
    "maxLinksReached": false
  }
}

XML Response

200 OK

<Root>
  <status>ok</status>
  <error />
  <data>
    <url>http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/concepts.html</url>
    <linkCount>16</linkCount>
    <links>
      <text></text>
      <href>http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/concepts.html/pdfs/AWSEC2/latest/UserGuide/ec2-ug.pdf#concepts</href>
      <external>false</external>
    </links>
    <links>
      <text>Documentation</text>
      <href>http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/concepts.html/index.html</href>
      <external>false</external>
    </links>
    <links>
      <text>Amazon EC2</text>
      <href>http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/concepts.html/ec2/index.html</href>
      <external>false</external>
    </links>
    <links>
      <text>User Guide</text>
      <href>concepts.html</href>
      <external>true</external>
    </links>
    <links>
      <text>Amazon EC2 Instance Types Guide</text>
      <href>https://docs.aws.amazon.com/ec2/latest/instancetypes/instance-types.html</href>
      <external>true</external>
    </links>
    <links>
      <text>PCI DSS Level 1</text>
      <href>https://aws.amazon.com/compliance/pci-dss-level-1-faqs/</href>
      <external>true</external>
    </links>
    <links>
      <text>Amazon EC2 Auto Scaling</text>
      <href>https://docs.aws.amazon.com/autoscaling/</href>
      <external>true</external>
    </links>
    <links>
      <text>AWS Backup</text>
      <href>https://docs.aws.amazon.com/aws-backup/</href>
      <external>true</external>
    </links>
    <links>
      <text>Amazon CloudWatch</text>
      <href>https://docs.aws.amazon.com/cloudwatch/</href>
      <external>true</external>
    </links>
    <links>
      <text>Elastic Load Balancing</text>
      <href>https://docs.aws.amazon.com/elasticloadbalancing/</href>
      <external>true</external>
    </links>
    <links>
      <text>Amazon GuardDuty</text>
      <href>https://docs.aws.amazon.com/guardduty/</href>
      <external>true</external>
    </links>
    <links>
      <text>EC2 Image Builder</text>
      <href>https://docs.aws.amazon.com/imagebuilder/</href>
      <external>true</external>
    </links>
    <links>
      <text>AWS Launch Wizard</text>
      <href>https://docs.aws.amazon.com/launchwizard/</href>
      <external>true</external>
    </links>
    <links>
      <text>AWS Systems Manager</text>
      <href>https://docs.aws.amazon.com/systems-manager/</href>
      <external>true</external>
    </links>
    <links>
      <text>Amazon Lightsail</text>
      <href>https://docs.aws.amazon.com/lightsail/</href>
      <external>true</external>
    </links>
    <links>
      <text>Amazon Lightsail or Amazon EC2</text>
      <href>https://docs.aws.amazon.com/decision-guides/latest/lightsail-or-ec2/lightsail-or-ec2.html</href>
      <external>true</external>
    </links>
    <maxLinksReached>false</maxLinksReached>
  </data>
</Root>

YAML Response

200 OK

status: ok
error: 
data:
  url: http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/concepts.html
  linkCount: 16
  links:
  - text: ''
    href: http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/concepts.html/pdfs/AWSEC2/latest/UserGuide/ec2-ug.pdf#concepts
    external: false
  - text: Documentation
    href: http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/concepts.html/index.html
    external: false
  - text: Amazon EC2
    href: http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/concepts.html/ec2/index.html
    external: false
  - text: User Guide
    href: concepts.html
    external: true
  - text: Amazon EC2 Instance Types Guide
    href: https://docs.aws.amazon.com/ec2/latest/instancetypes/instance-types.html
    external: true
  - text: PCI DSS Level 1
    href: https://aws.amazon.com/compliance/pci-dss-level-1-faqs/
    external: true
  - text: Amazon EC2 Auto Scaling
    href: https://docs.aws.amazon.com/autoscaling/
    external: true
  - text: AWS Backup
    href: https://docs.aws.amazon.com/aws-backup/
    external: true
  - text: Amazon CloudWatch
    href: https://docs.aws.amazon.com/cloudwatch/
    external: true
  - text: Elastic Load Balancing
    href: https://docs.aws.amazon.com/elasticloadbalancing/
    external: true
  - text: Amazon GuardDuty
    href: https://docs.aws.amazon.com/guardduty/
    external: true
  - text: EC2 Image Builder
    href: https://docs.aws.amazon.com/imagebuilder/
    external: true
  - text: AWS Launch Wizard
    href: https://docs.aws.amazon.com/launchwizard/
    external: true
  - text: AWS Systems Manager
    href: https://docs.aws.amazon.com/systems-manager/
    external: true
  - text: Amazon Lightsail
    href: https://docs.aws.amazon.com/lightsail/
    external: true
  - text: Amazon Lightsail or Amazon EC2
    href: https://docs.aws.amazon.com/decision-guides/latest/lightsail-or-ec2/lightsail-or-ec2.html
    external: true
  maxLinksReached: false

Live Test API in Playground

Response Structure

All API responses follow a consistent structure with the following fields:

Field	Type	Description	Example
status	string	Indicates whether the request was successful ("ok") or failed ("error")	ok
error	string \| null	Contains error message if status is "error", otherwise null	null
data	object \| null	Contains the API response data if successful, otherwise null	{...}

Response Data Fields

When the request is successful, the data object contains the following fields:

Field	Type	Sample Value
url	string	"http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/concepts.html"
linkCount	number	16
links	array	[{"text":"","href":"http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/concepts.html/pdfs/AWSEC2/latest/UserGuide/ec2-ug.pdf#concepts","external":false}, ...]
maxLinksReached	boolean	false

Headers

Required and optional headers for API requests:

Header Name	Required	Example Value	Description
X-API-Key	Yes	your_api_key_here	Your APIVerve API key. Found in your dashboard under API Keys.
Accept	No	application/json	Specify response format: application/json (default), application/xml, or application/yaml
User-Agent	No	MyApp/1.0	Identifies your application for analytics and debugging purposes
X-Request-ID	No	req_123456789	Custom request identifier for tracking and debugging requests
Cache-Control	No	no-cache	Control caching behavior for the request and response

Client Libraries

To get started with minimal code, most of our APIs are available through client libraries and clients:

Error Codes

The API uses standard HTTP status codes to indicate success or failure:

Status Code	Message	Description
200	OK	Request successful, data returned
400	Bad Request	Invalid request parameters or malformed request
401	Unauthorized	Missing or invalid API key
403	Forbidden	API key does not have permission for this endpoint
429	Too Many Requests	Rate limit exceeded, please slow down requests
500	Internal Server Error	Server error, please try again later

What's Next?

Continue your journey with these recommended resources

Browse All APIs

Explore our complete catalog of APIs

Authentication Guide

Learn about API keys and security

Rate Limits

Understand usage limits and best practices

Was this page helpful?

Help us improve our documentation