Get data from any page

We’ve made it to the final day of Launch Week, and we’re wrapping up with our biggest product offering yet: the AgentQL REST API!

This simple API turns any page into an endpoint for content. With only a URL and a query string, you can query web pages and retrieve the same precise, structured data you have come to expect from our SDKs and Playground—even from dynamic content or complex websites. The possibilities are endless, from scheduling scraping jobs to building data pipelines, or even empowering AI agents to interact with the web in all new ways.

Like the demo? Get the Zapier template and add your free API key!

What is the AgentQL REST API?

The AgentQL REST API allows you to send queries via HTTP to retrieve structured data from any publicly available website with a single request. This flexibility makes it a powerful tool for developers, automation experts, and AI agent providers working with dynamic data applications.

Use AgentQL with any programming language or tool that supports HTTP. By integrating with tools like Zapier or embedding it in backend systems, AgentQL’s REST API opens up new possibilities for automating large-scale data acquisition and training data collection. It’s an ideal solution for users who need the precision of AgentQL’s smart locators without the SDK setup—just plug in your query and get your data!

Unlock the Internet’s Data

There’s so much information available on the web that’s been dammed up in HTML soup. We get it. Not every organization can build an API to share their information. That’s why AgentQL was designed to do the heavy lifting for both publishers and consumers!

This REST API is for anyone looking to add fast, precise, structured data collection to their workflows. It’s handy for:

• AI Agents: The REST API is a perfect match for autonomous AI agents that need to pull in real-world data dynamically. With AgentQL’s REST endpoint, agents can retrieve and interact with live data, making them more adaptable and versatile.

• Web Scraping & Monitoring Applications: If your use case involves tracking product prices, following market trends, monitoring real estate listings, or scraping job boards, the REST API lets you automate it from one endpoint.

• Developers & Data Engineers: Integrate AgentQL into your backend or data processing pipelines for efficient web scraping, analytics, and content retrieval.

• Automation Experts & No-Code Users: Set up automated data flows in platforms like Zapier without needing extensive coding knowledge.

How to Get Started with the AgentQL REST API

1. Get your API key: Every request requires an X-API-Key header. Generate your free API key here—no credit card is required!

2. Define Your Request: Pass the URL you want to fetch data from and an AgentQL query.

{
  "url": "https://scrapeme.live/shop",
  "query": "{ products[] { product_name product_price } }"
}

1. Set your headers: Before making the API request, include the necessary headers for authentication and content type. These headers authorize the request and specify the data format being sent.

• X-API-Key: use your AgentQL API key for authentication.

• Content-Type: use application/json to indicate that the request body is in JSON format, allowing the server to interpret the data correctly.

1. Send the Request: Use your preferred HTTP client (like curl, Postman, or an HTTP library in Python or your preferred language) to make a POST request to the AgentQL REST API endpoint.

curl -X POST "https://api.agentql.com/v1/query-data" \
  -H "X-API-Key: $AGENTQL_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://scrapeme.live/?s=fish&post_type=product",
    "query": "{ products[] { product_name product_price } }"
  }'

(Make sure to replace $AGENTQL_API_KEY with your actual API key!)

1. Get Results: The API response will return data as structured JSON, ready to be used directly in applications, AI agents, or analytics workflows.

{
  "data": {
    "products": [
      {
        "product_name": "Qwilfish",
        "product_price": "£77.00"
      },
      {
        "product_name": "Huntail",
        "product_price": "£52.00"
      },
      ...
    ]
  },
  "metadata": {
    "request_id": "ecab9d2c-0212-4b70-a5bc-0c821fb30ae3"
  }
}

Use Cases for the REST API

The REST API unlocks a wide array of possibilities for developers and agent builders:

• Automated scraping activities: Add fetching data from any web page to your automation tasks, whether it's dynamic content or JavaScript-based content—AgentQL can handle it. Extract products, product details, and more.

• Large-scale data acquisition: For training data collection, nothing beats structured, real-time data, fresh from the Internet.

• Automated Data Pipelines: Feed up-to-date data into your backend processing or data analytics systems. This is perfect for e-commerce monitoring, news aggregation, or lead generation.

• Agentic AI Integrations: AI agents can now pull dynamic data on-demand, making decisions based on up-to-date information. Whether your agent monitors trends, analyzes competitor data, or interacts with external content, the REST API equips it with live data capabilities.

• Zapier Integrations for No-Code Automation: Use the REST API with Zapier to trigger data extraction workflows and automate repetitive tasks without writing code.

Ready to get real-time data from the web?

Get started making some AgentQL REST API calls today!

• API Reference docs

• Guide to scraping data from live web pages with the API

• Zapier template (just add your free API key!)

That’s a wrap for Launch Week!

This Launch Week we shared a new JavaScript SDK, Fast Mode, Stealth Mode, Query Generation from Natural Language and now this. We hope the REST API becomes an invaluable part of your toolkit, whether you’re a developer, automation expert, or building the next generation of agentic AI. We'd love to see what you build and are here to help on our community Discord.

Thank you for being part of AgentQL Launch Week! We're working hard to make sure AgentQL is your favorite tool for automation and data retrieval. Please let us know what you think—take our survey!

Happy coding and automating!

—The Tiny Fish Team Building AgentQL