Scrape Webpage

POST /scrape

python

Python

from notte_sdk import NotteClient
from pydantic import BaseModel


class CompanyInfo(BaseModel):
    name: str
    foundation_year: int
    location: str


notte = NotteClient()
company_info = notte.scrape(
    url="https://www.ycombinator.com/companies/nottelabs",
    instructions="Extract the company info from the webpage",
    response_format=CompanyInfo,
    only_main_content=False,
)

200

{
  "markdown": "<string>",
  "images": [
    {
      "url": "<string>",
      "category": "favicon",
      "description": "<string>"
    }
  ],
  "structured": {
    "success": true,
    "error": "<string>",
    "data": {}
  }
}

Authorizations

Authorization · string · header · required

The access token received from the authorization server in the OAuth 2.0 flow.

Headers

x-notte-request-origin · string | null

x-notte-sdk-version · string | null

Body

application/json

url · string · required

headless · boolean · default: true

Whether to run the session in headless mode.

solve_captchas · boolean · default: true

Whether to try to automatically solve captchas

max_duration_minutes · integer · default: 15

Maximum session lifetime in minutes (absolute maximum, not affected by activity).

Required range: x <= 1440

idle_timeout_minutes · integer · default: 3

Idle timeout in minutes. Session closes after this period of inactivity (resets on each operation).

Required range: x <= 30

proxies · (NotteProxy · object | ExternalProxy · object | TailnetProxy · object)[] boolean

List of custom proxies to use for the session. If True, the default proxies will be used.

NotteProxy

ExternalProxy

TailnetProxy

Show child attributes

browser_type · enum<string> · default: chromium

The browser type to use. Supported values are chromium and chrome. chrome-nightly and chrome-turbo are legacy aliases for chrome.

Available options: chromium, chrome, chrome-nightly, chrome-turbo

user_agent · string | null

The user agent to use for the session

chrome_args · string[] | null

Overwrite the chrome instance arguments

viewport_width · integer | null

The width of the viewport

viewport_height · integer | null

The height of the viewport

aspect_ratio · enum<string> | null

Viewport shape preset. When set, the backend fits the largest rectangle of this aspect ratio inside the sampled available screen area. Cannot be combined with explicit viewport_width/viewport_height.

Available options: 5:4, 16:9

cdp_url · string | null

The CDP URL of another remote session provider.

use_file_storage · boolean · default: true

Whether FileStorage should be attached to the session.

screenshot_type · enum<string> · default: last_action

The type of screenshot to use for the session.

Available options: raw, full, last_action

profile · SessionProfile · object | null

Browser profile configuration for state persistence

Show child attributes

web_bot_auth · boolean · default: false

Whether to use web bot authentication.

extra_http_headers · Extra Http Headers · object | null

Extra HTTP headers to be sent with every request.

Show child attributes

vault_id · string | null

The vault to use for the session

auth_ids · string[]

Managed Auth connection IDs to verify and, when necessary, authenticate inside this session before it is returned.

Maximum array length: 10

wait_for_authentication · boolean · default: true

Whether to wait for Managed Auth before returning the session. Defaults to true. When true, authentication failure or timeout fails session creation; when false, authentication continues in the background after the browser is ready.

selector · string | null

Playwright selector to scope the scrape to. Only content inside this selector will be scraped.

scrape_links · boolean · default: true

Whether to scrape links from the page. Links are scraped by default.

scrape_images · boolean · default: false

Whether to scrape images from the page. Images are scraped by default.

ignored_tags · string[] | null

HTML tags to ignore from the page

only_main_content · boolean · default: false

Whether to only scrape the main content of the page. If True, navbars, footers, etc. are excluded.

only_images · boolean · default: false

Whether to only scrape images from the page. If True, the page content is excluded.

response_format · any | null

The response format to use for the scrape. You can use a Pydantic model or a JSON Schema dict (cf. https://docs.pydantic.dev/latest/concepts/json_schema/#generating-json-schema.)

instructions · string | null

Additional instructions to use for the scrape. E.g. 'Extract only the title, date and content of the articles.'

use_link_placeholders · boolean · default: false

Whether to use link/image placeholders to reduce the number of tokens in the prompt and hallucinations. However this is an experimental feature and might not work as expected.

Response

200

application/json

Successful Response

markdown · string · required

Markdown representation of the extracted data

images · ImageData · object[] | null

List of images extracted from the page (ID and download link)

Show child attributes

structured · StructuredData[BaseModel] · object | null

Structured data extracted from the page in JSON format

Show child attributes