Web Scraping Statistics 2026: Usage and Crawl Benchmarks

Direct Answer

Apify reported that API-based Actor runs accounted for 87% of its total Actor runs in the platform period covered by its 2025 report. Monthly API calls increased from 3.6 billion in January 2023 to 6.8 billion in October 2024, while scheduled Actor runs increased from 1,256,973 to 3,221,685. In the report’s separate developer survey, product pricing was the most-selected scraped target at 34.8%.

Those figures are not market-size estimates. The platform counts measure activity inside Apify, and the public report does not disclose the survey respondent count. This page therefore keeps vendor telemetry, survey responses, current billing rules and our small controlled crawl benchmark in separate evidence groups.

Web-scraping benchmark	Figure	Scope
API-based share of Apify Actor runs	87%	Apify platform telemetry
Monthly API calls	3.6B → 6.8B	Jan 2023 to Oct 2024
Monthly scheduled Actor runs	1.26M → 3.22M	Same period
Survey respondents building more than 20 scrapers	21.7%	Developer survey; N not disclosed
Product pricing as a scraped target	34.8%	Same survey
Survey respondents using proxy providers	39.1%	Same survey; multi-select
Controlled crawl attempts	4	Two providers, two public docs targets

Downloads: CSV · JSON · JSONL

Platform Activity

API-based runs accounted for 87%

Apify says API-based Actor runs represented 87% of total Actor runs in the platform period covered by its report. This describes execution inside one vendor platform; it does not measure all web scraping or the number of organizations using it. Source

Monthly API calls rose from 3.6 billion to 6.8 billion

Apify reported 3.6 billion API calls in January 2023 and 6.8 billion in October 2024, an 88.89% increase. An API call is not the same as a unique job, customer, record or page. Source

Scheduled runs rose from 1.26 million to 3.22 million

Monthly scheduled Actor runs increased from 1,256,973 to 3,221,685 over the same endpoints. Apify reports 156.2% growth; direct calculation from those displayed endpoints rounds to 156.3%. Repeated runs from one workflow are included in platform activity. Source

Together, these three figures show increasing activity on Apify. They do not establish a global web-scraping market size or adoption rate, so the old market forecasts have been removed from this page.

Developer Survey: Activity and Targets

Apify says its report combines 2023–2024 platform data, a developer survey and external industry research. It acknowledges that developers in its reach may not represent the wider developer community. The public article does not state the survey sample size, so every percentage below needs that caveat.

21.7% built more than 20 scrapers

In the survey, 21.7% of respondents said they built more than 20 scrapers in the previous year. Another 13% reported 11–20, while 21.7% reported 6–10. Source

These bands describe reported output among the respondents. They should not be cited as a rate for all developers.

Product pricing led the listed targets

Survey respondents named product pricing at 34.8%, social-media content at 26.1%, news articles at 21.7% and customer reviews at 13% among scraped targets. Source

The useful operational distinction is target type: frequently changing prices require different crawl frequency, quality checks and storage rules from slower-changing news or review archives.

BeautifulSoup and Crawlee led the reported tool mix

Among open-source tools, respondents named BeautifulSoup at 43.5%, Crawlee at 34.8%, and Selenium and Playwright at 26.1% each. Tool use can overlap. Source

This is not a performance ranking. A parser, browser-automation library and crawling framework solve different parts of a pipeline.

Proxy providers were the most-selected third-party service

Respondents reported using proxy providers at 39.1%, web-scraping APIs at 34.8% and cloud platforms at 26.1%. The categories were not mutually exclusive. Source

Legal perceptions were mixed

In the survey, 17.4% viewed scraping as legal and unrestricted, 43.5% viewed it as legal with restrictions and 21.7% were unsure. These are respondent perceptions, not legal findings. Source

Whether a collection project is permissible depends on the jurisdiction, data, access method, contract and intended use. This page is not legal advice.

Billing Units Are Not Directly Comparable

Official pricing pages use different units, which makes “cost per million” comparisons unreliable unless a workload is replayed.

Apify jobs can combine several cost components

Apify says Store Actors can use pay-per-event or pay-per-usage pricing. A job can also consume platform compute, proxy, storage and transfer resources. Source

ScraperAPI credits can vary by target

ScraperAPI says a standard page request can cost one credit, while targets such as Amazon and Google can consume different amounts. Source

Bright Data uses successfully delivered records

Bright Data says its Web Scraper API charges for successfully delivered records and does not charge for failed deliveries. Source

Compute units, credits and delivered records are different denominators. Use the web-scraping pricing comparison for current plan facts, then run the same bounded workload before comparing vendors.

Controlled Crawl Observations

The companion benchmark ran four capped attempts across two public documentation hosts. Each provider pair used the same starting URL and a five-page cap.

Returned rows and unique URLs can differ

The Tugelbay Website Content Crawler returned five rows and five unique pages from the Apify documentation target in 7.523 seconds. On the Firecrawl documentation target, it returned five rows but four unique pages in 8.398 seconds because one canonical URL was duplicated. Controlled dataset

A production check should count unique canonical URLs, not only output rows.

Empty output needs a failed state

The two local Firecrawl-route attempts returned no pages before their 90-second timeouts. One coincided with a locally degraded host state. These single local runs do not support a provider-wide reliability conclusion. Controlled dataset

The defensible lesson is about instrumentation: record status, elapsed time, attempted target, returned rows, unique URLs and error reason. An empty dataset should not silently pass as a successful crawl.

What to Measure in Your Own Scraping Pipeline

• successful jobs and failed jobs by target;
• pages attempted, rows returned and unique canonical URLs;
• records accepted after validation;
• retry multiplier and timeout rate;
• content completeness and required-field coverage;
• target-change breakages and maintenance time;
• cost per accepted record, including paid tools and review;
• data retention, access restrictions and deletion rules.

This produces a useful internal benchmark without pretending that one vendor’s API calls, another vendor’s credits and a third vendor’s records are equivalent.

Methodology and Limitations

• Apify platform data: internal metrics from 2023 through October 2024, published January 30, 2025.
• Apify developer survey: promoted within and beyond Apify’s reach; the public article does not disclose N. Apify warns that its reachable audience may not represent the broader developer community.
• Pricing rules: direct captures of official Apify, ScraperAPI and Bright Data pages on August 20, 2026. Prices themselves remain in the dedicated pricing article.
• Controlled crawl: four single runs on July 10, 2026, two public documentation targets and a five-page cap.
• Exclusions: secondary market forecasts, bot-traffic figures from older reports, unsupported company-adoption percentages, hourly cost ranges and legal-risk percentages were removed.
• No cross-source averaging: platform calls, scheduled runs, survey percentages, credits, records and crawl rows remain separate measures.

Last verified: August 20, 2026.

Cite This Research

Tugelbay Konabayev. “Web Scraping Statistics 2026: Usage and Crawl Benchmarks.” Konabayev.com. Updated August 20, 2026. https://konabayev.com/blog/web-scraping-statistics-2026/

Frequently Asked Questions

How common is API-based scraping on Apify?

Apify reported that API-based runs accounted for 87% of total Actor runs in the platform period covered by its report. This is Apify telemetry, not a global market share.

What data do developers scrape?

In Apify’s survey, product pricing led at 34.8%, followed by social-media content at 26.1%, news articles at 21.7% and customer reviews at 13%. The sample size was not disclosed in the public article.

Can scraping-API credits be compared directly?

No. Vendors meter compute, credits, events, requests and successfully delivered records differently. Compare a defined workload and calculate cost per accepted output.

Is web scraping legal?

There is no universal answer. The applicable rules depend on jurisdiction, data, access method, contracts and use. Survey opinions do not determine legality; consult qualified counsel for sensitive work.

Related: web-scraping pricing, website crawler benchmark, and SEO automation statistics.