The short answer
Our custom web scraping starts at $100. That is the floor for a simple, well-defined job: one or a few public sources, a clear list of fields and a single export. The exact price depends on the factors below, and we give it after a short brief, once we have looked at the actual sources.
If someone quotes a fixed price without asking which sites, which fields and how often, the price will usually move later, or the scraper will be built in a way that breaks at the first change on the source.
What actually drives the price
Most of the cost sits in a handful of factors. The table shows each one, why it matters and what you can do about it.
| Factor | Why it changes the cost | How to keep it down |
|---|---|---|
| Number of sources | Every site has its own structure, quirks and failure modes. Ten sources are closer to ten small projects than to one. | Start with the two or three sources that matter most and add the rest once the first ones prove useful. |
| How the data is served | A plain HTML page is the cheapest case. Pages that load data with JavaScript, need a login or show different content by region take more work. | Check whether the site offers an official API, a feed or a downloadable file first. |
| Protection against automated access | Rate limits and anti-bot checks call for slower collection, retries and sometimes a headless browser, which costs more to build and to run. | Collect less often or only the pages that change, instead of the whole site every time. |
| Volume | Hundreds of pages and millions of pages need different infrastructure: queues, storage and monitoring. | Narrow the scope to the categories, regions or products you actually use. |
| Frequency | A one-off export is a different job from collection every hour with alerts when something breaks. | Match the schedule to how often you act on the data, not to how often it could change. |
| Cleaning and matching | Raw data is rarely usable as is. Normalizing units and prices, removing duplicates and matching the same product across sources is often the largest part of the work. | Agree on the exact fields and rules up front, and share an example of the result you expect. |
| Delivery | A CSV file is simple. Writing to your database, Google Sheets or an API, or building a dashboard, adds work. | Start with a file or a sheet; connect other systems when the data has proven its value. |
| Upkeep | Websites change their layout. A scraper that ran fine last month can stop tomorrow. | Plan for maintenance from the start rather than paying for emergency fixes. |
API, pages or files: three ways to get the same data
Before writing a scraper, we check how the data can be obtained at all. The answer often changes both the price and the reliability of the result.
- Marketplaces usually provide APIs for managing your own store or listings, not for reading other sellers' data. Competitor prices therefore usually come from public pages.
- Where an official API exists and covers the task, we use it.
| Method | When it fits | Trade-offs |
|---|---|---|
| Official API | The source offers an API that covers the data you need, and you can get access. | Stable and usually the cleanest option. Access rules, request limits and the set of available fields are decided by the source. |
| Feeds and files | The source publishes a price list, an XML or CSV feed or regular exports. | Cheap to process and rarely breaks. Updates only as often as the source publishes. |
| Reading web pages | The data is only available on the pages themselves. | Works for almost any public page, but needs more care: page changes, rate limits and protection against bots. |
One-off export or scheduled collection
A one-off export answers a question once: what does the market look like today, which products a supplier carries, what a catalog contains. It is the cheapest format, and a good way to check whether the data is useful before investing in automation.
Scheduled collection is a small system. It runs on its own, retries failed requests, keeps a history of changes and tells you when a source stops responding or changes its layout. It costs more to build, and it needs upkeep, but it removes manual work for good.
A common path is to start with a one-off export, look at the data together and then decide which sources and fields are worth collecting on a schedule.
The cost after launch
A scheduled scraper has running costs that a one-off export does not: a server, storage, sometimes proxies or a headless browser, and fixes when a source changes.
When a source changes, our scrapers alert you to the failure. The fix is covered by a monthly support plan, from $350 a month, or billed hourly, from $35 an hour. For a single stable source, hourly fixes are often enough; for many sources that change often, a support plan is usually more predictable.
Ask any provider how changes on the source are handled and paid for. That answer tells you more about the real cost than the build price does.
How to make a quote cheaper and more accurate
The more precise the brief, the less a quote has to cover the unknown. Before you ask for a price, prepare the following:
- The list of sources, with two or three example pages for each.
- The exact fields you need, ideally as a sample spreadsheet with column names.
- How often the data has to be updated, and what you do with it after each update.
- The approximate volume: how many products, pages or records.
- Where the result should go: a file, Google Sheets, your database or another system.
- What should happen when a source is unavailable or a field is missing.
What about legality?
Collecting publicly available data is generally permissible, but it depends on the specific site's terms of use and the type of data. We don't collect personal data without a legal basis, and we use an official API wherever one exists. For sensitive cases we recommend a separate legal consultation.