Market data collected every night for a hotel-intelligence company
A US hotel-intelligence company needed market data collected every night at a volume its team could not watch by hand. Python collectors with proxy rotation write into the client's MariaDB through stored procedures, and the engineers we placed watch them.
8M+requests a month against the sources the collectors watch
169properties reviewed inside one business day, May 2026
About 80%lower data-acquisition cost than the third-party feeds before it
200+hotel properties worked through by browser research agents
Nightly scheduleEvery collector on its own cadence
Proxy rotationResidential and mobile, with retries and anti-blocking
Python collectorsAPI-first, one per market source, with logging
Stored proceduresRows land through the client's own procedures
MariaDBThe client's database, in the client's account
MonitoringFailed runs surfaced before the morning
Placed engineersWatch the runs, fix the breakages
The system, drawn from the story.
Screens from this system are being added.
The situation
A US hotel-intelligence company ran its competitor-rate collectors on self-hosted servers and checked its data by hand. It needed Python scraping capacity that stays reachable against sites that block automated traffic, engineers it could trust with that work, and, from early 2026, an outbound engine and a marketing site it could own.
What we built
API-first Python collectors with residential and mobile proxy rotation, retry and anti-blocking strategy, writing results into the client's MariaDB through stored procedures
Scheduled collection and monitoring for a pipeline that has run every night since the 2025 contract, and the operational work of keeping it healthy
Engineers placed on the collection work after a documented screening task, a 24-hour reverse-engineering exercise with no Selenium allowed, and a paid trial, with a backup pool behind them
A same-day property review, May 2026, briefed and delivered inside one business day
Browser research agents that worked through the client's property list, resolving booking-platform identifiers, rebrand status and opening dates
A prospect list for the client's outbound, February to March 2026
8,869 contacts, cleaned address by address and filtered from tens of millions of records down to the US companies and people worth writing to, with a bulk classification pass over 9.44 million rows across nine files feeding it
A marketing-site rebuild as a Next.js front end over headless WordPress, deployed on Vercel in April 2026
A technical discovery and security audit of the client's legacy Laravel portal, June to August 2026, with a takeover tracker, a controller security review and a deployment-validation runbook gated on a tested restore
A documented hiring pipeline for the client's backend support
An NDA, a screening phase and a paid-trial phase, with recorded interviews
What changed
The client has a collection pipeline that stays up against hostile sites at a fraction of what the third-party feeds it replaced were costing, a bench of engineers it did not have to find itself, and a web estate it can take over on its own terms.
Along the way
December 2024
Staff augmentation scoped for the client's collection work: the screening process written and the first two candidates presented.
February 2025
Trial kick-offs for the placed engineers.
July 2025
Fixed-price Python scraping contract begins, structured as fortnightly milestones, and converts to a retainer in September.
February to March 2026
Prospect list built for the client's outbound, with the first outreach sent on 20 March.
April 2026
Marketing site built as a Next.js front end over headless WordPress and deployed on Vercel, the phased implementation opened as a single pull request on 18 April.
28 May 2026
A property review briefed and delivered within the same business day.
June to August 2026
Technical discovery and security audit of the legacy portal delivered on 14 June; the deployment-validation and takeover runbook followed on 12 July, with the backend-support hiring pipeline running alongside into August.
Read next
A list of addressesCSV in, with a checkpoint per batch
DNS and catch-all checksCached per domain
SMTP probingStaged, with provider profiles and tarpit handling
Our own list-cleaning engine: DNS and multi-stage SMTP checks on every address, and one of four verdicts with the reason behind it instead of a single opaque score. 34 reason codes, so every verdict points at the check that produced it.
Lists cleaned before a single send, mailboxes warmed on their own domains, sequences written per persona, and a deliverability read every week. Run on our own pipeline first, then for a partner under their brand. About 225K unique recipients a month, four touches each, July to October 2025.
A scheduled Python pipeline that reads Costco product pages, stores them in MongoDB and creates and updates the matching Shopify products, prices and inventory by SKU.