Request-based vs headless browsers: the economics of social automation

The standard way to automate a social platform is to drive a headless browser that loads the same pages a human would see. The alternative is to speak the platform’s request surface directly, plain HTTP with no browser anywhere in the stack. Between the two sits a gap of roughly 1000x in bandwidth per action, and that one ratio propagates into proxy cost, into reliability, and ultimately into what the product can charge.

Hermenea is built the second way, end to end. This post is about what each approach actually costs, and why the cheap one is the hard one.

What a headless browser actually costs

When a headless browser performs an action, it does not fetch the action. It fetches the page: the document, the stylesheets, the fonts, the JavaScript bundles, the images in the feed, the analytics and tracking scripts, the ad calls. Then it executes most of what it fetched, because the page does not work until it is rendered. A modern social feed typically weighs several megabytes before the first interaction happens. The one thing you came to do, posting a single reply, rides on top of all of it.

For a person on a home connection, that weight is invisible. For automation, it is the bill. Accounts operated at any serious scale sit behind residential proxies, and residential proxy bandwidth is typically metered by the gigabyte: every byte that moves through the proxy is paid for, in both directions, including the tracking scripts and the thumbnails nobody asked for. Multiply a page measured in megabytes by every login, every thread read, every reply, every account, every day, and proxy bandwidth becomes one of the heaviest recurring costs in the whole operation.

It is worth being honest about why nearly everyone builds on browsers anyway. The browser is the path of least resistance: record what a user does, then replay it. It sees exactly what a user sees, so anything a human can do through the interface can be automated without ever studying how the platform works underneath. And it generalizes. The harness that drives one site can be pointed at the next with modest changes. For a prototype, for an internal tool, for a product that wants to list ten platforms by next quarter, the browser is the rational choice. The cost arrives later, and it arrives as bandwidth. A careful operator blocks images, fonts, and analytics scripts via request interception, which does cut the bill; the floor stays high because the page still requires its documents, scripts, and API responses before any action can be performed.

The request-based alternative

Hermenea’s stack is HTTP-native. No Selenium, no Puppeteer, no browser process anywhere between an agent and the platform. Each action is the request that matters and very little else. As a rough order of magnitude, a comment is kilobytes where a rendered page is megabytes, because nothing is fetched only to be thrown away and nothing is rendered at all.

What the approach demands is the part that never shows up in a feature list. You have to understand the platform’s actual request surface: which endpoints do what, how sessions behave, what a well-formed request looks like. None of that is documented for you, and none of it is promised to stay still. Platforms rework their internals without notice, so a request-based integration is never finished. It is a surface you maintain for as long as you operate, and that permanent engineering work is the true price of the architecture.

What it gives up is generality. A browser harness reaches a new platform quickly precisely because it never needed to understand the previous one. A request-based stack earns each platform separately, with the same depth of work every time. There is no shortcut, which is much of the reason so few teams choose this road.

Where the 1000x lands

Follow the ratio through the stack. Bandwidth per action sets the proxy bill, because residential bandwidth is priced by volume. At browser weights, metered pricing is punishing, and operators respond by rationing actions or compromising on IP quality. At request weights, the meter barely moves: an agent’s entire month of activity consumes so little bandwidth that the proxy line all but disappears from the cost of running it.

The proxy bill sets the marginal cost of an agent. Marginal cost sets margin, and margin sets the shape of the pricing. This is the structural reason Hermenea’s volume pricing can reach its lower tiers at all: pricing is per agent per month, starting at 99 euros and stepping down across volume tiers, and the steps exist because the cost of one more agent stays small as a fleet grows. The full curve is on the pricing page. The lower tiers are not a promotion. They are the architecture, passed through.

Reliability compounds the same way. A browser fails in browser ways: a layout shifts and a selector stops matching, a page stalls mid-render, an instance leaks memory until something kills it. Retrying one costs kilobytes rather than another full page load, and fewer moving parts per action means fewer ways for an action to half-happen.

Trade-offs, honestly

None of this makes request-based the universal answer. It is harder to build, because a platform’s request surface has to be mapped before the first action can be performed, and that mapping is substantial work. It is costlier to maintain, because the platform owes you no stability and watching for drift is an operational discipline rather than a one-time task. And it is slower to extend, since every new platform is a new mapping project rather than a configuration change.

Browser stacks win the opposite cases. They are more general, faster to point at something new, and entirely sensible for breadth-first products, for jobs that run once, or for any workload where the bandwidth never accumulates into a number that matters.

Hermenea chose the request side because it is an infrastructure product. Agents run continuously for months, so per-action cost is the unit economics and per-action reliability is the product. When the same small actions repeat thousands of times across a fleet, cost and reliability compound while generality does not, and the expensive mapping work amortizes across every agent and every month it supports. That trade only makes sense for infrastructure, which is the point. It is the trade infrastructure should make.

The rest of the stack, orchestration, safety gates, tenant isolation, is covered in the engineering notes, and if you would rather watch it run than read about it, you can join the waitlist.