Manual HTML Paste for Bot-Blocked URLs

The problem: bot-blocked pages

Some pages block automated HTTP requests. Cloudflare, custom WAF rules, anti-scraping middleware, and other bot-protection systems detect and reject requests from non-browser user agents.

When PageScorch tries to fetch a page that is bot-protected, the roast fails with a BOT_BLOCK_DETECTED result instead of a conversion score.

The manual HTML paste fallback solves this without needing a screenshot.

How the fallback works

The URL roast fails with BOT_BLOCK_DETECTED
You open the page normally in your own browser (the page loads fine for real browsers)
You copy the page's HTML source
You paste it into the fallback field on the failed roast result page
PageScorch processes the pasted HTML through the same analysis pipeline as a normal URL roast

The result is a completed roast with a full conversion score, problem list, and fix suggestions — identical to what a normal URL roast produces.

Step-by-step instructions

Step 1: Identify the failed roast

Open the failed roast from your roast history or batch detail page. The status will show Failed and the reason will be BOT_BLOCK_DETECTED or URL_BOT_PROTECTED.

Step 2: Open the page in your browser

Navigate to the URL in Chrome, Firefox, or Safari. The page loads normally for you because you are a real browser, not an automated bot.

Step 3: Copy the page source

Use your browser's View Source feature:

Chrome / Edge: Ctrl+U (Windows/Linux) or Cmd+U (Mac), or right-click → View Page Source
Firefox: Ctrl+U (Windows/Linux) or Cmd+U (Mac)
Safari: Cmd+Option+U

Select all the source code (Ctrl+A / Cmd+A) and copy it (Ctrl+C / Cmd+C).

Step 4: Paste into the fallback field

Return to the failed roast result page in PageScorch. Click the Paste page source button. Paste the copied HTML into the text area and submit.

Step 5: Wait for the result

PageScorch processes the pasted HTML and completes the roast. This usually takes under 30 seconds. The roast result replaces the failed status and is accessible from your history and any linked batch.

What gets analyzed from pasted HTML

PageScorch sanitizes pasted HTML the same way it handles fetched pages:

Scripts, styles, and non-visible markup are stripped
Visible text content is extracted
The AI analyzes the copy and messaging structure

The result is a URL roast result — it analyzes copy and content, not visual layout. If you also want a visual analysis, run a screenshot roast separately.

Using the fallback in bulk batches

When a batch item fails with BOT_BLOCK_DETECTED:

Open the batch detail page
Click the failed item to open its result page
Use the Paste page source button on that result page
Submit the pasted HTML

When the pasted-HTML roast completes, the batch item is marked completed and the batch progress updates accordingly. If the batch was failed (all items failed), it may transition to partially_failed or completed once the retried items finish.

Notes and limitations

The paste fallback is available to signed-in users on all plans. Guest users cannot use it.
Only failed roasts with BOT_BLOCK_DETECTED or URL_BOT_PROTECTED status are eligible. You cannot use the paste fallback on a completed roast to replace its result.
Pasted HTML is processed and discarded. PageScorch does not store the raw HTML after analysis.
The size limit for pasted HTML is generous but finite — extremely large pages (e.g. a full SPA bundle) may exceed the limit. If that happens, try pasting just the <body> section of the source.

When to use a screenshot instead

If the pasted HTML fallback is inconvenient (too many pages, HTML too large, SPA that doesn't have meaningful source), a screenshot roast is often faster:

Open the page in your browser
Take a full-page screenshot
Upload it as a new screenshot roast

The screenshot roast captures the visual experience and does not require copying source code.

See Screenshot roasts in depth for details.