Using proxies in Puppeteer is clearly a topic that many developers need. I’ve put together this post with small code examples for most common cases – static proxies, per-page routing, authentication, and rotation.

  • Introduction
  • Static Proxy with --proxy-server
  • Page-Level Proxy / Request Interception
  • Handling Proxy Authentication
  • Proxy Rotation
  • Troubleshooting
  • Notes for Devs

Static Proxy with --proxy-server

Use a single proxy for all pages. This is simple and native.

import puppeteer from 'puppeteer'; const browser = await puppeteer.launch({ args: ['--proxy-server=http://HOST:PORT'] }); const page = await browser.newPage(); await page.goto('https://httpbin.org/ip'); console.log(await page.evaluate(() => document.body.innerText)); await browser.close(); 

This will show the IP of your proxy instead of your local IP.

Page-Level Proxy / Request Interception

Puppeteer doesn’t support per-page proxies natively. You can use puppeteer-page-proxy to route requests individually.

npm install puppeteer puppeteer-page-proxy 

Example:

import puppeteer from 'puppeteer'; import useProxy from 'puppeteer-page-proxy'; const browser = await puppeteer.launch(); const page = await browser.newPage(); await page.setRequestInterception(true); page.on('request', req => useProxy(req, 'http://user:password@host:port)); await page.goto('https://httpbin.org/ip'); await browser.close(); 

Each request goes through the proxy you specify.

Handling Proxy Authentication

This method of authentication won’t work:

const browser = await puppeteer.launch({ args: ['--proxy-server=http://host:port] }); 

Use the previous example with puppeteer-page-proxy, or for HTTP/HTTPS proxies that require login, use page.authenticate().

const browser = await puppeteer.launch({ args: ['--proxy-server=http://host:port] }); const page = await browser.newPage(); await page.authenticate({ username: 'USER', password: 'PASS' }); await page.goto('https://httpbin.org/ip'); console.log(await page.evaluate(() => document.body.innerText)); await browser.close(); 

Always authenticate before navigating to the page.

Proxy Rotation

Rotate proxies to avoid bans. Combine with random user-agents for better results.

const pool = [ 'http://user:password@host1:port', 'http://user:password@host2:port', 'http://user:password@host3:port' ]; let i = 0; function getNextProxy() { return pool[i++ % pool.length]; } const proxy = getNextProxy(); const browser = await puppeteer.launch({ args: [`--proxy-server=${proxy}`] }); const page = await browser.newPage(); await page.goto('https://httpbin.org/ip'); console.log(await page.evaluate(() => document.body.innerText)); await browser.close(); 

You can expand this with headers rotation or retry logic for blocked proxies. Here’s a link to a list of latest User Agents.

Troubleshooting

Most common errors:

  • 407 Proxy Authentication Required. Check username/password and call page.authenticate(). This can also happen if you used the wrong authentication method.
  • ERR_PROXY_CONNECTION_FAILED. Test host and port, try connecting with curl first.
  • 403/CAPTCHA. Rotate proxies, headers, or user-agent.

Notes for Devs:

Full Guide on How to Set Up Proxy in Puppeteer
More examples on Github
Join our Discord
If you want any examples I might have missed, leave a comment and I’ll add them.


Source: DEV Community.


Leave a Reply

Your email address will not be published. Required fields are marked *

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

The reCAPTCHA verification period has expired. Please reload the page.