Puppeteer users struggle to navigate with POST requests and headers
Browser automation developers and web crawlers need to load pages using the same method, body, and headers as a captured request, but Puppeteer’s navigation API does not offer a straightforward way to set them on the initial request. The evidence describes workarounds such as form submissions and request interception, which add awkward setup. Browser rules can also prevent setting certain headers, such as Cookie, so a helper cannot solve every authentication case.
For browser automation developers and web crawlers. Mentioned from Oct 2017 to Sep 2018 on GitHub.
5 different people described this problem in 2 separate discussions.
- Indie fit
- 3.0/10
- Pain
- 6.3/10
- Frequency
- 6.5/10
- Willingness to pay
- 0.0/10
- Momentum
- 5.0/10
- Who pays
- Professionals
- Competition
- Medium
- Build difficulty
- Medium
What people said
Quoted word for word. Follow a link to read the whole discussion.
There really does need to be an easy straightforward way to do a POST request directly from puppeteer (without interacting with a form) -- I created a local mini-server with an auto-posting form that I can send requests to and it will redirect it to the proper destination with a POST request (and then I have to interce…
Both of these solutions are very hackish… It would be better to have a second parameter to page.goto() in which we can specify the request params like request.continue():
Build brief
See what to build and who will buy it
- 1 product idea with the smallest useful version and pricing
- 3 places to find your first customers
- 4 more quotes from people who have this problem
- Current workarounds, existing solutions and risks