As you know a lot of websites have their own identification system. Behind this identification, there is often highly valuable data for your business or personal use. It could also be data you generated with your personal account on the target site, and now you want to get it out to integrate it somewhere else.
Most of the scrapers out there don't handle this kind of stuff. But don't worry, with Phantombuster it's easy. Let's see!
First of all, how do you log in to an account on a website?
When you do it manually as a human, you click on each input and fill them one by one, then you click to submit the form. That's exactly what you can do with Phantombuster and the following Puppeteer's methods:
await page.type("#email", "[email protected]") await page.type("#password", "johnjohn") await page.click("button[type=\"submit\"]")
Whoa... so many things? I am going to explain all of this.
- The first argument of both these methods is a CSS selector, you can find it using your developer console on the website you want to scrape. Here the login form looks like this:
<form method="post" action="/login/auth"> <div class="form-group"> <label for="email">Email</label> <input type="email" class="form-control" name="email" id="email"> </div> <div class="form-group"> <label for="password">Password</label> <input type="password" class="form-control" name="password" id="password"> </div> <button type="submit" class="btn btn-default">Connect</button> </form>
We can see that the form contains two fields (html
id attributes. The CSS selector for an id is
#<the_id>. In our case:
- We fill both these fields with our credentials.
- The form is ready to be submited, we just have to click the submit button! The button's selector is
button[type="submit"], meaning a
button tag having a type attribute of value "submit".
Updated 7 months ago