Log in to a website

A simple guide to understand how to log in to any website with Phantombuster

As you know a lot of websites have their own identification system. Behind this identification, there is often highly valuable data for your business or personal use. It could also be data you generated with your personal account on the target site, and now you want to get it out to integrate it somewhere else.

Most of the scrapers out there don't handle this kind of stuff. But don't worry, with Phantombuster it's easy. Let's see!

Let's fill a log in form

First of all, how do you log in to an account on a website?
When you do it manually as a human, you click on each input and fill them one by one. That's exactly what you can do with Phantombuster and the fill() method:

await tab.fill("form", {
  email: "john@doe.com",
  password: "johnjohn"
}, {submit: true})

Whoa... so many things? I am going to explain all of this:

  • "form" is the selector of the form containing the inputs you want to fill. You will need to search for its selector in the website you are targeting (in this case we're just selecting the form tag but it's not going to work if the page contains multiple forms, you'll need to be more precise).
  • {email: "john@doe.com", password: "johnjohn"} is the data you want to enter. The keys (here email and password) must correspond to the inputs name attribute (in this case, the first input was <input name="email" />).
  • {submit: true} are the options you can add to fill(), for now only the submit option is available. When set, the form is immediately submitted.

Complete example

For this example, I am going to use our challenges site.

You'll need to log in with these credentials:
john@doe.com as email and johnjohn as the password.

  1. First create a new agent.
  2. Then write your script using fill(), waitUntilVisible() and open(). At the end you can take a screenshot() to see if it worked.
"phantombuster command: nodejs"
"phantombuster package: 4"
"phantombuster flags: save-folder"

// The part in the upper part is phantombuster configuration
// Don't bother understanding it all
// just that the flag "save-folder" save all files created
// on the phantombuster cloud (here the screenshot)

const Nick = require("nickjs")
const nick = new Nick()

nick.newTab().then(async (tab) => {
  await tab.open("http://scraping-challenges.phantombuster.com/login")
  await tab.waitUntilVisible("form")
  await tab.fill("form", {
    email: "john@doe.com",
    password: "johnjohn"
  }, { submit: true })
  await tab.waitUntilVisible(".panel-body")
  await tab.screenshot("screenshot.jpg")
})
.then(() => {
  nick.exit(0)
})
.catch((err) => {
  console.log(`An error occured: ${err}`)
  nick.exit(1)
})
"phantombuster command: nodejs"
"phantombuster package: 4"
"phantombuster flags: save-folder"

// The part in the upper part is phantombuster configuration
// Don't bother understanding it all
// just that the flag "save-folder" save all files created
// on the phantombuster cloud (here the screenshot)

const Nick = require("nickjs")
const nick = new Nick()

nick.newTab().then(tab => {
  return tab.open("http://scraping-challenges.phantombuster.com/login")
  .then(() => tab.waitUntilVisible("form"))
  .then(() => tab.screenshot("homepage.png"))
  .then(() => tab.fill("form", {
  	email: "john@doe.com",
	  password: "johnjohn"
  }, { submit: true }))
  .then(() => tab.waitUntilVisible(".panel-body"))
  .then(() => tab.screenshot("screenshot.jpg"))
})
.then(() => {
  nick.exit(0)
})
.catch((err) => {
  console.log(`An error occured: ${err}`)
  nick.exit(1)
})
"phantombuster command: nodejs"
"phantombuster package: 4"
"phantombuster flags: save-folder"

// The part in the upper part is phantombuster configuration
// Don't bother understanding it all
// just that the flag "save-folder" save all files created
// on the phantombuster cloud (here the screenshot)

const Nick = require("nickjs")
const nick = new Nick()

nick.newTab((err, tab) => {
  if (err)
    nick.exit(1)
  tab.open("http://scraping-challenges.phantombuster.com/login", (err) => {
    if (err)
      nick.exit(1)
    tab.waitUntilVisible("form", 5000, "and", (err) => {
      if (err)
        nick.exit(1)
      tab.fill("form", {
        email: "john@doe.com",
        password: "johnjohn"
      }, { submit: true }, (err) => {
        if (err)
          nick.exit(1)
        tab.waitUntilVisible(".panel-body", 5000, "and", (err) => {
          if (err)
            nick.exit(1)
          tab.screenshot("homepage.png", (err) => {
            if (err)
              nick.exit(1)
            nick.exit(0)
          })  
        })
      })
    })
  })
})

Watch your bot do its job

Launch it and enjoy your data!