Twitter limits the amount of followers you can retrieve over a period of time. It's called the rate limit. In this case after scraping 5000 followers you'll need to wait for about 90min before being able to scrape again.
Re-launching the API during this period simply won't work. Once the rate limit has been lift off, you'll be able to scape again. Go to the Repetition Setup step to program re-launches.
⏳ Execution speed
This API will extract 5000 profiles in about 3 minutes. Nevertheless Twitter won't allow you to extract more than about 5000 profiles every 90min.
How to start extracting Followers step-by-step.
0. Necessary information
In order to get started, you'll need to fill out 2 mandatory information:
"Spreadsheet URL": In order to know who to extract from, Phantombuster needs an input. This input can either be a single Twitter handle such as @phbuster or a spreadsheet with one account URL per row. We recommend using Google Spreadsheet to build this input file. Click on Share the URL to make it publicly accessible to Phantombuster.
"Session Cookie auth_token": This is your Twitter auth_token session cookie. You'll have more details in Step 5 of this tutorial.
1. Create an account on Phantombuster.com 💻
If you haven't already, create a FREE account on Phantombuster. Our service will browse the web for you. It’s a website automator which runs in the cloud. Once done we'll follow up.
2. Use this API on your account.👌
Now that you're connected to Phantombuster, Click on the following button (it will open a new tab).
3. Click on Configure me!
You'll now see the 3 configuration dots blinking. Click on them.
4. Aim a specific Twitter account or several at a time
In the 1st argument field "Twitter account or Spreadsheet URL" you have the choice to use this API for 1 twitter account at a time or several in one go.
One at a time: Just fill the form with a Twitter username, example: @phbuster
Several at a time: Paste the URL of a spreadsheet filled with Twitter username and/or Twitter account URLs on column A
5. Get your Twitter Session cookie
The Session Cookie you'll need to make this API work is called "auth_token",
Here's how you can get yours:
Using Chrome, go to your Twitter homepage and open the inspector
→ Right click anywhere on the page and select “Inspect”
→ CMD+OPT+i on macOS
→ F12 or CTRL+MAJ+i on Windows
Locate the “Application” tab
Select “Cookies” > “https://www.twitter.com” on the left menu.
Locate the “auth_token” cookie.
Copy what’s under “Value” (Double click on it then Ctrl+C) and paste it into your script Argument)
Once your API is ready, you can set up repetitive launches. This allows scraping to spread over days, weeks or even months. You can also specify the number of profiles to process per launch, or leave that field blank to process every profile from your list.
To do so, simply hit the “Settings” button to define when your API is launched:
Then, select a frequency:
Don't forget to click 💾 Save at the bottom of the page!
For example, 10 profiles processed per launch, 8 launches per day: you'll process a total of 80 profiles per day.
Click on Launch & Enjoy!
It’s done! All that is left to do is to click on "launch" to try your script!
This API will output CSV and/or JSON containing the following fields:
profileUrl (Profile url of the person following the targeted account)
name (Name of the person)
bio (Bio of the person)
userId (Their User ID)
imgUrl (Their profile picture)
backgroundImg (Their background profile image)
certified (If they are a verified Twitter profile)
query (Url of the profile targeted)
Share this API
Your friends & colleagues need to know about this!