Data
The data we will be using in the workshop is to practice working with twitter data. Please download the data files
hashtag_gasprices.jsonl
anddehydratedCapitolRiotTweets.txt
and keep it in an accessible place (such as your desktop). We used twarc to search for all mentions of the hashtag#gasprices
and created this . The Capitol Riot tweets is a dataset from Kaggle.
- tweets containing #gasprices data download link
- Capitol Riot Tweets download link
- One Tweet download link
Google Drive may change the file extensions when you download these. Make sure you that hashtag_gasprices and one_tweet are
.jsonl
, NOT.jsn
. capitol_riots.txt really is.txt
.
You can force this by right clicking on the page link, choosesave as
, then choose “all file types” and remove the “.txt” ending and keep the “.jsonl” choose the appropriate filename extension.
Additional Utilities
We will also be using an extended set of utilities that will help explore the data collected through twarc.
twarc utilities download linkMake sure you download and unzip this folder onto your desktop. Later in the workshop, we will upload this set of tools to our JupyterHub.
Getting the Necessary Privileges
Generally, we would call this ‘getting your API key’, but like many API’s, the process of getting access to Twitter’s has become a complicated process. These instructions should create a new developer account with Essential Access privileges.
To get started, you will need:
-
A Twitter Account:
Valid email account and cell phone number to receive texts are necessary.
-
Developer Account:
You must create the Twitter account first, in order to link to the Developer account.
Getting developer access requires you to validate yourself as a human, so the form is particular about having a valid phone number and email address added to your Twitter account. This is so Twitter can authenticate a user.
Once you are logged on to Twitter, sign into
the Developer Portal
Fill out the application questions as listed here:
Essential Access
Make sure this page is titled ‘Just a few questions to get you Essential Access’.
We are not applying for elevated or academic researcher access.
- What country are you based in?
- What’s your use case? Choose
Teacher
- Will you make Twitter content or derived information available to a government entity or a government affiliated entity?
No
(recommended) - Click through the user agreement and verify your email.
Making your Project and App
Right after verifying your email, you will be taken to your Developer Portal Dashboard and prompted to create a new Project. Every instance of Twarc is a Twitter App and Twitter Apps live inside of Projects. After selecting ‘Exploring the API’ as your use case, you will be given “Essential Access”. Essential Access allows you to have one project with one App inside of it.
Fill out the four questions to create your first Twitter App. It will ask you for the project name, use case, and description. The project name is your App. App names must be unique across the entire platform, so if you get an error, choose a more unique name. These can be edited and amended later if you choose to change something in the App.
Acquiring Keys and Tokens
After you have completed setting up your project, you will see the App details below. You can edit the name and app icon if you’d like to personalize it.
Under the project name, you will see Settings and Keys and Tokens tab. You will need to navigate to Keys and Tokens to generate your 5 keys and tokens. You will need to click the generate button to get each of your keys and tokens. They are not automatically made for you so you must generate them the first time.
If you forget to save, or lose your tokens, you can click regenerate to regenerate your keys and tokens, and a new set is made for you. However, if you do regenerate any keys and tokens, you’ll need to reenter them when you try to access the API again.
Necessary Keys and Tokens
SAVE these keys in an accessible place like a password manager or in a document. You will not see these keys again, so this is essential. You should have:
- API Key
- API Secret
- Authentication Bearer Token
- Access Token
- Access Token Secret
You will have succeeded when you have these 5 keys and tokens
If you are having problems generating the 5 tokens please email us immediately or else you will not be able to follow along with the lessons