INF 133 - User Interaction Software - Fall 2018


A2: Runkeeper Tweet Report in JavaScript and TypeScript

Due Saturday, October 27th, 4:00am

In this assignment, you'll use JavaScript and TypeScript to create a webpage report which helps a potential researcher understand a week’s worth of Tweets from RunKeeper, a popular running app. You’ll follow much of the methodology from one of Professor Epstein’s research papers, which looked to understand whether the content of RunKeeper tweets influences how or whether audience members respond.

Daniel A. Epstein, Bradley H. Jacobson, Elizabeth Bales, David W. McDonald, Sean A. Munson. From "Nobody Cares" to "Way to Go!": A Design Framework for Social Sharing in Personal Informatics. CSCW 2015.

Starter code

A starter repository is on GitHub Classroom.

Repository structure

The repository contains 14 files (excluding the .gitignore, which specifies what files should not be committed). You will edit half of them in this assignment.

The files you should not need to edit are:

  • tsconfig.json: a project configuration file which sets compiler options for TypeScript.
  • css/style.css: a CSS stylesheet for the assignment. Most of the layout is imported from Bootstrap.
  • index.html: the root HTML file, which js/about.js will edit the DOM of to display information about the Tweets in the dataset.
  • activities.html: another HTML file, which js/activities.js will edit the DOM of to display information about the activity types people posted to Twitter.
  • description.html: another HTML file, which js/description.js will edit the DOM of to display an interface for searching through the Tweets.
  • data/saved_tweets.json: a JSON object containing the same week’s worth of saved tweets. Your code won't use this file, but the format may be useful as you’re looking through the Tweets.
  • js/get_saved_tweets.js: a JavaScript file containing an asynchronous function which returns a week’s worth of saved tweets. This file will be used in place of parsing the data/saved_tweets.json file.

Note that this means you should not need to edit any HTML or CSS files. You may choose to, but you should only need to if you’re adding functionality beyond the assignment requirements.

The files you should edit are:

  • js/about.js: a JavaScript file which edits the DOM of index.html to display information about the Tweets in the dataset.
  • js/activities.js: a JavaScript file which edits the DOM of activities.html to display information about the activity types people posted to Twitter.
  • js/description.js: a JavaScript file which edits the DOM of description.html to display an interface for searching through the Tweets.
  • js/get_live_tweets.js: a JavaScript file containing an asynchronous function which loads recent Tweets using the Twitter API. Initially, the function returns undefined.
  • js/live_button_handler.js: a JavaScript file which should set the Click event handler for the button for loading live tweets. Initially, no Click event handler is specified.
  • ts/tweet.ts: a TypeScript file which creates a Tweet class used to parse different parts of the text of each Tweet.
  • readme.txt: a readme file where you describe what you made for the assignment.

Following the instructions listed in the Running the Twitter Proxy section, you should create a twitter_proxy_config.json file. Because this file contains secret information (your login information for Twitter), it should not be committed to the repository and is therefore listed in the .gitignore file.

One additional file, js/tweet.js, will be created by transpiled ts/tweet.ts into JavaScript. Because this file can be generated, it does not be committed to the repository and is therefore listed in the .gitignore file.

This assignment can be completed without writing any additional files or functions. But you may add a file or function if you want (such as to complete one of the bonus features).

Setting up your Workspace

You will need to install a TypeScript transpiler and a Twitter Proxy for this assignment. The TypeScript transpiler converts TypeScript files to equivalent JavaScript. Like many APIs, Twitter doesn’t support making API requests on a client browser, so the Twitter Proxy creates a server running on your local computer at port 7890. The next assignment will explore how to make a server, but it’s not the focus of this assignment. It's also recommended that you install a live reloader to simplify debugging.

The easiest way to install these two packages is through npm, the Package Manager in Node.js. Follow these instructions to install npm: https://www.npmjs.com/get-npm

Once npm is installed, open your favorite terminal and install the three packages for the assignment. Both are applications which run in the terminal rather than libraries, so it’s recommended that you install them with the -g (global) option.

Running the TypeScript Transpiler

The typescript transpiler, tsc, takes a project configuration file as an optional argument. The configuration file is in the root folder of the repository, so run tsc from there: tsc --p tsconfig.json. It’s also recommended that you add the --watch flag so your code automatically re-compiles. This code will create the JavaScript version of the Tweet class injs/tweet.js.

Running the Twitter Proxy

The Twitter developer account and Proxy is only required for the last of the four parts of the assignment (Loading Live Tweets). It's good to create a development account and app sooner, in case they take some time to get approved. But setting it up and making API requests is not required to complete the first 75% of the assignment.

First create a development account on Twitter.https://developer.twitter.com/en/apply/user. Once your development account is approved, you can make a development application. https://developer.twitter.com/en/apps. We’re only going to use the API to read tweets, so some of the required parameters aren’t applicable. Name the app whatever you’d like. For the Application Description and explanation of how the app will be used, enter:

This App is used to read tweets from a running app as part of a course assignment for IN4MATX 133, User Interface Software, at the University of California, Irvine. It will only be used to analyze Tweets, no Tweeting/Retweeting/Liking will be incorporated. The app will analyze tweets by parsing the text of each tweet for user-generated content and sentiment, like weather information or descriptions of how difficult the run was. The Tweet content will be displayed at an individual level (rather than aggregated), but it will not be displayed with Twitter user accounts and will not be visible anywhere on the web. The app will only be used internally for the course.

Put the course website as the Website URL http://inf133-fa18.depstein.net/. Do not enable sign-in or provide a Callback, Term of Service, or Privacy Policy URL.

Email the course staff if your application does not get approved or you run into other options creating a development application. We have an approved Twitter for Education account as a backup.

Once you have created the development app, find the Consumer API keys for the app under "Keys and Tokens". Create a file, twitter_proxy_config.json, with your consumer key and secret. It should be of the form:

{
 "consumerKey": "[Your Consumer Key]",
 "consumerSecret": "[Your Consumer Secret]"
} 

Once this file is created, you can run your twitter proxy in the terminal from the same directory as your config file: twitter-proxy twitter_proxy_config.json. This will enable you to access the Twitter API by swapping https://api.twitter.com/ in all API requests with the proxy, localhost:7890/. Make sure to keep the 1.1 in the localhost url.

Running Live Server

You may find the live-server package helpful, which automatically refreshes your webpage when you make changes to it. It’s not perfect, but can be a helpful resource. Run it in your terminal from the root directory with live-server.

Requirements

There are four parts to this assignment: summarizing tweets on the about page, identifying the most popular activities to the activities page, adding a text search interface to the description page, and loading live tweets. You’ll need to add features to the Tweet TypeScript class throughout. You can also optionally add any of a number of bonus features.

The much of this assignment involves using JQuery to modify the content of different classes or ids with computed values (for example, calculating the number of tweets of each type of activity in the dataset). The classes and ids which need editing appear with three question marks (???). With a few exceptions, the values should be computed dynamically. Meaning, your report would compute and display different results if next week’s tweets were loaded instead. The last part of the assignment will help test this by loading the most recent RunKeeper Tweets.

You will not receive credit for these modifications if you edit the HTML directly or hard-code a variable to the value that the span should be set to. There are a few spans where hard-coding in activities.js is expected/allowed.

Summarizing Tweets (about.js)

In this part, you will provide some summary information to a researcher interested in the Tweet data. You will identify the dates of the earliest and latest Tweets in the set, how many of each category exist, and how many of the completed tweets contain written text. Use JQuery methods to programmatically edit the spans for each question.

The time attribute of the Tweet class contains the date of each Tweet. Write out the Month, Day, and Year of the dates (e.g., Sunday, October 7, 2018). The JavaScript function toLocaleDateString() may be helpful for formatting the dates.

The tweets can be divided into four categories:

  • Completed events, where the person is Tweeting an activity they recently finished.
  • Live events, where the person is Tweeting that they are currently doing an activity.
  • Achievement, where the person is indicating an achievement they have reached or a goal they have set.
  • Miscellaneous, for all RunKeeper-related discussion which did not involve posting about an activity.

Some of the Tweets in each category also contain text written by the user.

Look through the text of the tweets in the dataset to devise rules for categorizing the tweets and identifying whether it contains user-written text programmatically. Write those rules into the source, written, and writtenText getter functions in tweet.ts. The JavaScript string startsWith, endsWith, and includes functions may help with this.

You may arrive at slightly different percentages depending on the exact rules you create, and we will not evaluate whether your rules pull exactly the same numbers that ours did. That said, about 95% of RunKeeper Tweets in the Epstein et al. research paper dataset that were of completed events, and about 25% of those completed events included any written text. Your numbers will deviate from the percentages in the research paper, but probably not by much (+/- 5%).

Format all percentages with exactly two decimal places. The header of about.html loads the Math.js library from a Content Delivery Network (CDN). You may find the .format() function in mathjs useful for formatting percentages.

Identifying the Most Popular Activities (activities.js)

In this part, you help a researcher understand the activities people are logging in RunKeeper by identifying the types of activities and distances in the completed tweets and visualizing how distance varies by activity type and day of the week. Use JQuery methods to programmatically edit the spans for each question and Vega-Lite to create graphs.

First, update the getters for activityType and distancein tweet.ts. Note that these values only have to be identified for completed tweets, which follow a fairly rigid format. Syntax for regular expressions is beyond the scope of this class, but may be helpful for calculating these values. indexOf or search should also be sufficient.

Because RunKeeper is used all around the world, some distances will be expressed in Miles and others in Kilometers. Convert them all to one unit. You might typically use a library to help with this (Math.js has some helper functions). However, our setup for TypeScript makes importing libraries challenging, so approximating is fine (e.g., 1 mi = 1.609 km). You’ll make more use of libraries in future assignments.

Again, the exact breakdown of activity types may vary slightly depending on the rules you create to parse. But there should be a major dropoff in Tweets between the three most common activity types, with thousands or many hundreds of Tweets each, and the other activity types.

Some of the activity types logged, such as Yoga, are better expressed with time durations rather than distances. Each of the three most frequently-logged activity types should be distance-based, so you do not need to parse the time duration activity types.

Once activityType and distance are calculated, they should be plotted with Vega-Lite. You will make three plots in total:

  • A plot of how many of each type of activity exists in the dataset. The specification of a potential visualization appears in activities.js.
  • A plot of the distances by day of the week for all of the three most tweeted-about activities. Day of the week should be encoded on the x-axis, distance on the y-axis, and activity type by color. There are a lot of points on this plot, so it’s hard to interpret which activity tended to be longest and on what day of the week.
  • A plot of the distances by day of the week for all of the three most tweeted-about activities, aggregating the activities by the mean.
Vega-Lite examples or the online editor may be helpful as you build the visualizations.

The two plots should alternate appearing when the button with the "aggregate" id is pressed. As shown in this video (the activityType legend is intentionally cut off):

Use the information shown in your graph to answer the questions about what activity people tended to do the longest and when. The answers to these questions can be hard-coded, since calculating them programmatically would require a couple of annoying maps and filters.

Adding a Text Search Interface (description.js)

In this part, you will implement a “search” interface for running tweets which allows a researcher to look through the tweets and their corresponding RunKeeper activities.

Update the table with the tweet number, activity type, and tweet content when the researcher types in the search box. The searchCount and searchText spans should also update as a researcher types in the search box. Your code should check the search box and update the table after every character is entered into the box. Your code should also clear the table when the search text is deleted.

The links in the Tweet should also be clickable to allow the researcher to explore the data more. To parse these, use similar strategies to how you parsed the type of activity and identified the tweet category. The getHTMLTableRow()function in the tweet TypeScript class is a good place to parse the tweets and generate clickable links.

Loading Live Tweets (get_live_tweets.js and live_button_handler.js)

In this part, you will use JavaScript’s fetch API to scrape the latest Tweets with the RunKeeper hashtag. You may use other APIs instead, so long as the function returns a Promise.

You should use Twitter's search API to get your tweets. Your code should get the maximum number of tweets for a single query (e.g., 100). Your code should get the most recent tweets rather than the most popular ones. You will need to encode the hashtag character (#). You might also consider specifying the Tweets to only be English (en). Implementing this feature will allow a researcher to browse the prior three parts with the 100 most recent #RunKeeper Tweets.

Note that the query will gather only the most recent 100 Tweets, which is substantially fewer than a week’s worth of Tweets. That is sufficient for demonstrating the proof-of-concept. Following Twitter’s pagination methods is a bonus feature for the assignment.

Optional/Bonus Features

The four parts of the assignment create a basic report tool. There are many ways of extending this tool, and we can offer a small amount of extra credit for implementing an extension or two. We’ve enumerated some potential extensions:

  • Following Twitter’s pagination scheme to load the maximum number of recent tweets (one week, unless you pay for one of the premium APIs). Use the URL specified in next_results.
  • Rather than alternating between two visualizations, using Vega-lite’s streaming data or interactive charts to dynamically change one chart when the aggregate button is pressed.
  • Mining sentiment from the text of each Tweet and presenting it in the table. One way of identifying sentiment is to use a list of positive and negative adjectives, such as the ones devised in these research projects.
  • Understanding and plotting when people do their longest time-based activities like Yoga.
  • Programmatically calculating the longest and shortest class of activity and whether it tends to occur on weekdays or weekends. This calculation can be influenced by outliers, so remove any Tweets with distances greater than three standard deviations away from the mean for that activity.

We are open to other suggestions for bonus features.

Submitting

Your code repository on GitHub Classroom is your submission. The service will take the latest snapshot of your repository as your submission. Any commits made after the deadline are considered late and are subject to the course's late policy.

Please update your readme.txt with how long the assignment took, who or what online resources you consulted with, any bonus features you added, and anything else we should know.

Grading

This assignment will be graded on a scale of 10 points, broken down as follows:

  • Summarizing tweets (2 points)
  • Identifying the most popular activities (3 points)
  • Adding a text search interface (3 points)
  • Loading live tweets (2 points)

Each bonus feature completed will earn you 1 point of extra credit. However, you can only earn a maximum of 2 points this way (e.g., no points given for beyond two bonus features). The maximum grade for this assignment is therefore 12/10.

In prior courses, you've been asked to follow good principles for indentation, naming variables, commenting, etc. We expect you to do the same in this course, but aim to avoid being draconian in our enforcement of these principles. Egregiously poor formatting, completely uncommented code, etc. may incur a small penalty (e.g., -1 point), but we expect this to be rarely applied.

As stated previously, we will not grade on whether the spans with values created in the report contain the exact same values as our calculations. We are just looking for them to be in the correct “ballpark” and will spend more energy evaluating the interactive portions of the report.