Final project

Data selection

Due: Oct 15 before class.

For the final project, you will work with a dataset of your interest. Please decide on a dataset. If you need help, let me know.

  • A few useful sources: Data.gov, Kaggle Datasets, UCI Machine Learning Repository, and Responsible Datasets in Context.
  • You cannot use a dataset that is part of an R package.
  • You cannot use a dataset that we have used in class or in homework.
  • You can use a dataset that you have used for a different project in a different class.
  • The dataset must contain at least 5 variables and 100 observations.
  • The dataset must contain at least one numeric variable (like age) and at least one categorical variable (like gender).

Once you have decided on a dataset, import it to R and pick ONE variable from the data. For the ONE variable, write a thorough description of its univariate distribution. Submit your code, code output, your description, and a link to your dataset on Moodle by Oct 15, 2:40 pm.

Content and Grading Guidelines

Presentation + Defense (20pt) + Participation (10pt)

The presentation serves as an intermediate step to discuss initial findings and gives you a chance to ask questions. Your presentation should be about 10 minutes, with 5-10 additional minutes of asking and answering questions. It should include:

  • Introduction: introduce the topic of your project, data, data source, or anything you find interesting
  • Descriptive Statistics (univariate): identify two variables and describe their univariate distribution; include tables and graphs as needed in your slide
  • Descriptive Statistics (multivariate): discuss the relationship between a few variables; include tables and graphs as needed in your slide
  • p-value: obtain one p-value from an appropriate test and discuss your findings
  • Future work: discuss what your next steps are; ask and answer questions

For the “Defense” part, you should be prepared to answer questions about any part of your presentation (e.g., you should not include code or graph that you don’t fully understand). For the “Participation” grade, you are expected to participate in others’ presentations by asking questions and providing feedback.

Written report (70pt)

Your report should include the following sections. You can include relevant R code and output at the end of each section, or you can insert R code in text wherever appropriate. Your report should be free of misspellings, incomplete sentences, and grammatical mistakes. You should write the report in languages that a general audience can understand.

  1. (5pt) Writing function. As you analyze your data, identify a task that is done repeatedly (e.g., are you generating a specific plot more than once? Are you obtaining information in the same way for more than one variable?), or a task that can be applied to other dataset. Write a function for this task. Clearly state the input(s) and output(s) of this function, and use it at least once in your project. You might want to circle back to this question later, but the function you write should appear here (i.e., code for this function should be run before the rest of the project).
  1. (5pt) Introduction. Provide some background information of the dataset (at least 5 sentences). What is the dataset about? What is the source? How many variables are in the dataset? How many observations? What are some of the variables? What makes you interested in this dataset? What type of research questions could potentially be answered by this dataset?
  1. (12pt) Descriptive Statistics (univariate). Pick three variables you find interesting. You should pick at least one numeric variable and at least one categorical variable. For each variable, write 2-3 sentences to describe its distribution. (For instance, for the numeric variable, describe the distribution in terms of location, spread, and shape; for the categorical variable, describe the categories and percentage of values in each category.) You should include at least one base R plot (e.g., histogram, boxplot, etc.) to support your descriptions in this section.
  1. (6pt) Transformation. Pick a numeric variable and standardize it by calculating its z-score. Store the z-score and pick at least two z-score values that are interesting (e.g., very large or very small z-score values). For each value you pick, write an interpretation of the value.
  1. (15pt) Information Extraction (multivariate). Investigate the dataset in detail and discuss three interesting findings using functions in the tidyverse package. You can refer to Homework 5 for examples (an example of a finding would be: ``Platform, Shooter, and Role-playing games have the highest global sales revenue, on average, but the Action game genre has the most number of games.”) Each finding must involve two or more variables, and the findings should not be repetitive (e.g., you should not use the same code but just change the variables for each finding). Describe each finding in clear language and discuss why it is interesting in the context of your data.
  1. (15pt) Visualization (mutivariate). Make three visualizations using ggplot. Each plot must involve two or more variables, and the type of the plot should not be repetitive (e.g., you should not do three scatterplots of different pairs of variables). All of the plots should be polished with correct labels, titles, axis, and good choices of color, point type, etc. For each plot, write 2-3 sentences to describe what the plot shows and discuss what you find interesting about the plot.
  1. (6pt) Statistical significance. Discuss one finding that involves an evaluation of statistical significance (i.e., p-value). Essential components you need to include in your finding are summary statistics, what test is used, p-value, and a conclusion based on the p-value. You can find examples of other tests (t test, correlation test, ANOVA) in Lecture 6.
  1. (6pt) Conclusion and Discussion. In a paragraph (at least 5 sentences), summarize your findings from the dataset and discuss anything you find challenging or interesting when working on this project. Does your dataset have a lot of NA values? Do you have to do any (a lot?) data clean up? Does any aspect about dealing with this dataset surprise you?