Data Analysis Using Python - University of Pennsylvania

Data Analysis Using Python

 - a course by the University of Pennsylvania 

My dairy about the journey of studies:

• Instructor: Brandon Krakowsky
• Coursera Course - Data Analysis Using Python
• Part of Introduction to Programming with Python and Java Specialization
• 
  

 

Data Analysis Using Python

 - Part 1 of "Introduction to Programming

 with Python and Java Specialization"

• Module 1: Loading, Querying, Filtering Data using the csv Module

          

     

• Module 1 - (3) Importing and reading a file using the csv module

• Loading Data:

• Download and install Anaconda (Python 3 + Jupyter Notebook all at once)

• Coding demo 1: Analyzing the 500 Greatest Albums of All Time
• Coding demo 2: 
• Lambda: 
  
  
  
• e.g.: Github Module 1 - 
• Line #30: 

  

  
  
• 

  
  
  
• 

  
  
  
• 

  
  Built-in Function, Sorted( ) Examples:
•   
• 

  
  
  
• 

  
  
  
• 

  
  
  
• 

  
  
• 

  
  
  
• 

  
  
  
• Quiz 1:
• 
  
  
• 

  
  
• 

  
  
• 

  
  
• 

  
  
  
• Quiz 2:
• 

  
  

  The correct answer is:

  
  

  ✅ filtered_flights = [row for row in flights if row["Departure"] == "Philadelphia" and row["Destination"] == "Los Angeles" and float(row["Price"]) < 1200]
• 

  
  
  
• 

  
  
  
• 

  
  
  
• 

  
  
  
  
• Run Module 1 - Lab 1 - Work with ufo sightings data (hints: better to create your own github account and "fork" the whole project space (as in your own account space) to run.

• Module 2 - (1) Resources
•  Module 2 - (2) Pandas module - Loading, Inspecting & Querying Data 
• 
  
  
  
• 

  
  
• 

  
  
• 

  
  
  
• Quiz 3: Loading, Inspecting, & Querying Data
• 

  
  
• 

  
  
• 

  
  
• 

  
  
• 

  
  
• 

  
  

  
  

• (*Step) Clean Data & Deal With Missing Data
• 
  
  
• 

  
  
• 

  
  
• 

  
  
  
• Try Github: Data Analysis, Joining & Filtering Data(.ipynb)
• 

  
  
• 

  
  
• 

  
  
• 

  
  
• 

  
  
  
• 

  
  
• df[-1:]['name'] is the same as that at line #47 above.
• ----- ----- -----
• Module 2: Querying data using boolean indexing
• 

  
  
  
• 

  
  
• 

  
  
• 

  
  
• 

  
  
• 

  
  
  
• Computations - Sum( ) and mean( ): Hyperink
• 

  
  
  
• Other Methods - Video Hyperlink
•  
• 

  
  
• 

  
  
• Module 2 - Code Along Exercise : Add rating column - Video Hyperlink
• 

  
  
  
• Quiz 4 - Joining & Filtering Data
• 

  
  
• 

  
  
• 

  
  
• 

• Module 3: Summarizing & Visualizing Data - NumPy Library

• 

  
  
• 

  
  
• Type Errors: 

  

  
  
• There are four options, and three of them can run /wo errors:
• 

  
  
• .....
• 

  
  
• 

  
  
  
• Module 3: aggregate functions
• 

  
  
  
• Above pivot table is created in index, on state and city. For the values to aggregate (合計), specify the column "review_count" as shown above. Also, specify the aggregate function (e.g. sum) (where np is the numpy)
• To segment our results using the "columns" parameter:   
• 
  

  

  
  
• To pass as an argument to aggfunc( ), a dict object containing different aggregate functions to perform on different values: ("review_count" is the key and Numpy sum method is applied to the column "review_count" below)
• 

  
  
  
• Quiz 5: - Summarizing Data 
• Q1:   

  
  
• It splits the data into different groups by 'genre'. Here's why: df.groupby(['genre']) groups the DataFrame by the values in the 'genre' column.   It creates separate groups for each unique genre value (e.g., Action, Comedy, Drama, etc.).   This allows you to perform aggregate operations on each group separately (like .mean(), .sum(), .count(), etc.)
• Q2: 

  

  
  
• Q3: 

     

     

     Practical Example:  

  

  

  
  

  

  
  
• Q4:   

     

     

     

  

  
  

  
  

  
  

  
  
• Q5:   

     

     

  

  
  
• 
  Jupyter Notebook magic functions: The magic function %pylab inline allows the pylab library to load and let's our visualization to show up inside of a notebook.
  
  
• Module 3: Lab - Summarize Movie & Ratings Data (Jupyter Notebook);    Instructions
• Lab Results: Mod 4 - Summarize Movie & Ratings Data - Lab File in My Github Jupyter Notebook
• Module 3: Histogram Demo
• My Github's Jupyter Notebook File (please fork to your Github account to run)
• Module 3: Quiz 6 - Visualizing Data
• Q1: 
  Q1
  
  
• Q2: 

  

  
  
• Q3: 

  

  
  
• Q4: 

  

  
  
• Q5: 

  

  
  
• Q6: 

  

  
  

• Github: Data Analysis Python - Penn
• Nice free tutorials on: W3schools
• Run VS Code in Github. 
  

(You can also download the files from the Github and run in VS Code under Code Space, or in your local machine)

-----   -----   -----   -----   -----

REPL: Using help( ) function / Help Learning

• To confirm sorted() is built-in by checking its availability in an interactive Python session:
• 
  
  

• Questions to Grok: 

• How to show all the all built-in functions and objects? 
• How to get help on the documentation for sorted() ~ similar with help(sorted)?
• e.g. help('module')?

1. Using dir(__builtins__): 

   

   
   
2. Using help('modules') : Type "python" in Terminal of VS Code to start REPL and then type "help('modules'): 

   

   
    

   

   
   

   
   
3. Using help( ): > Type the module name or use help( ): e.g. help(sorted)

     

    

   

   
   

   
   

   
   
4. Online Documentation website python.org: 

     

   

   
   

   
   
5. Using pydoc to list modules: Anaconda   >   Powershell Prompt  >   python   > (need to import first) (e.g.)  import csv   >   csv

      

     

   

   
   

   
   

   
   

   

   
   
6. Using sys to check Standard Modules:   

   

   
   

   

   
   
7. Interactive Exploration with dir( ) and help( ) :   

      

     

      

   

   
   

   
   

   
   

   
   
8. Learning from Community and Tutorials:   

   

   Python.org: https://docs.python.org/3/tutorial
   
   
   
9. Checking Your Python Installation & Run practical Examples: