Numpy, Pandas, Matplotlib lab

Here’s a set of 14 short programming problems that focuses on NumPy, Pandas, and Matplotlib that can be attempted whilst waiting for your viva voce quiz.

These problems will test various aspects of using NumPy, Pandas, and Matplotlib, including array and DataFrame creation and manipulation, basic statistical calculations, data filtering and grouping, and data visualization.

This lab is self-directed- the answers are given here but should be looked at only after you have attempted the solutions. Any questions can be discussed at next week’s lab or on the forum. There are many good resources on the web- a numpy and pandas cheatsheet have been linked for initial help. numpy cheatsheet pandas cheatsheet

  1. Create a NumPy array containing 20 random integers between 1 and 100, and then calculate the mean, median, and standard deviation of the array.
  2. Generate a 5x5 NumPy array with random integers between 0 and 50, and find the minimum, maximum, and sum of the array elements using a vectorised approach. For comparison, also compute the minimum using explicit looping over the rows and columns of the array.
  3. Create a Pandas DataFrame containing two columns: ‘A’ with 30 random integers between 1 and 100, and ‘B’ with 30 random integers between 50 and 150. Calculate the correlation between columns ‘A’ and ‘B’.
  4. Load the iris.csv CSV file into a Pandas DataFrame and display the first 10 rows, last 10 rows, and a summary of the dataset (use the Pandas read_csv, head, tail, and describe methods).
  5. Filter this Pandas DataFrame to include only rows where the specific column “Sepal.Length” value is greater than a given threshold 5.1.
  6. Using NumPy, create an array of 100 x-values linearly spaced between -10 and 10. Calculate the corresponding y-values using the equation y = x^2, and store the results in a Pandas DataFrame with columns ‘X’ and ‘Y’.
  7. Using Matplotlib, create a scatter plot of the ‘X’ and ‘Y’ columns from the DataFrame created in problem 6. Add axis labels and a title to the plot.
  8. Generate two NumPy arrays ‘X’ and ‘Y’ with 50 random values each. Compute the covariance matrix between ‘X’ and ‘Y’ using NumPy, and then calculate the correlation coefficient.
  9. Using Pandas, group the iris DataFrame from problem 4 by the categorical column “Species” and calculate the mean, median, and standard deviation for each group in a the “Sepal.Length” numerical column (hint: look at the “agg” member function).
  10. Create a bar plot using Matplotlib to visualize the mean value of the numerical column for each group from problem 9. Add axis labels, a title, and a legend to the plot.
  11. Create a 6x6 NumPy array with random integers between 1 and 100. Extract the first 3 rows and last 3 columns of the array using indexing.
  12. Create a NumPy array of 25 random integers between 1 and 50, then reshape it into a 5x5 matrix. Extract the elements from the main diagonal of the matrix.
  13. Create a Pandas DataFrame with 3 columns (‘A’, ‘B’, ‘C’) and 10 rows of random integers between 1 and 100. Select the rows where the values in column ‘A’ are even and column ‘C’ is greater than 50.
  14. Using the DataFrame created in problem 13, select the values in column ‘B’ corresponding to the first, third, and fifth rows.
bars search times