Introduction to Neural Networks and PyTorch - IBM

 

Introduction to Neural Networks and PyTorch


Data Science   >   Machine Learning   >

  • There are 6 modules in this course: 
  • PyTorch is one of the top 10 highest paid skills in tech (Indeed). As the use of PyTorch for neural networks rockets, professionals with PyTorch skills are in high demand. This course is ideal for AI engineers looking to gain job-ready skills in PyTorch that will catch the eye of an employer.
  • AI developers use PyTorch to design, train, and optimize neural networks to enable computers to perform tasks such as image recognition, natural language processing, and predictive analytics.  During this course, you’ll learn about 2-D Tensors and derivatives in PyTorch. You’ll look at linear regression prediction and training and calculate loss using PyTorch. You’ll explore batch processing techniques for efficient model training, model parameters, calculating cost, and performing gradient descent in PyTorch. Plus, you’ll look at linear classifiers and logistic regression.   
  • Throughout, you’ll apply your new skills in hands-on labs, and at the end, you’ll complete a project you can talk about in interviews.  If you’re an aspiring AI engineer with basic knowledge of Python and mathematical concepts, who wants to get hands-on with PyTorch, enroll today and get set to power your AI career forward!













    • Using Sigmoid function ​as the activation function.....
  • Module 1: Artificial Neural Networks - Forward Propagation - Lab
    •  => Launch app


    • Labs.cogntiveclass.ai - Hyperlink




    • Quiz 01: 




    • Module Quiz: 








    • Module 1 Quiz: 

  • Module 2: Basics of Deep Learning - Gradient Descent
    •  Objectives




  • Module 2 - b- Backpropagation Algorithm














  • Module 2 - c - Lab Backpropagation  - Click (right) Github Hyperlinks
  • Module 2 - d- Vanishing Gradient


  • Module 2 - e:- Activation Functions




    •  Sigmoid Function, widely used in the hidden layers of a neural network


    •  Limitation: 1) Its gradient becomes small once the function falls in that region; 2) Not symmetrical


    •  Hyperbolic tangent function (tanh) - also has vanishing gradients


    •  ReLU function, not activating all neurons simultaneously.


    •   Softmax Function


    •  Conclusion of activation functions


  • Module 2 - f - Lab - Vanishing Gradient & Activation Functions  - Click (right) Github Hyperlinks

    •  
    •  Module Not Found Error


    •  "Dirty Installation"




  • Module 2 - Practice Quiz
    • Your grade: 100%

      Your latest: 100%
      Your highest: 100%
      To pass you need at least 60%. We keep your highest score.
      Question 1

      _____________ is an iterative optimization algorithm for finding the minimum of a function.

      Do you understand?.

      Nice work

      Gradient descent is an iterative optimization algorithm for finding the minimum of a function. To find the minimum of a function using gradient descent, we take steps proportional to the negative of the function's gradient at the current point.

      1 / 1 point
      Question 2

      How does the backpropagation training process start?

      Do you understand?.

      Nice work

      Backpropagation begins after forward propagation and computing a differentiable loss function (e.g., cross-entropy or mean squared error).

      1 / 1 point
      Question 3

      Which of the following types of activation functions can cause the vanishing gradient problem? Select three answers.

      Do you understand?.

      Nice work

      The binary step activation function can cause the vanishing gradient problem.

      Nice work

      The sigmoid activation function can cause the vanishing gradient problem.

      Nice work

      The hyperbolic tangent activation function can cause the vanishing gradient problem.

      1 / 1 point
      Question 4

      Which of the following activation functions does not activate all neurons simultaneously?

      Do you understand?.

      Nice work

      In addition to being nonlinear, the main advantage of using the ReLU function over the other activation functions is that it does not activate all the neurons simultaneously.

      1 / 1 point
      Question 5

      Which of the following activation functions is ideally used in the classifier’s output layer?

      Do you understand?.

      Nice work

      The softmax function is ideally used in the classifier's output layer, where we try to get the probabilities to define each input's class.

  • Module 2 - Graded Quiz











    • Your grade: 100%

      Your latest: 100%
      Your highest: 100%
      To pass you need at least 70%. We keep your highest score.
      Question 1

      Which of the following algorithms is used to optimize weights and biases in a neural network?

      Do you understand?.

      Nice work

      Gradient descent is an iterative optimization algorithm that finds the minimum of a function by calculating gradients and updating parameters in the direction of steepest descent.

      1 / 1 point
      Question 2

      For a cost function, J = Σ(zi - wxi - b)², that we would like to minimize, which of the following expressions represent updating the parameter, w, using gradient descent?

      Do you understand?.

      Nice work

      This expression correctly shows the gradient descent update rule where the parameter moves in the opposite direction of the gradient to minimize the cost function.

      1 / 1 point
      Question 3

      While reviewing a chart of an activation function, you see a function that outputs zero for all negative inputs and returns the input itself for positive values. What type of activation function is this?

      Do you understand?.

      Nice work

      ReLU (Rectified Linear Unit) is defined as f(x) = max(0, x), creating a piecewise linear function that eliminates negative values while preserving positive ones.

      1 / 1 point
      Question 4

      You're analyzing an activation function that outputs values between -1 and 1 and has an S-shaped curve centered at the origin. Which activation function does this describe?

      Do you understand?.

      Nice work

      The tanh function is symmetric around the origin and maps any real number to the range [-1, 1], making it zero-centered, unlike the sigmoid function.

      1 / 1 point
      Question 5

      For multi-class classification problems, which neural network component typically contains the softmax activation function?

      Do you understand?.

      Nice work

      Softmax transforms the final layer's raw logits into a probability distribution, enabling the network to output confidence scores for each possible class.

      1 / 1 point
      Question 6

      What is the correct sequence of steps in the backpropagation training algorithm?

      Step a: Calculate the error between the ground truth and the estimated or predicted output of the network. Step b: Update the weights and the biases through backpropagation. Step c: Calculate the network output using forward propagation. Step d: Repeat the previous steps until the error between the ground truth and the predicted output is below a predefined threshold.

      Do you understand?.

      Nice work

      This sequence follows the logical flow of neural network training: forward pass to generate predictions, error calculation, backward pass to update parameters, and iteration until convergence.

      1 / 1 point
      Question 7

      You're working on a 15-layer neural network for image recognition, but the early layers seem to learn very slowly compared to the later layers. What is the most likely cause of this training issue?

      Do you understand?.

      Nice work

      In deep networks, gradients can become exponentially smaller as they pass through many layers, especially when using activation functions whose derivatives approach zero. This causes earlier layers to learn extremely slowly.

      1 / 1 point
      Question 8

      In a simple neural network implementation, what is the correct mathematical relationship for computing the gradient of the loss function with respect to a weight connecting neuron i to neuron j?

      Do you understand?.

      Nice work

      This correctly applies the chain rule where the gradient flows from the loss to the activation, then to the weighted sum, and finally to the weight parameter.

      1 / 1 point
      Question 9

      Which activation function is most effective at preventing the vanishing gradient problem in deep neural networks?

      Do you understand?.

      Nice work

      ReLU has a constant derivative of 1 for positive inputs, which prevents the exponential decay of gradients that occurs with saturating activation functions like sigmoid and tanh.

      1 / 1 point
      Question 10

      You're coding a neural network from scratch and need to implement the forward propagation for a hidden layer neuron. What sequence of operations should your code perform?

      Do you understand?.

      Nice work

      This follows the standard neuron computation: z = Σ(wi × xi) + b, then a = f(z), where f is the activation function.

  • ----------------------------------------------
  • Module 3: Deep Learning Libraries


    •  TensorFlow most popular


    •  (2:10) PyTorch in 2016


    •  (02:40) Keras - hi-level API


    •  Conclusion


    •  (03:43) Recap/Summary1


    •  Recap/Summary2


  • Module 3b - Regression Models with Keras
    •  (00:09)


    •  (00:40) e.g. Different materials set to build concrete mix - Pandas data frame





    •  (01:23) Neural network to output the compressive strength of a concrete sample (most 50 ~ 100 hidden layers)


    •  (02:10) Split Data frames -> Predictors, Target


    •  (02:45) Keras Code with two models: e.g. Sequential (another model: functional API)


    •  (03:35) module 2 uses gradient descent as min / optimization algo, and the MSE (mean-Sq Error) as our loss measure % the predicted value & the ground truth.


    •  (04:00) Adam Optimizer algo and the MSE - no need to specify the learning rate


    • (04:13) fit method to train our model
    •  (04:17) we can do Predictions by: predictions = model.predict(test_data)



  • Quiz 1
    • Your grade: 100%

      Your latest: 100%
      Your highest: 100%
      To pass you need at least 60%. We keep your highest score.
      Question 1

      Which of the following libraries is particularly suited for running machine learning algorithms on GPUs?

      Do you understand?.

      Nice work

      PyTorch is the cousin of the Torch framework, which is in Lua, and supports machine learning algorithms running on GPUs in particular.

      1 / 1 point
      Question 2

      Which of the following statements is correct?

      Do you understand?.

      Nice work

      Keras is a high-level API for building deep learning models, and it has gained favor for its ease of use and syntactic simplicity, facilitating fast development.

      1 / 1 point
      Question 3

      What are the two model classes in the Keras library? Select all that apply.

      Do you understand?.

      Nice work

      There are two models in the Keras library. One is the Sequential model, and the other is the model class used with the functional API.

      Nice work

      There are two models in the Keras library. One is the Sequential model, and the other is the model class used with the functional API.

      1 / 1 point
      Question 4

      When adding a ‘dense’ layer to the first hidden layer in a regression model, what three things do we specify?

      Do you understand?.

      Nice work

      When adding a ‘dense’ layer to the first hidden layer in a regression model, we need to specify the number of neurons in each layer, the activation function, and the number of columns in our data set.

      1 / 1 point
      Question 5

      When building classification models with Keras, what do we need to do to the target column before we can use it?

      Do you understand?.

      Nice work

      When building classification models with Keras, we need to transform the target column into an array with binary values before we can use it.

    • 1. Which of the following libraries is particularly suited for running machine learning algorithms on GPUs?

      • Correct Answer: PyTorch

      Note: PyTorch and TensorFlow are designed with native support for CUDA to accelerate tensor computations on GPUs, whereas standard libraries like Pandas, NumPy, and Scikit-learn run primarily on the CPU.

      2. Which of the following statements is correct?

      • Correct Answer: Keras is a high-level API that facilitates fast development and quick prototyping of deep learning models.

      Note: Keras acts as a user-friendly interface built on top of lower-level frameworks (primarily TensorFlow) to streamline model building.

      3. What are the two model classes in the Keras library? Select all that apply.

      • [X] The model class used with the functional API

      • [X] Sequential model

      Note: Keras provides two primary ways to construct models: the straightforward Sequential class for linear stacks of layers, and the flexible Model class used with the Functional API for complex architectures.

      4. When adding a ‘dense’ layer to the first hidden layer in a regression model, what three things do we specify?

      • Correct Answer: The number of neurons in each layer, the activation function, and the number of columns in our data set.

      Note: For the very first hidden layer, Keras requires the input shape (input_dim or input_shape), which corresponds to the number of feature columns in your dataset. The number of targets is only defined later in the final output layer.

      5. When building classification models with Keras, what do we need to do to the target column before we can use it?

      • Correct Answer: Transform it into an array of binary values

      Note: Categorical labels must typically be converted into a one-hot encoded format (an array of binary vectors via tools like to_categorical) so the neural network can compute categorical cross-entropy loss effectively.

  • Module 3 d - Classification Models with Keras


    •  Our target variable of 4 categories


    •  (01:30) Four decisions


    •  (01:55) Divide dataset into Predictors and Target


    •  (03:00) Add method to create two hidden layers, each with 5 neurons, & neurons are activated using Relu fcts.


    •  (04:15) Keras predict method


    •  (04:43) Should be acceptable - 2nd class are higher


    •  (05:30) Recap


  • Module 3 - Lab - Classification with Keras



    • ------
  • Quiz 2 - Graded Quiz - Classification with Keras
    • Your grade: 100%

      Your latest: 100%
      Your highest: 100%
      To pass you need at least 70%. We keep your highest score.
      Question 1

      Imagine you’re rapidly building a deep learning model and want to minimize boilerplate code. Which of the following would best support fast development and easy prototyping?

      Do you understand?.

      Nice work

      Keras is a high-level API for building deep learning models. It has gained favor for its ease of use and syntactic simplicity, facilitating fast development.

      1 / 1 point
      Question 2

      Which library is known for its high-level, user-friendly API that makes deep learning more accessible to beginners?

      Do you understand?.

      Nice work

      Keras is designed with a focus on user experience, providing a simple and intuitive API that makes deep learning accessible to beginners and enables rapid prototyping.

      1 / 1 point
      Question 3

      You’re using Keras to build deep-learning models. Which model types does the Keras library support?

      Do you understand?.

      Nice work

      The Sequential model simplifies neural network construction by allowing layers to be added individually, perfect for beginners learning model design.

      1 / 1 point
      Question 4

      For classification models in Keras, which activation function is most appropriate for the output layer when dealing with multi-class problems?

      Do you understand?.

      Nice work

      The softmax activation function is ideal for multi-class classification as it converts raw output scores into probability distributions that sum to 1, making it suitable for categorical outputs.

      1 / 1 point
      Question 5

      For regression problems involving continuous target variables, which output layer activation function provides the most flexibility?

      Do you understand?.

      Nice work

      Linear activation enables the network to output any real number without bounds, providing the flexibility to predict continuous values across the entire range of possible outcomes.

      1 / 1 point
      Question 6

      For a neural network to predict categorical outputs, which loss function optimally measures prediction accuracy?

      Do you understand?.

      Nice work

      Categorical cross-entropy calculates the logarithmic difference between predicted probabilities and actual class labels, providing efficient gradient information for optimizing classification models.

      1 / 1 point
      Question 7

      For continuous outcome prediction models, which evaluation metric best captures prediction accuracy?

      Do you understand?.

      Nice work

      MSE penalizes larger prediction errors more heavily than smaller ones, providing a comprehensive measure of how well the regression model predicts continuous values.

      1 / 1 point
      Question 8

      Which Keras compilation parameters should be used for optimal performance in regression model implementation?

      Do you understand?.

      Nice work

      This combination provides an appropriate loss function for continuous outcomes, efficient optimization, and meaningful performance metrics for regression assessment.

      1 / 1 point
      Question 9

      You’ve developed a Keras model to classify customer reviews as positive, negative, or neutral. Which metric best indicates your model’s effectiveness?

      Do you understand?.

      Nice work

      Accuracy provides a clear percentage of correctly classified reviews, making it easy to understand how well the model performs across all three sentiment categories.

      1 / 1 point
      Question 10

      Why is Keras particularly suitable for deep learning beginners compared to lower-level frameworks?

      Do you understand?.

      Nice work

      Keras simplifies neural network development by providing high-level APIs that hide low-level complexities while allowing beginners to experiment with different architectures and parameters.

  • Module 4 - Deep Learning Models
  • Module 4 - Shallow Versus Deep Neural Networks
    •  (02:45) Recap




  • Module 4 - Deep Learn Models - 4b - Convolutional Neural Networks
    •  (00:25) Outline


    •  (01:00) Intro - CNN - difference? CNNs are best for solving related problems ~: image recognition, object detection, and other computer vision applications.


    •  (01:30) CNN architecture


    •  (01:50) Input Layer - images: no. 3 represents R G B components of each pixel


    •  (02:33)


  • d









留言

  1. Great post! It's very informative and valuable for readers. Keep sharing more blogs with useful insights.
    ipera.ai

    回覆刪除

張貼留言

這個網誌中的熱門文章

Intro to Predictive Analytics Using Python - University of Pennsylvania

Get started with Python - Google

Data Analysis Using Python - University of Pennsylvania