Tag: Matlab

So in this class we’re going to talk about generating a random variable from a certain distribution. This could be useful for doing simulations of systems that have uncertainty. Matlab has some built-in functions to help us do this. So the first one we’re going to talk about is called randi, and it takes one argument called imax, and this function gives us a uniformly distributed variable between one and imax. So let’s try it. So 9 is in between 1 and 10. Now there’s another function randi which takes in a maximum value, and another parameter called n. So let’s set n to 3, so that returns an n-by-n matrix of random values between 1 and imax. So suppose I wanted to generate random values between 10 and 20, how would we do that? Because randi can give us values anywhere between 1 and imax, so what we could do is we could just add 10 to all the values that randi returns. So this gives us a 3 by 3 matrix with values only between 10 and 20. Another useful function is just rand by itself. So this function gives us a random number between 0 and 1, so it’s different from the previous one where we don’t get integers we get real numbers. rand returns a number with a uniform distribution, so the probability of getting point .25 is the same as the probability of getting 0.75. So let’s try and imply histograms for different values of n. Alright, so this is a histogram for random numbers between 0 and 1, and N equals 10 array. Alright, so it’s not quite uniformly distributed, let’s try a bigger n value. Alright, so immediately it starts looking more uniformly distributed as n increases, so let’s try a bigger n. Alright, so it looks even more uniformly distributed. Now, 10,000. Alright, so it’s almost flat even. That’s 100,000, and this is a million, it looks almost perfectly flat. So that’s the idea with the frequentist view of probability is that when n approaches infinity, your probabilities approach their true values. So now let’s think about a different problem. Suppose I want a specific discrete distribution, so say I want to simulate an unfair coin. So, to write it out I want p of heads equal to .25, and I want p of tails to equal 0.75. How could I write a function to give me random values that could draw from this distribution instead of a uniform distribution? So we can create a function to do this. We can call it biased coin, it’s going to take in one value little p which represents, let’s call it P heads which is probability of getting heads, and it’s going to return the coin face. So we’re going to generate a random value, if it’s less than P heads we’re going to return heads, else we’re going to return tails. Let’s try our function. Alright, so now we’re going to try our new biased coin function by initializing an array of say size 1,000…you know what we’re going to do this in a separate function. We’re going to initialize a n by 1 array, we’re going to count from 1 to n, and we’re going to use the biased coin function to generate a value for each element of the array. Alright, so let’s try the function we just made. Test coin .25 for n equal to 1,000. Alright, so you see the number of heads which resolves to the integer 104, and then tails resolves to the integer 116. So you see this is about 250 and this is about 750 which is what we would expect in the thousand coin tosses.

Welcome to this third lecture on matlab and probability. In this lecture we’re going stray from your typical probability course. So we’re going to have some data matrix, and we’re going to measure the probability of a variable that the data matrix represents. So what I want you to do is I want you to go to in your web browser github.com/lazyprogrammer/matlab-probability-class. Once you go there, you’re going to copy and paste this SSH clone URL, and I want you to go into your terminal and type in and git clone, and then paste that URL. I’ve already done it so I’m not going to do it again. This is going to give you some files that are relevant to this class that I’m going to use for the coming lectures. Okay, so, now that you have those files you want to go into matlab, change your directory to work in that same directory that you guys checked out from git. So we’re going to load this data. Ok, so, r is a 100 by 1 matrix, so let’s just plot r amd let’s see what it looks like. Alright, so, it has a bunch of random values that are between -5 and 5. So now let’s say I wanted to calculate the probability that r is equal to -5. How would I do that? So, one way is I could sum all the values where r is equal to -5 and divide it by the total number of values in r. So it gives me .07. I can do the same thing for every other value in the matrix. So we get about .06 to .11. So this is what we call the frequentist view of statistics. It means that if we flip a coin 1000 times, and that’s a fair coin with heads or tails, the probability is 50% and we should get heads about 500 times, and we should get tails about 500 times. The idea is that as the number of coin flips approaches infinity, our measurement of the probability of heads should approach 0.5. So we can also plot the histogram of r, and this should give us an idea of the shape of the distribution. Alright, so, let’s do a little more complex example. Let’s say I want to calculate the probability r is even. How can we do that? So in the same way as we did before, we might want to say r equals -4 or r equals -2, but that wouldn’t be the best way to do this. We would use the modulo function. So if mod(R,2) is equal to 0, that means r is even, divided by the length of r. Alright, so the probability that r is even is .43, and we can do the same thing to determine the probability that r is odd. And so notice that these two events are disjoint. You can either have r equals even or r equals odd, and those two events are the only possible events. So their probabilities should add up to 1, and we can verify that .43 plus .57 is equal to 1. Now so let’s say I want to test if r takes on a specific value, so let’s say I want to calculate the probability that r is equal to -5 or positive 5. How would I do that? So we would use the or operator. So r equals 5 or r equals -5, divided by the length of r. So the probability that r is equal to 5 or negative 5 is .21. Notice that we can use the same method for our first problem which was to determine even or odd. Which is the way I suggested not doing it, but we want to check our answers, and so that gives us the same answer. The probability that r is equal to -4, -2, 0, 2, or 4 is the same as the probability that r is even.

Welcome to this course on matlab and probability. This first lecture will focus on the introduction and course outline. Because this course uses matlab it won’t follow a traditional probability and statistics course outline, rather i’ll show you as we go along how the concepts of probability can be applied, or viewed, through the lens of matlab. More generally, I want to show you how a programmer might approach problems in probability. So now let’s talk about some of the topics we’re going to go through in the second ecture. We’re going to talk about what is probability, and we’re going to give some definitions and examples. In the lecture after that we’re going to talk about how we can measure probability given some data. So, how to open a file, and measure the probability of some of the features of your data. In the next lecture we’ll talk about how do you generate random data, so how can you do a probabilistic simulation. In the next lecture we’ll talk about a famous problem in probability called the birthday problem, or the birthday paradox. In the next lecture we’ll extend the idea of probability from discrete variables to continuous variables. In a lecture after that we’ll talk about a special continuous variable distribution called the Gaussian distribution, or the normal distribution. In the next lecture we’ll talk about if you have some data that is continuous, how do you test if it is Gaussian distributed? In the lecture after that, we’ll talk about if you have two different Gaussian distributed groups of data, how can you compare the two? And then in the last lecture, we will extend the idea of the Gaussian to a multi dimensional Gaussian. So, what will you be able to do by the end of this course? You’ll understand mathematical problems that contain uncertainty. You’ll be able to quantify uncertainty both in theory and in practice. You’ll be able to use matlab to measure uncertainty in your data. You’ll be able to use matlab to simulate systems that contain uncertainty. You’ll be able to understand and calculate probabilities in the famous birthday paradox. You’ll understand the mathematics behind the very famous bell curve, or a normal distribution, or Gaussian distribution. You’ll know whether or not the data you’re working with is Gaussian distributed, and you’ll know how to handle Gaussian distributed data. I look forward to teaching you.

So now let’s take our Gaussian and convolve it with the image. So now remember that A is 512 x 512 x 3, which is a three-dimensional matrix, and H is a two-dimensional matrix. So if I try to do this I’m going to get an error I’m sorry we are using the CONV2 function, and you still get an error because A is not supposed to be a three-dimensional matrix, right. So if we look at the definition of convolution we’re working with two dimensional matrices, if we have two dimensional convolution. So what we want to do is let’s just take the red channel and it doesn’t matter which one, because remember when we were looking at the red, green, and blue channels we pretty much saw the same image for every channel. Ok, so, we still get another warning and that’s that A is still in values of UINT8, so let’s change that to double. Ok, so, now let’s imshow(C) and see what we get. So it’s all white, and so what does white mean? That means all of the values are too high, right, so there’s too much intensity in this image. So, you have to play around with the values a little bit, so let’s try making a smaller filter. Ok, so, let’s say H is equal to my_gaussian 25 and then sigma is 5. So now the filter’s smaller and the sigma is smaller, so the blur is going to be less, but the intensity is still too high. So that could mean just the values are too high and we need to multiply by lower values. So let’s divide the filter values H x 1000, and imshow(C) again. So now we’re starting to see some black, right, so that’s good. Ok, so, I’m decreasing values and I’m starting to see the image. Okay, so, here’s the original image but blurred using a Gaussian of sigma equals 5. Now one thing I didn’t show you guys because I wanted you to go through the exercise of creating a two dimensional Gaussian filter yourself, is that we already have a function called fspecial in matlab that create filters for us, and so fspecial can create many different types of filters. So, in addition to the Gaussian it can create laplacian filters, an averaging filter which is another thing we’ve used, the Sobell filter which is useful for finding edges, so all different types of filters. So let’s try using fspecial instead. fspecial(‘gaussian’, 25, 5); Now let’s do our convolution. So I’m going to divide by a thousand, because I know I’m going to have to divide. So let’s just see what we see. Ok, and so it’s a little dark so I didn’t have to divide by a thousand, maybe I could had divided a lower number, but you can see the idea is that we’ve blurred the original image using the Gaussian filter given to us by fspecial instead of our own Gaussian filter that we built. Ok, so, like we did before I want to plot the filter that we created. So I’m going to go imshow little h, which is the filter we created with fspecial, and notice how it’s just all black. Remember that 0 is black and 1 is white. So what we could do if we want to look at what’s in h, let’s check the maximum value of h. So max, max because it’s two-dimensional. The maximum value is .0065, so of course when we plot that it’s going to be pretty close to black. So what we could do if we want to scale it by one, if we want the maximum value to be one, we could do imshow(h) divided by the maximum value. Ok, so, now we see what we expect to see which is white in the middle. So one question you might have is why is the value of h so small when we use fspecial? That’s because it does a thing called normalization. So if you have ever studied probability you know that a probability distribution has to sum to 1, so here it is a similar thing. So if we sum across both dimensions of h we get 1, and so that’s why the values of h are so small. And so now just for completion sake, I want to do sort of what’s the opposite of blurring, I want to do edge detection of the image. So I want to find all the edges and set those values to 1. So matlab has a method called edge that will do this, and it’s very simple to use. You pass in again the gray-scale image, right, so only a two-dimensional matrix that won’t work if you pass in the entire image matrix. Ok, so, edge just does all of that automatically for you. So if imshow(E) I can more or less see where all the sharpest edges are, and of course there are parameters you can pass into edge to make it more or less sensitive, but this is by default what it does.

Now we’re going to move on to the next step in order to implement our blurring tool, or our blurring filter. So the first thing we need to do is we need to extend this idea of convolution to two dimensions. So, the first modification we did so we looked at convolution at it’s most basic definition where it’s of a continuous variable filtered by another continuous variable, so we discretize it by turning it into a sum, and so we already have discrete signals because we’re working in matlab. Now, when we do two-dimensional convolution notice we now have two dummy variables. So before tau or k, now we have two dummy variables tau 1 and tau 2, or n1 and n2,. And so it’s basically what you would expect when you have one deconvolution you’re going from minus infinity to infinity along the one dimension that’s the independent variable, so you can think of that as time, and in two dimensions we go from minus infinity to infinity for both independent variables so that the x and y, or in other words the two spatial dimensions of the image. So now that we’ve extended our idea of convolution to two dimensions let’s think about how we could implement a blur using a Gaussian. So a Gaussian is basically a spread, right, there’s a middle point and then it spreads out over a radius in a circular fashion. So, my question to you is how can we build a two-dimensional Gaussian image that we can use as a filter on the original image, so that we can do a convolution between those two? So, I’m going to give you a minute to think about that, please pause this video and come back when you have figured it out. Ok, so, I have in fact actually been showing you the solution to this for multiple videos now. So I’ve created a function called my Gaussian, and it takes in two parameters n and Sigma. So n is going to be the size of the square, so the output is going to be an n x n matrix, and Sigma is going to represent the standard deviation of the Gaussian as is convention when we’re talking about Gaussians. So we have the output called H, now I only need the value of two Sigma squared ever in this equation, so I’m just going to calculate 2 sigma squared at the beginning so I don’t have to do that on every iteration of the loop. I’m going to initialize h to be an n x n matrix of all zeros, and then I’m going to use a for loop to assign every value of age. So, I going from 1 to n, and j going from one to n. Now I assign x to equal i minus n/2, and y to equal j minus n/2. Why is that? Because I want the center point of H, so that would be H of n over 2 over 2, to to be the highest point of the Gaussian, and so that’s when x is equal to 0 and y is equal to 0, and the exponent of 0 is 1. So that would be the maximum value of the Gaussian and then every point from there would be smaller, right, so when I is n/2 x=0, when j is n/2 y=0. And the formula for a Gaussian is x squared plus y squared over 2 sigma squared, and then you take the exponent of the negative of that. Ok, so, what does this actually give us? So let’s use the my Gaussian function, say n is a hundred and Sigma is 10. Now let’s imshow the H that I got. Ok, so you can see so remember that white is the maximum value 1, and black is the minimum value zero, and so this is a Gaussian what a Gaussian looks like when you plot it on an image. So what would I do if I wanted to see more white? I could increase Sigma, right? So, let’s say Sigma is 25, imshow H, and so you see the radius of the white part has increased.

In this matlab video we’re going to talk about convolution. So I mentioned this before when we were talking about the low-pass filter, because they are very similar and related concepts. So, when we’re talking about the low-pass filter we did a very simple filter called the moving average, and so to give you a sense of what that’s doing again you’re taking a window, say five samples at a time, and then you’re sliding that along the signal and taking the average, and then that is the output of the filter. So this sliding motion and then applying some function to that window that’s sliding along is called convolution. So what you’re really doing when you’re applying a filter is you’re convoluting one function with another. So let’s look at the definition of convolution. OK, so, convolution is also known as the star operator, and it’s the integral of one function with this dummy variable. So you can see this sliding motion that I was talking about. Now this will not make a lot of sense to you if you haven’t studied calculus before, but there is one important result from convolution, or the study of convolution, that we should talk about and then you should know, and that is that convolution in the time domain, and we use time as a sort of dummy variable so time to mean actual time, or time can mean space, the important distinction is that you’re going from time in one domain to the frequency in the other domain, so the result that’s important is that convolution in the time domain, so if I convolve one signal with another, this is equivalent to multiplication in the frequency domain. So, what is the significance of this? So that means they’re two equivalent ways of computing the convolution of the signal with its filter. So one way is to just do the convolution using the formula for convolution which is here and of course in matlab it would be a sum not an integral since we have to discretize the signals, but the other way we could do is since convolution in time is equal to multiplication in frequency, we could take the Fourier transform of both signals first, multiply them, and then do the inverse fourier transform to go back to the time domain, and so that would be equivalent to doing convolution. One application of this is the Fourier transform, or the Laplace transform, can be used to solve differential equations. So you can solve them in the frequency domain and then convert that to the time domain to get the signal of interest back. So, now again when we think about convolution a very useful analogy is this sliding motion, and so that’s exactly what the moving average was doing. And so one physical manifestation of this is the blurring of an image, and so you’ll see in a later lecture that I’m going to do that the blurring of an image is actually convolution, but the sliding motion you will actually do you cando by hand and it has same effect. So this is to say if you’ve ever used photoshop and you use the blur tool on an image, you notice that you click on the blur tool and you get a point, and then you slide it across the image on the places where you want to blur, and then it blurs those points.

So, in this matlab tutorial we’re going to do some basic exercises just to get you working with image matrices and they’re going to be very similar to the exercises we did for sound. So the first exercise I want to do is how do we flip and image upside down? I’m going to give you a minute to think about that, so please pause this video and then come back when you have figured it out. Ok, so, we want to flip an image. It’s going to be exactly the same as we did for sound. So, if you recall sound was stored in using different samples in different rows, and essentially that’s what’s happening with an image also. Alright, so, the top of the image is the top of the matrix and the bottom of the image is the bottom of the matrix. So if we do flipud, which stands for flip-up down, of the image that we loaded and then we imshow this image, we see the original image flipped upside down. So now here’s something we couldn’t do with sound because sound is one-dimensional, so you can only flip it one way. Now let’s suppose I want to flip the image in the horizontal direction. How would I do that? So I’m going to give you a minute to think about that. Please pause this video and then come back when you figure it out. Ok, so, if you remember when we were talking about sound we looked at two different kinds of flip. There was flipud which flipped vertically, and fliplr which flipped horizontally. So all we have to do is call the other flip function. I’m going to imshow this flipped image, and then so this is the image flipped horizontally. And of course you could flip the image both vertically and horizontally to get this. Ok, so, now the next exercise we’re going to do involves working with color. So, I said before that the third dimension in the matrix are the red, green, and blue channels of the color. So how can we show this? So let’s say I only want to view the blue channel. How can I visualize that? So, I’m going to give you a minute to think about this, please pause the video and then come back once you’ve figured it out. Ok, so, usually we say RGB because red is the first component, green as a second component, and blue is the third component. So I’m going to assign the original matrix a to a temporary variable B. I want to view blue so what I want to do is I’m going to set red and green to 0. So how would I do that? I say colon to select all of the rows, another colon to select all of the columns, and then I put a 1 because I want to set the first channel which is red to 0. I’m going to do the same for the green channel, and so now only the blue channel has values that are not zero. So if I imshow B, now we see only the blue components of the image. Alright, and so if I imshowed one of the other channels that was set to zero, it’s just pure black. So 0 is black, it means no intensity at all. So, next we’re going to look at the green channel. So I’m gonna set G = A; G colon, colon, so I’m setting again red to 0, but now I don’t want to set green to 0 I want to set blue to 0, so it’s G (:, :, 3) = 0; and imshow(G), so here’s the green channel. So it’s a little bit more intense, or it has a higher intensity than the blue channel, we can see visually. So now let’s say we want to view the red channel. So we’re going to do something very similar to be above. So I don’t want to set red to 0 iIwant to set green to 0 which is the second channel, and I want to set blue to 0 which is the third channel. So imshow(R) and this is the red channel. So one thing to notice is that no matter which channel we look at, we can still pretty much just view the image. Now why is that? It’s because every color contains a red component, a green component, and a blue component. So, when you see colors think of them as sort of a mixture of those three.

Image processing in MATLAB

So, in this matlab tutorial we’re going to switch gears a little bit to a related topic image processing. The first thing I want to talk about is what is an image? So, we can compare it to soundwaves since we’ve already studied those. Now let’s think about the most raw data possible. So, firstly, whereas sound is a one dimensional function, images are two dimensional functions. We have an x, a y, and a z, and so that z is the value at that x and y. In fact, you can think of a heat map or a contour map as basically images. So, we have a 2d function, ok, x & y are continuous coordinates in space. And like with sound we have to sample the data in order to store it in a computer. So, with sound we’re sampling across time and we call that the sampling frequency. What’s analogous to the sampling frequency when we’re dealing with images? I’ll give you some time to think about that, this is actually probably pretty familiar to you. So, with images the analogous concept would be the resolution, right, so the amount of fine-grainess you put into each point in space. For example, Udemy videos are required to be recorded in HD resolution. So, now let’s suppose we have an image called a, a will have the indexes i and j referring to be xy coordinates. Just write this down if it helps you. So, we have an image a, and then a(i,j) would be the xy coordinates. So, now let’s think about what would be stored at a(i,j). In other words, how do we store a pixel? So, now let’s suppose we store just one number at a(i,j), so it’s a two-dimensional matrix or a table. A table of values at each point xy. That image would be, that matrix would represent a grayscale image, right, because if we only have one number we can only represent intensity. So, 0 would be black and 1 would be white, and so a(i,j) would be the intensity at the point ij. In order to store a color we need three parameters r, g, b, or red, green, and blue, and this is what we’ve been looking at for the entirety of this video so far. So, these values usually vary from 0 to 255 so we can have a total of 256 to the power of 3 possible different colors. So, let’s look at what the value of that is. So, that’s about 16.8 million different colors we can represent if we store values from 0 to 255 for RGB. Now what this also means is that if we have a 500 x 500 image, the matrix that we use to represent it isn’t 500 x 500 but it’s 500 x 500 x 3. Sometimes this is called an image bitmap. Alright, so now that we’ve talked about images in the abstract sense, let’s talk about how you would actually open an image in matlab. There’s an image processing tool box that we won’t be using a lot of in these tutorials. Matlab has a function called imread built-in. We’re going to use that function to read in a very famous picture often used in image processing, so if you ever take a computer vision course or multimedia course you’ve probably seen this image before. Ok, so, let’s look at the size of a. So, it’s 512 x 512 x 3, so a three dimensional matrix as promised. Notice the data type is uint8 instead of double that I have for my other matrices which is the default in matlab. So, a unint8 the u stands for unsigned, int stands for integer, and eight means it’s 8-bit. Ok, so, we think about how many values could be stored in an unsigned 8-bit integer let’s go ahead and calculate that. You should pause this video and try to think about the answer yourself first. Ok, so, the answer is 2 to the power of 8, and that’s 256. So, as we discussed previously values for R, G, and B go from 0 to 255 which is 256 different values. There’s a function called imshow that will essentially plot the matrix as an image. So, that’s the image, the famous Lena image, that’s used for image processing.

In this tutorial we’re going to talk about low pass filters, and how to apply those to an audio stream. So, to first recap and kind of give you a different perspective on what we’ve been doing, we’re just going to listen to a sound wave at different sampling frequencies. So, I’m going to create my x-axis (0, 50*pi,10000) samples; I’m going to say y = sin(x); So, this sound is only at one frequency, so you’re it’s going to sound kind of like a buzz. So, I’m going to play the sound at a common frequency that’s used for recording mp3’s and waves 44100. So, let’s listen to this. Ok, and so now what I’m going to do is I’m going to play the same sound wave but at a lower frequency, or sorry a lower sampling rate, and that should give me what sounds like a lower frequency even though it’s the same wave, the same matrix of values. And so it indeed sounds lower pitched, and so now what we’re going to do is we’re going to create a low-pass filter. So, what that does is say we have multiple different sine waves playing, which is essentially what the Fourier transform is giving us, we’re going to filter the signal so that only the low frequency components can be heard. So, if we look at the typical shape of a low-pass filter you generally see things like this, and remember that this is only the left half of the plots that we’ve been looking at. When we’re in the frequency domain convolution turns into multiplication. So, right here zero decibels is actually equal to 1 because it’s a log scale, so we’re multiplying 1 by the signal at these low frequencies, but then as we go down here we’re multiplying a very small number by those higher frequencies. And so after we’ve multiplied one signal by the other one of them being this low pass filter, those high frequencies are going to be silenced. So, if you’ve never studied about low-pass filters before you may want to go and look up the different kinds, what’s their function representation, and the time domain and the frequency domain. We’re going to use a very simple version of the low-pass filter called the moving average. So, matlab already has a function for us that can create filters, we just need to pass in the b and the a which is essentially the different multipliers on the original signal. So, I’m going to set b = ones (40,1)/40; I’m going to call…I’m going to load my data first. [d,fs] equals audioread(‘helloworld.wav’); So, I’m going to say dlp = filter(b, 1, d); so I want 1 as the numerator and then b as the denominator, pass in the data. So, now I want to play the original hello world for reference “hello world” and I’m going to play the new low pass filtered version to compare “hello world” So, you can see the, or you can hear, the low-pass filtered version sounds very muffled, sounds like you’re speaking through a pillow or something, and that’s because high frequencies tend to not go through different medium, so low frequencies can go through a wall. So, if you hear your neighbors talking for instance it will sound a little muffled, and that’s the same thing that’s happening your wall is acting like a low-pass filter. So, let’s do another way of comparing these, I’m going to plot them on the same axis. It’s convenient because they’re the same size, ok. So, now if I zoom in a little bit..so you can see all the sudden changes in the dark green signal don’t happen in the light green signal which is the low-pass filtered version. That’s because when you see sudden changes that’s actually you can think of it as a very fast sine wave that’s super positioned on a low-frequency signal, and so we’re essentially just getting rid of those. Now yet another way to compare these two signals is to plot them in the frequency domain. So, for this we want to use subplots so that they do not overlap each other. Let me just do it all in one line…ok. So, remember that the original signal is on the top and the filtered version is on the bottom. So, now you can see that we’re keeping the low frequencies, but the high frequencies which seemed to have some components that are nonzero are all damping or they’re set to zero in the filtered version. Now if you’re interested in this what I would recommend is an exercise is to try to implement a high-pass filter. As a hint you’re going to want to use the same function filter, and everything else is going to stay the same.