Place an aluminum cake pan into the freezer and let it chill while you proceed with the next steps.
Heat a sauce pan over low heat. Add the 2/3 cup of heavy cream to the sauce pan, then pour the lavender into the cream and let it steep for 7-10 minutes. Do not cover the pan and make sure the heat is not too high. You do not want the mixture to bubble! You just want the lavender to steep. Mix every 2-3 minutes. NOTE It's going to taste very strong, but don't worry, this cream is going to flavor the entire ice cream and the flavor will dilute as you mix it with the rest of the ingredients.
While the lavender steeps, pour the sweetened condensed milk into the mixing bowl. Add the vanilla extract, salt, and a generous amount of purple gel food coloring, if using. Set aside.
When the lavender is done steeping, strain the buds out with a mesh strainer and pour the flavored cream onto the chilled aluminum pan. Place the aluminum pan in the freezer for 5-10 minutes, or until the newly infused cream is cold. You want to make sure its cold before you proceed to the next steps.
Once the lavender cream is cold, take it out of the freezer. Pour the 2 1/2 cups of cold heavy cream into a new mixing bowl and add the infused lavender cream. Beat the cream with a hand mixer on high, until firm peaks form.
Pour the sweetened condensed milk mixture into the whipped cream mixture and lightly fold in, until fully incorporated. If the color is too light for you, feel free to add some more gel food coloring and fold in gently.
Cover with cling wrap and freeze for at least 6 hours. It's best when chilled overnight! Store in the freezer for up to a month. Also, when storing, place in the back of the freezer to make sure its in the coldest part. Enjoy!