I would have to disagree with some of the folks that have previously posted here.  In my area - products at Whole Foods are typically $1 or $2 MORE than other retailers.  I don't even bother going in there.  
I find that the farmer's market and local grocery store is reasonable, but you have to shop around and read the labels.  I give myself plenty of time to hit the grocery store once a week and stay away from the preprocessed foods.  

Based on my shopping experience buying organic fresh vegetables it is quite a lot more expensive - I researched which fruit and vegetables have the highest pesticide load and try to focus on buying the higher exposure ones organic and being more flexible on the ones that are low on the scale to save some money. I found this link and printed off the list to keep it with me when I shop. http://www.foodnews.org/fulllist.php. Green peppers for example can be around $1.50 to $2.99 a pound depending but the organic ones are $5.99 for two small peppers and they are covered in plastic, probably have been in transit for weeks because they sure aren't fresh. In my area (upstate NY) it definitely costs me quite a bit more to shop organic. On some items the difference isn't that bad so I do it as much as I can afford and then I also have used the link I provided to help prioritize as well...

Oh so many great suggestions!  I noticed that the organic 365 brand was cheaper than the organic brands at other grocery stores too.  A little more than conventional brands, but I like to think of the 10 or 20 cents as an investment in my family's future.  I am new to this way of shopping though, and find myself getting reciept shock due to purchasing new staples and convenience foods.  I am trying to restrain myself and remember what Alicia wrote in the book about buying ingredients  for a few recipes at a time and gradually build up my new pantry.  After all Rome wasn't built in a day ;-)

Also, I just bought a share in a local organic CSA and it feels so great to know that not only am I supporting a local friendly farm, but that I will be able to feed my family of five for about $34 a week for 13 weeks!  Of course I will need to purchase grains and other staples,  but I am just drooling when I think of all of the yummy veggies!
Please keep all of the great budget cutting ideas coming.  I hope to convince the hubbo that not only is organic vegan eating going to make him feel better, support our earth, but also that it is not cost prohibitive.  In fact, I want to try to show him that it can even be cheaper.  Think I can do it?

Amber, good for you! CSAs are awesome, which makes me want to bring up another point concerning saving money while shopping produce: eat seasonally & locally.  For me, organic winter squash are a great price right now because they're grown right here & don't have to travel; organic peppers are expensive because they are no where near being in season &, therefor, being shipped thousands of miles to reach me.  Seasonal food is more nutritious, fresher (obviously), kinder on the planet & kinder to our bodies (I don't need cooling, tropical fruits in the middle of winter in the high mountains; I need foods that are indigenous to my area.)  Of course we all have favorite out-of-season or out-of-area indulgences, but we should really try to limit these.  Kinder on our wallets; kinder on Mother Earth.

If you have any Asian or Mexican markets in your area, shop there. Very cheap and there is usually high turnover. Also, you can usually find fun items that you aren't used to. Even if you don't buy them, it's fun to look!