This is less of a problem now than previously. There are street markets in many neighborhoods, including some outdoor organic food markets. For example, there is a market on Blvd Richard Lenoir near the Bastille or Chemin Vert metro stops. Another is very near the Pantheon on selected days. There is a huge organic market (that my wife and I did not visit) at Batignolles (near Rome metro stop) but it is only open one or two days each week. There is a nice smaller street market with a variety of goods — mainly food — on Rue du Turrenne near Filles du Calvaire or Republique. I forget what days it is open. It is mostly regular shops augmented by some street vendors. The larger regular groceries (Shopi, Franprix) carry some organic goods and I was able to find the things I needed (soy milk, soy yogurt, etc.) there. That was totally not the case 20 years ago. Also, there is a huge (really huge) epicurean market next to Bon Marche in the 6th or 7th arrondissement and it carries darn near everything. It is unfortunately rather expensive.

Hope this all helps.