A friend sent me an opinion piece by New York Times food columnist Mark Bittman on Christmas Day: “Stop Subsidizing Obesity.” (Find at: http://opinionator.blogs.nytimes.com/2012/12/25/stop-subsidizing-obesity/?hp)

He asserts: obesity is a crisis, soda causes obesity, obesity is associated with serious diseases, and the government subsidizes the makers of sugared drinks through the food stamp program (now called SNAP). His policy solution is to prohibit the use of food stamps to buy soda. He offers some facts and writes persuasively, but I am tough to convince.

Sometimes it pays to wait before forming an opinion. A week later a study hit the media with new findings that suggests a few extra pounds won’t kill you. Now I had paradox.

For whatever reason, this issue captured my attention. My intention here is walk through the process of trying to make sense of a policy proposal dealing with an issue that is somewhat familiar. I am finding it to be complicated and am glad that no pollster called asking for my opinion!The Process of Asking Questions

There are many ways to go about assess a policy proposal. I will share my process. I begin by reflecting on what I already believe. Not that it is right, but it will keep me aware of my biases going in. First, I believe that soda is not real food; it is manufactured, whether made with sugar, high fructose corn syrup or a sugar alternative.

Secondly, I had seen this proposal earlier this year and while I was initially surprised to learn that the USDA considers soda a food, I also had gone through Washington’s State attempt to tax soda and candy in 2010. Defining what constitutes sugary food becomes challenging. Yes, we know what soda is, but where is the line drawn on other drinks containing sugar—like sports drinks and organic juicy drinks with added sugar? What about other sugary foods like Twinkies, Pop Tarts, and Fruit Loops? The voters rejected the tax in November 2010 for many reasons (I voted for it), but this definitional complexity stayed with me.

Thirdly, I am immediately suspicious of the assertion that soda causes obesity because I have rarely found a single cause explains any complex problem. My hunch is that it is likely a contributing factor, but it probably just one voice in a pretty large choir. If that is the case, than why target soda?  That is what the beverage lobby asks as well—so let me be clear: I am not on their payroll. It just seems to me to be a fair question to ask whenever anyone claims a single cause for a complex problem.

Lastly, I am highly suspect of any policy fix that focuses solely on the poor unless they are the only group being impacted.  Bittman does some fancy wordsmithing but he presents no convincing evidence. I wonder two things: 1) is this really a problem?, and 2) is this policy solution one of those symbolic actions has no chance of making any difference because there is no real problem?

After I look at my gut reaction, I then begin to look for background information that will help me understand the issue and the proposed policy solution.  Then it is on to seeing what data sources  already out there that document the extent of the problem and demonstrate a causal relationship that highlights root causes. I also like to have information about a wide array of options and some data that might help predict which options might have the greatest impact.

If the goal of a public policy is to make a positive impact, then it becomes important to fully understand the issue and to seek solutions that focus on root causes rather than on symptoms.

Using the Research Process

I am not an expert in nutrition or nutrition science, so there is a great deal I do not know. I begin, however, with a researcher’s mind that focuses on the basic elements of good research. I still might have to call in the experts but I will have a better idea of the key questions I need to ask as opposed to “tell me everything you know.”

Start at the beginning: Defining the Terms

 What is obesity and how is it measured?

It is something we think we know, but as a researcher, I want to know exactly how it is measured. How is normal weight, overweight, and obese defined? More specifically, is it really about weight or about excess fat? Next, I want know whether the measure is valid, that is, whether it really does a good of measuring obesity. I look for disagreement, and it typically does not take long to find out whether there is some debate about the measure. Sometimes a measure is flawed but there is general agreement to use it anyway because there is no perfect measure. But knowing the arguments can help gauge whether it is a valid measure.

Then I want to know whether the measure is stable over time. A reliable measure is measured in exactly the way can and every time. Flawed though the measure might be, it does provide a degree of reliability when analyzing trend data over time: is obesity increasing or decreasing over time. If the measures change, then the comparisons may not be useful.

Lastly, I tend to look for possible distortions in how data is summarized and reported. For example, by combining overweight and obese categories, the numbers will be higher, and possibly more dramatic. I prefer to see the distinct categories over time, even difference in the “obese” category.

 How is soda defined?

Is it soda with added sugar or added high fructose corn syrup? What about no-or-low calorie sugar substitutes? Should it just be about fizzy? What about sports drinks and other beverages with added sugar?

What do we know about the prevalence of obesity?

I like to have a snapshot of the number and percent of adults and children in terms of weight and measures of obesity at the most recent point in time, as well as the trend over time. This gives me a picture of the prevalence and a better gauge of the seriousness of the problem. Of course, this data will be impacted by any definition changes.

Are poor people more likely to be obese than those that are not poor?

Because Bittman’s policy solution focuses on people receiving food stamps, I will look for more in-depth analysis. What do we know about the prevalence of obesity among people on food stamps (or poor or low income people) as compared to those not on food stamps (or non-poor people). If obesity were not any different among the poor as compared to the non-poor, then I would pretty much dismiss his policy recommendation.

However, if the poor were more likely to be obese than the non-poor, I would then look to find out what researchers know about the impact of soda as compared to other factors. Bittman quotes an expert worrying about poor people buying junk food, but that does not constitute evidence.  What I want to know is the extent to which, if at all, people on food stamps actually by junk food, or more specifically here, sugary soda.  Bittman acknowledges that there is no data available, but quotes another expert who offers an opinion that is not backed by any published research. I know. I am a tough policy customer and an advocate would need something more than persuasive writing and negative stereotypes to convince me. If they are not drinking soda, then prohibiting it is a moot point.

I bring a bit more experience to the Food Stamp angle. The average Food Stamp benefit for a family of 3 in 2010 was $401. That is $133 per person for the month, or about $4.40 per day per person. I know I spend more than $4.00 per day on food for myself, not including eating out. How much food does that actually buy? I used to ask students to live on the equivalent of the maximum amount of food stamps for their family size for two weeks as a research project.  They kept a record of what they bought and I don’t recall seeing soda. I did see things like milk, eggs, bread, cereal, cookies, juice drinks, and peanut butter along with the fixings for inexpensive meals with canned vegetables, rice, beans, pasta, potato chips, ground beef, chicken, and hotdogs. Some nutritionists would suggest that this kind of high-carb diet would be filling but that it would also cause excess weight gain.

Exploring Causality

Embedded in Bittman’s argument are two causal beliefs. One is the soda causes obesity and the second is that obesity causes diseases, such as diabetes.

Several conditions have to be met in order to demonstrate causality. First, there has to be a reasonable theory to explain the relationship. The argument is made that soda provides empty calories and significantly increases calories in the daily diet. Generally, the research I’ve read points to the fact that consumption of soda has increased over time, as has obesity. Whether it is soda per se or the introduction of high fructose corn syrup, which is the main ingredient (after water), there appears to be a reasonable theory that can be tested. Secondly, there needs to be a clear time order: the causal factor (soda consumption) must happen before the effect (excess weight). Third, there has to be co-variation in variables: as the number of sodas consumed increases, weight will increase; and, as weight increases, diabetes will increase.

Lastly, to demonstrate causality between soda and obesity, and obesity and diabetes, all other possible explanations for the observed change have to be ruled out. Clearly, people are eating other things, so how do we know it is soda and not pasta, pastry, candy, fats, growth hormones in meat and dairy, or other chemicals in our environment that are no contributing to excess weight? Or maybe it is lack of exercise? There would little positive impact of excluding soda if another food or cluster of other factors were guilty of excess weight gain.

When looking at statistics, I would be looking at the measures of association to determine the strength of the relationship.

Causality is hard to demonstrate. It is very difficult to determine the single causal factor when people live in a world where many factors that affect their nutrition and health. And while some experiments can be done, not many will agree to do as Morgan Spurlock did in his 2004 documentary Super Size Me: to eat a diet he believed would cause him harm. Ethics are an issue.

As I began reading various studies and books, (and yes, I have reached the point of overwhelm fairly quickly), it struck me that the time-order can be questioned. Does soda cause obesity and diabetes, or is there an endocrine/metabolic dysfunction that first appears as a craving for sweets and rapid fat gain, perhaps accompanied by a diminished signal for satiety? Maybe instead of assuming that people are sitting around eating excessive sweets and calories because they lack will power, we should consider the possibility that these are symptoms of a bio-chemical malfunction that may manifest later on as other diseases? So, as I begin to look at the available research, I would be seeing if anyone has tried to test this alternative hypothesis.

 Excessive Weight: A Public Policy Issue?

The reason excess weight becomes a public policy issue is because it is believed to cause health problems, such as heart disease, hypertension, and diabetes. Aside from our concern about the well-being of people, there is an economic cost to treating these diseases. While the estimates may be less than precise, the cost argument will likely be part of the argument for intervention.

To develop the most effective public policy, I would be looking at how good the research is in demonstrating that causal relationships exist.

If obesity is a problem that does contribute to an increase in specific diseases, then my questions shift. Because causality is difficult to determine, I tend to start with seeing what research tells us about the likely contributing factors and situations are associated with obesity. Here we have to look at the vast array of research. Soda, sugar, high fructose corn syrup, carbohydrates, caloric intake, metabolic dysfunctions, exercise, etc etc etc.

When there is such a substantial body of research, decision-makers might consider putting together an expert panel to review the data and make recommendations. Because of the complexity of this issue, the experts should reflect a wide range of professional/academic disciplines and a wide range of traditional and alternative views on nutrition, medicine, disease and public health practices.

 What are effective policy interventions?

Policy analysts would have to look at research testing various interventions to determine which ones worked: What has been tried in the past? What has worked, what has not, and why? Could interventions in one community work in other places? In the policy arena, if you have not come up with 5 possible options, you have not thought hard enough. As policy options are considered, it is necessary to consider the likely sources of support and opposition to each option, and the likely arguments. This information helps make choices as well as develop a strategy for success.

 The Way Forward

While I am skeptical of Bittman’s proposal, it does not mean he is incorrect. A short opinion piece is not going to answer all my questions. It is also true that it is sometimes less important to get it right then to brink an in issue into the public in order to provoke public awareness and debate. Ideally, the debate will bring forth more, and hopefully better, information to guide policy interventions that will be effective.

It is also useful to keep in mind that assertions, no matter how loudly stated or repeated, are not necessarily true. Research can be used to separate fact from fiction, but sometimes it does not work that way. It is often weak in untangling complexity, but untangling is a must in order to determine causality. It takes time to conduct enough research to develop a sufficient body of evidence where the pattern becomes clear and rival explanations are eliminated.

Research is not designed to prove the truth, although it can sometimes help dismiss the noise and nonsense that surrounds policy arguments. At its best, over time, it can provide a preponderance of evidence that can help public administrators and policy makers make reasoned choices. It is also true that once research gets caught in political debates, distortions can arise. The more controversial the policy issue and the bigger the stakes, the more important it becomes to look at the issue dispassionately as a sophisticated user of research results.

Stay tuned for future blog posts where I present what I have learned about sugar, obesity and disease.