Phase 2 Data

Chocolate chip refrigerator cookies from a tube

Guided Activities
The audience is being asked to evaluate the quality of the Phase One results. This process involved sharing the results of the pre-surveys as well and the accumulated Phase One Data and my analysis of it.

Due to the changes in the project much of the data from the surveys was not very useful in developing the Phase Two Activities. It did lead to very interesting discussion with the students about epistemology (i.e. how they know what they know) is broadly relevant and did serve the purpose of creating a thoughtful and analytical mind-set for the dialogue.

The experimental results of the first round of taste tests continues to be confounding for them, as the concept of negative results providing positive data can be counter-intuitive. But it also is motivating their interest in repeating the experiment.

They have decided to do a non-blindfolded taste test of Chocolate Chip refrigerator cookies, and to introduce some additional sets of controls into the cooking process as referenced on the Phase One data page. Many of their suggestions will prove to be impractical or ineffective, but we will let them do their best at making them work.

Questions about the quality of the sampling process have also been prominent in their discussions. They found the blindfolds very difficult to deal with, likely mostly due to their autism and the related sensory issues. All were very insistent that the second round testing should be done without blind folds.

We have prompted them to continue to reflect on the uses of scientific inquiry in non natural history questions. During this phase they seemed to be much more able to address the taste test in a scientific context. The process of evaluating the first phase results and looking for ways to control the variables was active and most of the students participated.

Target Audience Solution
To answer the question; “what is the class’s preferred brand of refrigerator chocolate-chip cookie?” they understood following the inconclusiveness of the pahse one testing that they needed to first answer the ancillary question of how to best prepare the cookies for a fair test.

The solution produced in this phase was much more successful than the previous phase due to increased control of variables. Students have been much better able to understand the connection between data and results since the results were actually significant this time.

Summary of the CBL

Phase two testing began with a discussion reiterating the results we had achieved in the first phase of Sugar Cookie testing and the confounding factors of uncontrolled and unquantified variables. Various suggestions were reviewed and new ones developed, and a revised methodology was agreed upon.

Preparations for the second round of experiments were made as a group, but this time the actual cookie baking was not done at school, with all the concomitant complications of using the shared oven with its poor control. Their teacher, my sister and I baked the cookies at home under much more highly controlled conditions. The taste test took place the following day in the classroom, and the analysis and discussion of results the day after that.

I am gratified that we are making progress toward a better understanding of the process of scientific inquiry. This is a difficult population with some contributing too little and others too much. It took many prompts to get many of them involved, but some, once engaged had good suggestions which revealed excellent comprehension of the concepts.

This process of inviting student involvement creates a higher level of engagement as they feel involved and committed to the process. Whereas in a typical instructional delivery students often lose interest in the process.

Their immediate response was one of skepticism, but once we began adjusting the methodology to make it more rigorous, more of them became more analytical and exhibited more higher-order thinking.Some of their suggestions involved making inferences from the cookie ingredients. Also that we could contact the manufacturers who would give us information. Kevin made the  suggestion that we should go online to research the ingredients in the cookies to find out what effect they had on taste, texture and color.

These suggestions revealed the prevalence of the common fallacy in scientific thinking of Revelation from Authority. Science teaching too often involves offering facts based on faith in a particular authority. But, we do not know how gravity works because Newton says so, but because of the evidence he and others have collected. Scientific inquiry needs to be skeptical; it is de rigueur that all findings, regardless of the credentials of the researcher, be doubted and then tested and retested.

If our research were to uncover one authority claiming that ingredient X results in a poor texture, and another that it results in a good texture, the only reliable way to tell which was correct, in the context of our cookies within our population, would be to do testing, obviating any usefulness of the authority. In fact, the opinion of an authority might even result in increased bias. More dangerous would be if one’s research uncovered only authorities who were in agreement, which would bias opinion even more severely, and would seem to obviate the need for testing.

This idea of revealed knowledge is the exact opposite of the process of authentic scientific inquiry. This has led to the too common and false premise that science is just one belief system among many, something that this hands-on, challenge-based approach is intended to address.

In open dialogue the results of phase one were evaluated and a methodology was agreed upon by the group. Cookies were baked and then the audience tasted the cookies and recorded their responses, which were then aggregated.

Group discussion and reflection resulted in modification of the phase one methodology. Caleb suggested that we could use a color chart to compare the cookies to, a process we implemented, also that we could squeeze the cookies to judge their crispiness, and so we also decided to include survey questions of these variables. We prepared a new evaluation form based on these discussions.

Chocolate Cookie Form
Chocolate chip cookie taste test form

Another issue that was discussed was the consistency of the baking time and oven temperature, and they suggested that we keep records and make measurements with a thermometer and a timer to be able to take these variables into account. An oven thermometer was purchased and timer software: Apimac Timer was downloaded and installed. They also asked that we make sure that we were following the instructions on the package.

After the design specification of this form for the variables we wished to record, and final description of the test methodology, I bought 4 different brands of prepared cookie dough with the following baking instructions.

  1. Pillsbury Chocolate-Flavored Chips. “350 deg. for 10-14 min or until golden brown.”
  2. Nestle Toll House. “350 deg. for 12-13 min.”
  3. Pillsbury Simply. “350 deg. for 12-16 or until edges are golden brown.
  4. Immaculate Baking Co. – Gluten-Free. 325 deg. for 16-18 min.

To remove as many variables as possible we tried to control as many factors as we could. All cookies were made by weighing out one ounce balls of dough that were rolled by hand into spheres and placed in the same position, six to a tray, for twelve total of each variety, cooked in two batches. The two trays that were used in rotation were of the same type and manufacturer. All cookies were baked with only one tray in the oven at a time, sideways on the middle shelf, with 6 cookies in each batch. We recorded the temperature when the cookies were put in and when they were removed and we timed the baking with the timer.

Each batch was cooled for 5 min. and then put onto a separate paper plate and sealed in a labeled ziplock bag. below are the records of the baking times and temperatures.

bake times
Baking temperatures and times

They shared their cookies and results with other students in the G3 program, and the other teachers present were excited to see such an active process. The students involved appeared to be excited, especially at the prospect of eating cookies in class for science.

Data Collection and Evaluation



The most popular brand by sales and availability, Nestle Toll House, received the lowest scores in our test. One student commented that he preferred the taste of the dough in the A cookie but the chocolate in the C cookie. It was a surprise that the Gluten-Free cookie came in second, and that it was judged as best by some of the students. It was also judged to be both the darkest and the chewiest.

Other results were not as anticipated. It had been assumed in the formation of the hypothesis and methodology that darker would equate to more crisp and that those who preferred crisp would prefer the slightly crisper batches in each variety, but there was less correlation here than expected.

The unusual composition of the gluten-free variety had led to the assumption by some of them that this cookie would be “yucky”, but only one of the eight respondents judged it the worst. This is especially interesting and pertinent in that there is a common hypothesis in the autistic community that gluten allergies are one of the underlying causes of autism, and so many children with autism have been put on gluten-free diets by their parents.

Some of the final reflections we heard from the students were:
Kevin: “It was fun.”
Ana: “I really love cookies.”

Caleb: “Now I know which cookies to buy for Kevin.”
Alan: “I don’t know about that but I know that e=mc squared.”
Jacob: “A hypothesis is like a guess.”

We discovered that a fun activity can still engage the students in problem-solving skills. Some of their reflections during this second phase changed to become somewhat more comprehending of the scientific method. Listening to student reactions helps to focus our activities in ways that they find appealing. Collecting data lets us reflect on the process in a way that is not affected by faulty memories or bias.