Phase 1 Data

Holly Days Cookie Taste Test Experiment

Guided Activities
We began in an open process where I aided the teachers in addressing the process of scientific inquiry, and ways to use it. Various questions and possible foci of this process were solicited from the students. They asked about quantum mechanics and string theory and other more encompassing questions about the nature of the universe, but we explained that science was just the process that a physicist might use to find answers to such questions, but that in our environment, would not be able to be address such questions scientifically.

We continued to solicit potential subject inquiries with little success, having most students requesting that we give them “scientific facts” rather than suggesting ways they could apply the process themselves. This is a typical response and precisely the common fallacy that this project is trying to address.

I introduced them to the idea of solving non-scientific problems with a scientific methodology, but they had trouble relating to the concept, as for most of them, science instruction had always been done in the context of learning natural history facts.

An aide happened to interrupt our session to inquire about specifics of their tradition in this G3 home room of making cookies to share with other classes.

I seized on this occasion and their process of trying to decide what kinds of cookies they should make as an excellent question to which they could apply the scientific method. We decided to mount a Cookie Taste Test Experiment where they would bake a variety of sugar cookies and then record their own preferences as well as those of other faculty and students to decide which brand of cookies they should prepare to distribute at their Holly Day open house.

Recording Preferences

Target Audience Solution
Our main focus of discussion was the practice of scientific inquiry. We considered how we can answer questions in ways that reduce our bias and do not replicate our expectations.

To achieve this for this test solution, we acquired 5 different varieties of sugar cookies which we baked and then did blind taste tests of with the students and teachers in the target group as well as a number of their friends. We provided each with this form:

taste test form
Preference Form

Each tester used the above form to record their preferences of each cookie by ranking them 5 for their favorite, to 1 for their least favorite. The results from each form were tallied in the spreadsheet shown below, with the total number of points received by each variety shown as bold totals in the bottom row.

Taste Test Results Round 1
Taste Test Results Round 1

We discussed how we might interpret this data. They pointed out that they had liked some of the cookies a lot and some not at all, and were disappointed that their favorite variety had not “won”. But the similarity of the cumulative popularity of each variety was in contrast to the strength of the preferences that any one tester reported, therefore we pointed out that the results could not be interpreted to mean that similar results meant that all the cookies tasted similarly. We asked what could cause these variations in real taste differences to result in such even results, and they suggested that “some people like one and some another”, but we wondered is with this many people, how likely it would be that their preferences would cancel each other out so perfectly.

The students wanted to interpret these non-significant results to suit their own expected preferences, but during this discussion about why the data should be so random, two students shared their observation that the cookies in each batch had some that were crisp and some that were soft. Others reported that their preference was based mostly on how crisp or chewy the cookies were, with some preferring the crisp and others the chewy.

Their first hypothesis about this was that the cooking times had been different, but we had been careful to time them carefully, and all of the batches but the first had a similar range of crispness. The fact that the oven may not have been fully heated with the first batch was agreed upon as the most likely reason for the first batch being less crisp. Further, the most involved bakers reported that this variable of crispness seemed to vary depending on which shelf in the oven the cookies had been on while baking, and where on the cookie sheet they had been placed; that ones from the front of the trays and lower shelf of the oven tended to be less crisp, and the ones on the top shelf and in the back of the tray tended to be more crisp, with the top back shelf being the most and the lower front shelf being the least.

They felt discouraged that they had not gotten any results they found to be conclusive, and decided, in the absence of any usable taste preference data, to buy more of the cookies with the Christmas Trees on them for their party as they agreed that these were the prettiest.

Summary of the Challenge-based approach
I work in a post secondary environment, where I have always taught in a non-typical, constructivist way. My sister’s class is special education, and due to my expectations, I had trouble both with getting some students to participate, and with others participating too chaotically.

It was a challenge to get them to connect with real inquiry, and it required more prompting and explaining than I am used to. This time I was not able to let them use individual discovery as much as I would like due to the nature of the population. Some tended to perseverate on details that were insignificant, others had trouble understanding the subtlety of the concept of open inquiry.

Their initial response to my asking for them to come up with science questions was to ask unanswered questions about natural history that they were particularly curious about.  I pointed out that we could not answer these sorts questions in the classroom environment, but that they should try to think of simple everyday life questions they could answer. They questioned whether these sorts of questions were even within the domain of science, and I explained that science was just a process by which we answer questions.

About halfway through the actual implementation, to which we were applying a fairly high level of rigor, they all understood that what they were doing was indeed science. As opposed to solving string theory equations. Growing hydroponic food. Exploring outer space. Writing a video game.

We asked how they thought the process should proceed, and they suggested that we should test chocolate chip cookies against the Christmas tree cookies. We pointed out that the difference in taste and texture would not allow for a valid comparison that would be suited to such a test, and they agreed that we should use the various different patterns available in the same brand of cookie.

We asked what might prevent them being objective about their preferences. They suggested that they should wear blindfolds so that their disagreement about the Christmas tree variety would not affect their choice.

They created a form, baked cookies, did blindfolded taste evaluations which they recorded and then analyzed. Thet offered cookies to other non-blindfolded students and surveyed them on their preferences.

They have shared their Christmas Tree cookies they chose with the other classes at their annual Holly Days open house. They have not yet shared their experimental results as they were not conclusive, and want to wait to report until after phase two to make final conclusions.

The problems with the baking methodology and the diverse nature of the population became apparent fairly quickly. Most reflection was to deny the flaws in the methodology and try to describe a definitive solution, and claim “success”.

Data Collection
1.Pre/Post Survey Reflection:

One conclusion we took from our surveys was that students need to have a better idea of what inquiry is. The issue of the communication skills of this population becomes more apparent in this, and so in general these results are not very useful. My the alignment of this survey, which was directed at addressing the epistemological orientation and their preferred approach to science teaching, is generally beyond the ability of this population.

I have been modifying the survey approach based on the poor responses to the survey and requirements of this program. I had hoped first, to engage them and then to discover some patterns of understanding in their answers to the vampire and zombie questions. I was looking for a correlation between “There is no such thing”, and “They are already dead”, as rationalist. One third of them denied the existence, and that same third also responded they are dead. But there are too few responses and the population is too small and idiosyncratic to be useful to make judgments about the generally epistemological orientation of these students, if this was possible at all for these questions.

They did not respond to the questions about science education techniques. These results lead me to plan to completely revamp the existing and not yet deployed post survey to narrow it to reference only the single experiment we are conducting. We will meet to discuss way to improve the next round of testing, and then include questions about that process of revision as well as their evaluation of the results of the experiment in the new post survey. It is enlightening to see how interest most of them have in the process despite their strong motivation and descriptions of scientific curiosity.

These are some examples of their responses to the question: “What is science?”
Caleb: “A name of a subject for class.”
Matthew: “Science is a study of the things that are happening.”
Michael: “Science is a subject that you learn in school. It’s about the earth, life and physics.

We are going to run another set of tests using chocolate chip cookies, so that they will not be swayed by the patterns and not need blindfolds, which did indeed cause them to become anxious and confusedThey decided we should sort the cookies somehow by crispness or color so we only compare chewy to chewy and crispy to crispy.

Suggestions were: to build a cookie breaking machine that could measure how much force it takes to break a cookie, having a teacher sort them by color, cooking them in very small batches.