SC 130 Physical Science assessment report and grid

10 May 2010

College mandated assessment report format

Review of performance: SC 130 Physical science spring 2010. 29 students enrolled in course. Submitted by Dana Lee Ling

For the first three outcomes, student counts based on number who passed the class. For subsequent outcomes, student counts based on submission of laboratory reports for that outcome.

SC 130 Physical science assessment

Some files linked from this assessment grid use XHTML+ MathMl + SVG and require the use of browsers such as FireFox which can render these technologies. Sample laboratory reports are in Adobe Acrobat (.pdf) format.

Program Learning Outcomes

The relevant program learning outcomes are from the general education core.

• Define and explain the concepts, principles, and theories of a field of science.
• Perform experiments that gather scientific information and to utilize, interpret, and explain the results of experiments and field work in a field of science

Course Learning Outcomes

Learning themes cut across all activities and explorations in the course. These learning themes are the first three items on the course outline. The first theme centers the course on exploration and science as a process as expressed by the scientific method.

The second theme places mathematics at the core of the course. Many of the laboratories lead to linear relationships, linear regressions, with the attendant concepts of slope and intercept.

The third theme brings writing across the curriculum into the course. In addition to being marked for content, the laboratories are marked for grammar, syntax, vocabulary, spelling, format, cohesion, and organization.

The laboratories and the laboratory reports are the primary evidence that students explored physical science systems using scientific methodologies, that they generated mathematical models, and that they communicated their results.

Students will be able to...

Specific learning is centered on the laboratory experiences. Laboratory reports cited above are considered a primary measure of student performance in the course. Through the weekly laboratory reports, the course integrates writing into the core of the course curriculum.

Documenting actual activity during the laboratories is difficult. The laboratories are intended and designed to engender cooperative learning in scientific teams of exploration. One of the design intents is that the acquisition of scientific knowledge is a journey, not a destination. Science is not about a set of accumulated and memorized facts. Science is a process of discovery, careful thought and analysis. Science is about finding and testing explanations for systems, in physical science those explanations are typically mathematical models.

Tests can document acquired facts. Documenting the journey, as opposed to the acquired facts, is difficult. The table further below uses links to photo documentation as indirect evidence of science as an exploration.

The final includes single numeric problems typifying that area of study. Each final utilized a spreadsheet to randomly generate data values. A mail merge was used to produce individually unique final examinations for each and every student. The merge also generated an answer sheet for each student final, these were separated from the final and used to mark each unique exam paper.

The results of an item analysis for the spring 2008, fall 2008, spring 2009, fall 2009, and spring 2010 final are in the table further below. The item analysis is the percent of students answering the question in that area correctly.

Specific Learning			Final item analysis
Outcomes	Laboratory	Photo documentation	Sp 2008 n = 29	Fall 2008 n = 28	Sp 2009 n = 31	Fall 2009 n = 32	Sp 2010 n = 29
Explore dynamics of motion including performing calculations of velocity, acceleration, momentum, and kinetic energy, generating appropriate mathematical models such as linear regressions, making calculations of the conservation of momentum and energy	Linear	Rolling balls	0.62	0.89	0.61	0.56	0.93
	Acceleration	Falling balls	0.62	0.39	0.45	0.03	0.14
	Momentum	Marbles	0.59	0.57	0.48	0.31	0.62
	Force	Hooke's Law					0.86
Experiment with and determine the heat and electrical conductivity of materials	Heat	Conduction	0.90	0.86	0.81	0.91	0.90
Determine latitude, longitude, and find the mathematical relationship with standard linear measures; determine universal time	Lat Long	Lat Long	0.17	0.36	0.23	0.75	0.83
Observe and identify clouds, be able to describe precipitation processes in Micronesia such as collision-coalescence, Bergeron, and orographic precipitation; list the phenomenon associated with El Niño and La Niña	Clouds	Cloud formation and shape	0.48	0.79	0.61	0.22	0.48
Determine the speed of sound and perform experiments with sound	Sound	Echoes	0.21	0.14	0.48	0.22	0.45
Explore reflection and refraction, determining the mathematical relationships for reflected image depths, angles, and refracted image angles	Optics	Optics	0.72	0.36	0.65	0.53	0.80
List the primary and secondary colors of light, generate other colors from primary colors, explore systems of specifying colors	Colors of light	Spectra	0.34	0.36	0.29	0.61	0.28
Develop a mathematical model using measurements of current versus voltage across a resistance; determine and sketch open, short, and closed circuits	Electricity	Circuits	0.72	0.64	0.71	0.50	0.59
Determine whether substances are acids or bases using locally available pH indicator solutions	Chemistry	Acids and bases	0.52	0.86	0.61	0.72	0.66
Average:			0.54	0.57	0.55	0.49	0.65

Photo documentation for the above and other activities

Comments and notes spring 2010

The core of the course are the activities and laboratories. The laboratories involve a write-up using spreadsheet and word processing software. The laboratories are marked using a rubric. The course focuses on physical science as a process and method, an exploration in search of mathematical models of system behavior. The final examination is not a well aligned measure of process, method, and exploration. The final examination spring 2010 was a set of fifteen questions, one per laboratory, usually centered on the central mathematical relationship of the corresponding laboratory. As such, the final is an exercise both in remembered knowledge and calculations.

Spring 2010 the laboratories ranged from 20 to 60 points with an average of 44.62 points. With quizzes, tests, homework, and attendance, the course generated 949 points. The final was worth 36 points, less than 4% of the overall mark. While the final could be weighted, the true focus of the course are the laboratories. Laboratory 14 is a better measure of the achievement of outcomes than the final examination. The students are aware that the final has little impact on their grade, and this is reflected in the performance seen in the table above.

In the fall of 2009 I changed both the pretest review and the content of the final examination. In fall 2009 I deleted all of the formulas from the final examination. To prepare the students for the greater memorization load that this represented, I ran off a randomly generated copy of the final examination and handed it out on the last day of class. Thus the students knew exactly what questions would be on the final examination, and what formulas they would need to have memorized, only the numeric values would remain unknown.

In December 2009 one student told me after the test, "I did not study, no need." The students knew the final could not make a significance difference. The combination of the increased memorization component and the comprehension that the final could not move their grade significantly apparently underlie the drop in performance on the final examination.

Although I do not intend to shift the focus of the course from the laboratories, I did rethink how the final might be better handled spring term 2010. As memorization has never been an intent of the course, I added a sheet of formulas on a separate sheet of paper. The sheet had all of the formulas from the term, thus students still had to know which formulas to use. I also did not mention the relatively low impact of the final examination on their grade. I did hand out a randomly generated copy of the fall 2009 examination. Although some questions remained similar, some questions were changed.

Five questions on the final required only the recalling of a single fact. Six questions involved making a single calculation. Four questions required a chain of more than one calculation and are termed complex in the difficulty table below. Bear in mind that the formula sheet included all formulas encountered during the term, students still had to select the correct formula. The formula sheet was written in variable format only, for example, the formula for the acceleration of gravity was given as d = ½gt².

Fact based recall fell from the prior term despite the students being aware of what would likely be on the final examination.

The ability to calculate results involving a remembered formula and a single calculation had remained remarkably stable term-on-term until fall 2009. The addition of a formula sheet may entirely explain the spring 2010 performance improvement.

Complex calculations, while difficult for the students, also showed improvement. Again, the formula sheet provided spring 2010 may explain this improvement.

Comment on linear regressions

Spring 2008 the laboratories were rewritten to focus on mathematical relationships, specifically linear relationships. Students are introduced to using spreadsheet software in laboratories one and two to assist them in making xy scatter graphs and in finding the slope and intercept. The course requires access to a computer laboratory for the second half of the first two laboratories during a term in order to present this material to the students. An experiment will be attempted summer 2010 to explore keeping this focus without using the computer laboratory. The intent during summer 2010 will be to use calculators that can perform linear regressions, although not graphing calculators.

One might think that given the placement of linear relations at the core of the course, the course should have a math pre-requisite. The lack of a math pre-requisite is intentional. The course does not presume the student has anything more than high school level contact with algebra one material. The course not only undertakes to teach students to run linear regressions, the instructor presumes that using a computer to find slopes and intercepts is wholly new material for the students. The course intentionally seeks to introduce the algebra of linear equations to the students through the vehicle of physical science. The course also introduces non-linear mathematical relationships.

Analysis of mechanics of writing

Each laboratory is marked in four areas, grammar, vocabulary, organization, and cohesion. A rubric is used to mark these areas. The rubric is similar to the rubric used by the college to mark entrance test essays. Modifications include the addition of spelling to the vocabulary section and the organization section being based on the laboratory including all required sections.

Spring 2010 the scores for grammar, vocabulary, organization, and cohesion were compared for laboratory one and laboratory thirteen. There were statistically significant gains seen in all of the four areas. The gain, however, was less than one point in each area. The complication is that the students write fairly well as measured by that rubric at the start of the term. Admission to the college usually means that a student can score a four or five on the essay rubric. Put another way, there simply is not a lot of upside room on that rubric.

When surveyed late in the term, the students self-reported that they felt their writing skills had improved during the term.

An analysis of procedures, reasoning, strategies, communication, and concepts

A study done as part of a report on the general education program learning outcomes applied a rubric to ten laboratory reports from the SC 130 Physical Science course. In total, 61 laboratory reports were examined from a number of different courses at the college across five sites. The rubric is a four point rubric. Two graders looked at each laboratory report, their scores were added for a total possible of eight points for each factor in the rubric. Thus scores in the table below can be taken as being out of eight possible points.

Liked and disliked laboratories

A survey (results) was run spring 2010 to determine the most liked and most disliked laboratories. The students were also asked why they liked or disliked that particular laboratory. The following table presents the results in descending vote order for most liked and most disliked laboratories.

Possible the core finding is the top three most liked laboratories are also the top three most disliked laboratories. This result leaves no obviously resoundingly disliked laboratory which should be removed from the course. The specific comments, however, provide opportunities to find ways to improve the disliked laboratories.

Laboratory 14: Final practical laboratory

In the past the students were provided with the outline for performing the final practical laboratory. This term the students were given a pile of small, square glass panes and the question, "Determine the index of refraction of the glass." The students were given nothing else. They were told they could ask for equipment and if available, the instructor would provide that equipment. The majority of the students opted for a simple and direct measurement based on the manner in which they determined the index of refraction for water in laboratory ten. A few tried some unique approaches to the problem, none of which were likely to result in a correct result.

All of the students asked for rulers or meter sticks, some asked for wood blocks, one asked for a basin to hold water. These requests were granted.

• 25 of 28 students a procedure that could correctly yield the index value.
• 9 of 28 students obtained an experimentally correct result for the index of refraction.
• 5 of 28 studentslooked up and cited the correct theoretic value for the index.
• 3 of 28 students correctly calculated the percentage error for their experimental value.

The final practical laboratory is a true challenge for the student. Given no guidance, no directions, not even a recommended procedure, the students have to approach a new system that has some similarities and many differences from a system they have previously explored. This laboratory really pushes the student and allows insight into the ability of the student to explore a new system in a scientifically rigorous manner.

RipStik Physics

During spring 2009 my son learned to ride a derivation of a skateboard called a RipStik. He managed to extract from his mother a promise to buy him one during a summer trip to the states summer 2009. During the fall of 2009 closer observation suggested that the device might be useful for demonstrating a number of concepts in physical science. At that time a request for the acquisition of one by the division was made and granted. During the December 2009 between-term break I learned to operate the RipStik, although not without some serious falls and minor injuries.

The RipStik was then used as a demonstrator for linear motion, accelerated motion, potential and kinetic energy, forces, and wavelength, frequency, period. A quiz on the latermost material was based on the RipStik sine wave. The RipStik functioned both as a way to demonstrate basic physical relationships and to catch the attention of the students. Each demonstration drew their undivided attention. Learning begins with focusing on a system, and the RipStik garnered that focus.

Physci current term • Physci • Assessment • Courses • COMFSM