College of Micronesia-FSM

Student Learning Outcome Assessment Plan
for instructional affairs

Vision Statement

The College of Micronesia-FSM will assist the citizens of the Federated States of Micronesia to be well-educated, prosperous, globally-connected, accountable, healthy and live in harmony with the environment and the world community.

Mission Statement

Historically diverse, uniquely Micronesian and globally connected, the College of Micronesia-FSM is a continuously improving and student centered institute of higher education. The college is committed to assisting in the developing of the nation by providing academic, career and technical educational opportunity for learners in the Federated States of Micronesia.

Values Statement

1. Learner-centeredness - Learners are our primary focus and we provide quality instruction and services in a nurturing and safe environment.
2. Professional behavior - We are competent, service-oriented professionals with a commitment to life-long learning and a commitment to provide excellent and exemplary service to students, colleagues and the community.
3. Innovation - We provide a dynamic, creative, up-to-date, and innovative environment to allow the college community to function effectively in a global economy.
4. Honesty and Ethical Behavior- We are honest and abide by the COM-FSM Code of Ethics in all our personal and professional interactions to create and maintain trust and unity among ourselves and with our community.
5. Commitment and Hard Work - We commit and invest our time, energy and resources to create a rigorous, high quality learning environment.
6. Teamwork - We live in a community where collaboration, open-mindedness, respect and support for each other help us achieve our mission.
7. Accountability - We are responsible for and accountable in our daily activities to our partners and the community we serve. We comply with all applicable regulations and use our resources efficiently and effectively to maintain a high level of trust and confidence.

Goals

The College of Micronesia-FSM, through a continuous assessment and review process, will continuously improve to meet or exceed current accreditation standards and will:

1. Promote learning and teaching for knowledge, skills, creativity, intellect, and the abilities to analyze new information and communicate effectively.
2. Have sufficient and well-managed fiscal resources that allows for financial independence.
3. Foster effective communication.
4. Invest in sufficient, qualified, and effective human resources.
5. Create an adequate, healthy and functional learning and working environment.
6. Build a partnering and service network for student success, and workforce and economic development.
7. Promote the uniqueness of our community, cultivate respect for individual differences, and champion diversity.

Goal one: Promote learning and teaching for knowledge, skills, creativity, intellect, and the abilities to analyze new information and communicate effectively.

Goal one is implemented by six institutional student learning outcomes (iSLOs), alternately conceivable as institutional instructional objectives.

iSLO#	iSLO Category	institutional Student Learning Outcome: Students will be able to...
1	knowledge	define, describe, demonstrate, and explain knowledge within a field of study.
2	skills	apply, use, perform, exhibit, and demonstrate skills required of a particular career or field of endeavor.
3	creativity	plan, design, develop, seek, find, synthesize, and create solutions, strategies, documents, and products.
4	intellect	exhibit the capacity for independent thought and critical thinking.
5	communication	communicate effectively through writing, speaking, performing, exhibiting, or other forms of expression.
6	analysis	acquire, interpret, analyze, assess, and evaluate information.

Mapping program student learning outcomes to iSLO

• Program student learning outcomes (pSLOs) were assigned to a single best fit iSLOs.
• Only the assignments for a sample of six general education program are shown.
• The assignment of all associate degree pSLOs to iSLOs is available in a separate spreadsheet.

iSLOcat	Program	Program student learning outcome: students will be able to...
knowledge	GenEd	Demonstrate the various elements of the writing process, including collecting information and formulating ideas, determining relationships, arranging sentences and paragraphs, establishing transitions, and revising what has been written.
analysis	GenEd	Read accurately and critically by asking pertinent questions about a text, by asking assumptions and implications, and by evaluating ideas.
communication	GenEd	Use the conventions of standard written English to write an organized, coherent, and effective essay.
creativity	GenEd	Define mathematical concepts, calculate quantities, estimate solutions, solve problems, represent and interpret mathematical information graphically, and communicate mathematical thoughts and ideas.
skills	GenEd	Perform experiments that gather scientific information and to utilize, interpret, and explain the results of experiments and field work in a field of science.
intellect	GenEd	Demonstrate an ability for independent thought and expression in at least one of the following five areas: the arts, ethical thought and moral values, historical analysis, language, or literature.

pSLOs versus iSLOs: matrix view

The program student learning outcomes (pSLOs) can be cross-tabulated against the institutional student learning outcomes. Only four of ninety-eight pSLOs from the actual matrix are shown.

		iSLO category
Program	pSLO	analysis	communication	creativity	intellect	knowledge	skills
GenEd	Define and explain the concepts, principles, and theories of a field of science.					1
	Define mathematical concepts, calculate quantities, estimate solutions, solve problems, represent and interpret mathematical information graphically, and communicate mathematical thoughts and ideas.			1
	Demonstrate a basic knowledge and understanding of Micronesian history from prehistory to the present.					1
	Demonstrate an ability for independent thought and expression in at least one of the following five areas: the arts, ethical thought and moral values, historical analysis, language, or literature.				1
// Other program learning outcomes //

Distribution of pSLOs among iSLOs

The pSLOs were mapped 1:1 to iSLOs and then tallied to determine the number of pSLOs for each iSLO category.

	iSLO category
	analysis	communication	creativity	intellect	knowledge	skills	Sums
Total number of pSLOs	8	6	6	3	52	23	98
Percentage	8%	6%	6%	3%	53%	23%	100%

Analysis of pSLO distribution

• The college focuses on knowledge and career or field specific skills. This is probably typical for post-secondary institutions.
• The college focuses few program level learning outcomes on analysis, creativity, and communication.
• At a program level the college makes little to no conscious effort to develop the abilities of independent thought or critical thinking among our students.

Structure of proposal has meta-meaning

The preceding analysis is possible because of the physical structure and design of this assessment model. That the structure itself produces useful data argues that the structure is a useful design.

Moving down the structure

The same system by which pSLOs were mapped to iSLOs can be done for course level student learning outcomes (cSLOs) and pSLOs. The LA/HCOP program student learning outcomes are shown in the table below. Note that the iSLO assignment is retained for reference in this table.

iSLOcat	pSLO#	pSLOcat	LA/HCOP program student learning outcomes: students will be able to:
knowledge	1	human body	Describe the structure, function, and basic pathologies of the human body.
communication	2	communicate	Communicate health, nutrition, and premedical information in both written and oral formats.
knowledge	3	career	Describe health care and allied professions.
knowledge	4	foundations	Demonstrate a foundation in basic biology, chemistry, microbiology, anatomy, nutrition, health, and physiology.
skills	5	teamwork	Work effectively in groups to solve human life sciences and health problems.
analysis	6	problem solve	Quantify and analyze human life sciences and health problems using analytical, statistical, and computer methods.
knowledge	7	information	Acquire and synthesize human life science, health, and nutrition information in a critical, scientific, and technologically advanced manner.

cSLOS versus pSLOs in LA/HCOP

For every course level student learning outcome in a course that is a part of the major requirements of the LA/HCOP degree, an LA/HCOP pSLO can be assigned. Four cSLOs, pulled from three courses, are shown below. A more complete listing is retained in a spreadsheet.

seq	pslo#	pSLOcat	course	cslo level	cSLO: Students will be able to:
1	4	foundations	SC 101	course	Describe what it means to be healthy.
2	4	foundations	SC 101	course	Explain the wellness continuum and its impact on Personal health.
115	1	human body	SC 120	specific	Discuss the advantages and disadvantages and efficacy of several different methods of contraception.
118	6	problem solve	SC 230	specific	Convert between Kelvin, Celsius, and Fahrenheit.

Note that both course student learning outcomes (general objectives or cSLOs) and specific student learning outcomes (specific objectives or sSLOs) are maintained in a single list. An alternative would be to separate the course outcomes from the specific and map the pSLOS against the cSLOs and, in a separate list, map the cSLOs against the sSLOs. This alternative option, however, does not yield as much information on how well the program student learning outcomes are being met.

cSLOS versus pSLOs in LA/HCOP: matrix view

The cSLOs can be cross-tabulated against the pSLOs. A couple outcomes are shown below. Below the outcomes is the tally of cSLOs against pSLOs for three courses in the major: SC 101, SC 120, and SC 230. This is a work in progress, the other courses in the major are yet to be mapped into the matrix.

			pSLOcat
course	cslo level	Students will be able to:	information	foundations	human body	problem solve
SC 101	course	Define the concept of total fitness and the essential characteristic and consideration of a physical fitness program.		1
SC 120	specific	List distinguishing characteristics and examples off: bacteria, protozoa, algae, fungi, the major groups of plants (e.g., nonvascular and vascular plants, seed plants, flowering plants), and the major animal phyla.		1
SC 230	Specific	Write the structural formulas of different types of hydrocarbons.	1
Sum for three courses			2	194	44	8

Why change goal one?

The board approved a goal one that read:

Promote learning and teaching for student success and satisfaction.

Goal one goes astray in curriculum committee...

When tasked with developing objectives to meet that goal, the curriculum committee began an effort that led to the creation of objectives as if on a tabula rasa. As shown this was a top-down effort. The work was led by a small subset of the curriculum committee. The effort was neither inclusive nor broadly supported.

The threat of two different assessment systems

The curriculum committee was also tasked with developing assessment mechanisms for this list of objectives. This began to head in the direction of measuring enrollment, retention...

In the meantime the college had yet to determine how to measure course and program student learning outcomes. The curriculum committee's "objectives" clearly needed to align with the learning outcomes effort. The whole "stack" needed to align from the bottom to the top, from learning in the classroom to the vision of the college.

Connecting the vision, mission, and values to learning outcomes

Two pieces were needed to connect the vision, mission, and values to the program student learning outcomes effort:



Institutional student learning outcomes were needed to connect the course and program student learning outcomes upwards towards the values, mission, and vision. A goal that reflected these institutional student learning outcomes, a goal that would logically and obviously connect and make sense in between the iSLOs and the values, mission, and vision, was also needed.

Developing the institutional learning outcomes

The effort to connect was made from the bottom up. All the program learning outcomes for associate degree programs were used to build categories. An initial set of around a dozen candidate categories was combined and aggregated to form the six categories proposed in the Student Learning Outcome Assessment Plan. Working up from the existing program student learning outcomes provided de facto inclusion of the many voices who shaped the pSLOs.

iSLO#	iSLO Category	institutional Student Learning Outcome: Students will be able to...
1	knowledge	define, describe, demonstrate, and explain knowledge within a field of study.
2	skills	apply, use, perform, exhibit, and demonstrate skills required of a particular career or field of endeavor.
3	creativity	plan, design, develop, seek, find, synthesize, and create solutions, strategies, documents, and products.
4	intellect	exhibit the capacity for independent thought and critical thinking.
5	communication	communicate effectively through writing, speaking, performing, exhibiting, or other forms of expression.
6	analysis	acquire, interpret, analyze, assess, and evaluate information.

Developing the new goal one

The new institutional student learning outcomes were then aggregated to form a new goal that would effectively serve the values, mission, and vision of the college.

Goal one: Promote learning and teaching for knowledge, skills, creativity, intellect, and the abilities to analyze new information and communicate effectively.

Flowing assessment into the structure

The Student Learning Outcome Assessment Plan structure envisions assessment information as flowing up the structure. Assessment information is aggregated and summarized as the information moves up the structure. Information can enter the structure at the course or program level. Assessment information may consist of many different types of inputs.

Suggested types of assessment information

• Direct student-by-student, outcome-by-outcome measurements.
• Item analysis of selected or sample instruments.
• Embedded direct assessment such as a common question across all finals in a course.
• Alumni surveys. (IRPO)
• Employer surveys. (IRPO)
• Community survey. (IRPO)
• Instructor focus groups examining course level student learning outcomes in a course.
• Studies of cohorts performance on task as they progress through the college.

Plans for improvement

Assessment information would include plans for improvement.

Course level assessment information

Assessment information, in whatever form the information is gathered, would be recorded in the same structure as the assessment plan. The sample below is for only four sample outcomes from four courses.

pSLOcat	course	cslo level	cSLO: Students will be able to:	Assessment	Plan for improvement
problem solve	MS 150	specific	Calculate the slope of the least squares line.	On an item analysis of the final examination 95% of the students were able to perform this skill correctly.	-
foundations	SC 101	specific	Define the concept of total fitness and the essential characteristic and consideration of a physical fitness program.	Students generally answer questions on this topic correctly, but do not exhibit actual behavioral changes that indicate they are acting on the information.	Connect across the curriculum with ESS.
foundations	SC 120	course	List distinguishing characteristics and examples off: bacteria, protozoa, algae, fungi, the major groups of plants, and the major animal phyla.	Students are able to list distinguishing characteristics at a high rate of success.	-
human body	SC 230	specific	Write the structural formulas of different types of hydrocarbons.	Students are able to write simple hydrocarbon formulas with a high rate of success. Students have increasing difficulties as the hydrocarbons become more complex.	At present the course only intends to cover simple hydrocarbons, a full treatment of complex hydrocarbons is beyond the scope of this course.

Moving up the structure

Specific and course student learning outcome information is summarized or aggregated to report on a program's accomplishment of student learning outcomes.

iSLOcat	Program	program Student Learning Outcome: students will be able to:	Assessment	Plan for improvement
knowledge	HCOP	Describe the structure, function, and basic pathologies of the human body.	Students are able to demonstrate basic knowledge and competency in these areas.
communication	HCOP	Communicate health, nutrition, and premedical information in both written and oral formats.	Written communication limited to laboratory reports. Oral presentations, communications skills specifically measured only in EN/CO 205.	Program is content intensive, opportunities for student exposition are limited. Opportunities to achieve this outcome beyond the borders of the classroom are considered the most likely to succeed.
knowledge	HCOP	Describe health care and allied professions.	Occurs primarily only through the HCOP study session, a non-credit, non-required but important aspect of the program.
Knowledge	HCOP	Demonstrate a foundation in basic biology, chemistry, microbiology, anatomy, nutrition, health, and physiology.	Students show adequate competency in all areas except nutrition.	A proposal to make SC 112 Nutrition a required course is being considered by the curriculum committee spring 2006.

Triangulation

A key concept is that of assessment triangulation. Internal assessment would be only one aspect of determining program performance. The Institutional Research and Planning Office (IRPO) would also run surveys of alumni, employers, and the community to determine whether our programs are meeting the perceived needs of the stakeholders of the college. This data would also be reported and would examine programs at the program student learning outcome level.

Information flow diagram

Assessment options

The following screens comment on some of the proposed assessment options. This is not to say any one option should ever be adopted as the only way to assess courses. Different fields of learning will require different approaches to gathering assessment information. The only agreement will be that a given course will use the same assessment option system-wide.

Direct student-by-student, outcome by outcome

• Arguably the only direct method that truly characterizes student learning
• Workable where number of outcomes multiplied by number of students is less than 1000.
• Data entry and tracking task becomes overwhelming when outcomes × students exceeds 1000.

Item analysis of significant assessment event(s) in course

• Provides a form of direct measurement provided assessment event is aligned with outcomes in outline
• Workable on open ended instruments where number of items multiplied by number of students is less than 5000. A 2255 grid took the author three hours to item analyze. Note that scantron would not be an option: questions were open ended.
• Data entry and tracking task becomes overwhelming when outcomes × students exceeds 5000, although if the instrument is multiple choice and scantron is used then there might be no upper limit. Instructors, however, who prefer open ended questions should not be forced into multiple choice formats for the sake of item analysis.

Portfolios

• Provides another form of arguably direct measurement of each student.
• Only applicable to particular programs.
• Portfolio will have to be analyzed and summaries of analysis will have to be generated. The summaries themselves constitute a form of indirect or inferential assessment information.
• Capstone projects would function in a similar way and could also feed assessment information into the system,

Samples

In a course such as an expository writing course or other course that generates student produced creative work, samples of that work at different levels of performance could be submitted as evidence of learning. Thus instructors could submit samples of work that they scored using a rubric. Coupled with student performance metrics one could determine whether the work is being marked appropriately and whether the students are attaining the requisite learning. If an instructor at the Into Reef campus submits something that is clearly not competent and claims it shows high competence, then one can be certain the students are not attaining the proper level and learning is not occurring as it should.

Embedded Assessment

In some courses, posing a common question across all sections in an assessment event could generate assessment data.

Student attainment on outcomes

If a qualified instructor has implemented an outline based on student learning outcomes and has assessed that learning during a term, at the end of the term the instructor knows the strengths and weaknesses of the students. Instructors, however, rarely sit down and reflectively write this information or share this information.

Working with their unit supervisor and with other faculty teaching that same course, a focus group would work outcome by outcome through the outline. A dialog on the learning attainment of the students would be held, with notes on each outcome made by the unit supervisor.

Although indirect, this information can be very accurate if a careful, and thoughtful discussion is had. The agreement with an item analysis was argued for in a separate document.

Comments

• Faculty cannot be forced to all use one particular system.
• Force will generate false information: people simply doing what they feel they have to do to "get by the man."
• Some faculty will put more effort into generating assessment information that is passed up the assessment structure.
• Others will quietly undermine any system that adds to their workload.
• Long-term sustained assessment will depend on the system being "easy" for those who are not energetically committed to the effort. For these faculty members, the brunt of load will have to fall on the unit supervisors.
• The internal assessment information is not meant to "stand alone." Alumni, employer, and community surveys with questions specific to program learning outcomes in a program will provide triangulating information.

Next steps

• Program coordinators and supervisors including chairs (program heads) should begin assembling a spreadsheet of the course outlines for the major courses of their programs. This effort will be aided by the author's work to get all outlines on line in an electronic format.
• Program heads will work with faculty to choose common assessment systems on a course-by-course basis. There is the hope that similar courses will opt for similar systems.

Thanks

With thanks to the many faculty and staff who have provided feedback and ideas that helped form this concept.