Research Methods

Overview

The research project involves two related strands of data collection and analysis to capture student development with 21st century skills and the organizational context that supports this growth. The first strand targets student level data that tracks the knowledge and skills acquisition along with the learning resources and spaces youth access in this process. The Digital Youth Network (DYN) reaches students in grades sixth through eighth. To better understand learning and participation across time, we tracked the development of a cohort of students who entered sixth grade in 2006. From this group we narrowed our data collection efforts to the experiences of 10 case study students. The second strand focuses on the organizational structures of DYN to contextualize student learning and production. This work involves the formal school day media arts classes, the after school pods and Freedom Fridays, virtual spaces and programmatic planning and professional development meetings.

STUDENT LEVEL DATA COLLECTION AND ANALYSIS

Surveys: In the fall of 2006 we administered a survey to DYN students and a comparative set of students living in a technology-rich environment to assess students access to technology and their interests and experiences. End of the school year surveys in 2007 and 2008 as well as summer use surveys in the fall of 2007 and 2008 followed this initial query. The questions tapped the following aspects of student use: (1) access to technology at home and school, (2) history of technology use across multiple settings, (3) use of formal and informal learning resources, (4) motivation to learn about computing, and (5) self-rated knowledge of computing terms and software tools. A final survey will be given in May of 2008. Table 1

Interviews: We met with a sub group of the 2006 cohort for learning to better understand how students produced and presented artifact involving 21st century skills. We carried out two distinct types of interviews: learning ecology and artifact based interviews. The learning ecologies interview aimed to obtain more detail on the use of computers at home, school, and other spaces as well as between peers face-to-face and online. There were three main sections of the interview: (1) how students used and learned about technologies in various locations, (2) identification of the skills students thought necessary to “be good” with computers, and (3) student interest formation and future plans.

From the 2006 cohort we selected 10 case study students. We purposively sampled students along several dimensions, including: student roles in pods as teachers and learners, productivity, skills and interest growth, and regular attendance and engagement with pods. We met with these students to conduct artifact-based interviews in the spring of 2007 and 2008. These semi-structured interviews provide a focused look at student projects and an account from the learner’s perspective of how they learned, pathways for project creation, and the opportunities for fluency building within different projects.

Table 2

Analysis: With respect to the survey data, our current focus is on producing descriptive statistics to indicate the levels of home access to computing tools, and use inferential statistics to investigate changes in students’ experience with fluency building activities, interest, and confidence. These analyses represent a small portion of the data analyses that are possible and we will share additional results as they are generated.

Researchers are using the collected from the case study students to create narrative summaries and visualizations of learning activities across setting and time called “technobiographies” (Barron, Martin, & Lewis, 2007a; Barron, Martin & Takeuchi 2007b). Both the narrative and visual representations will help us theorize about the conditions that support the diversification of children’s learning ecologies over time. This includes the documentation of people, tools, and informational resources that supported activities.

ORGANIZATIONAL LEVEL DATA COLLECTION AND ANALYSIS

Ethnographic Field Work: The organizational study focused on the instructional practices and learning exchanges between mentors and students as well as mentor training sessions. A team of ethnographic field researchers observed DYN participants and conducted formal and informal interviews. Researchers attended multiple sessions of the weekly media arts classes, pods, and Freedom Fridays to produce extensive, thickly descriptive field notes. Given the newness of 21st century skill instruction, researchers assumed a grounded theory approach to these observations with attentiveness to the learning context, instructional approaches, and project creation. With respect to this final component, researchers targeted evidence of group formation, projection conceptualization, access to resources, and problems and the social elements that support and hinder solutions to those problems.

The 2006-2007 and 2007-2008 academic years were intensive periods of ethnographic work. Researchers used a collection of audio, video, and written reports to capture these events. Additionally, researchers collected student-produced artifacts, including student work in progress. In the 2008-2009 researchers reduced data collection efforts to focus on the development of the foundational course, the sixth grade media arts class. This work follows the final phase in the three-year development of this class, which serves as the vehicle DYN staff introduce students to 21st century skills.

Table 3

Note: Media Arts classes varied in course offering, one, two, or three quarters, and length of class time, either 45 minutes or 2 hours.

Researchers also observed organizational activities involving staff training, professional development and curriculum building activities. Finally, researchers observed school events, including history and science fairs, plays and report card distribution nights, to understand the transference of students skills and gain a better sense of the larger organizational context in which DYN operates. Formal staff interviews complement these observations in which we probed instructional planning, practices, and challenges and instructor perceptions of the case study students’ development since mentors have a unique window into student development.

Table 4

Analysis: After the first research year, a team of researchers conducted open coding (Straus & Corbin, 1998) analysis to identify themes and patterns. Using this list, team members formalized a code list to conduct focused coding (Straus & Corbin, 1998). For the 2006-2007 and 2007-2008 data two researchers coded each field note entry. Next, researchers compared the application of these codes to determine consistency. We are currently in the process of refining these codes to correct any inconsistent application of codes. As researchers explore various aspects of the organizational setting, they may amend or add to foundational work.

REFERENCES

Barron, B., Martin, C., Lewis, S. (2007a) “Following Learners in School and Out in the World: Constructing Technobiographies From Interviews at Multiple Time Points.” Part of symposium: Technobiographies: Researching Life Stories With Technology. Presentation at the Annual Meetings of the American Educational Research Association, Chicago, Illinois. April 9 – April 13.

Barron, B., Martin, C., Takeuchi, L. (2007b). “Technobiographies as a Tool for Conceptualizing Learning Across Settings and Time.” Part of symposium: Methodological Challenges and Innovations: Studying Learning in Informal Contexts. Presentation at the Annual Meetings of the American Educational Research Association, Chicago, Illinois. April 9 – April 13.

Straus, A. and Corbin, J. (1998). “Grounded Theory Methodology,” In Norman K. Denzin, Yvonna S. Lincoln (Eds) Strategies of Qualitative Inquiry. Sage Publications: Thousand Oaks, CA.