Data and Analysis

The Marginal Syllabus and Educator Open Learning

Social media has challenged and changed how educators pursue interest-driven learning. From Twitter chats to Edcamps, new technologies and social practices are reimagining where and how educators learn, and what counts as professional learning. Our research is interested in web annotation as a mediator of educators’ open learning. Specifically, we are exploring how the social and collaborative practices of web annotation mediate educator participation in public conversations, establish professionally-relevant connections with texts and contexts, and encourage discursive communities of practice.

Forthcoming summaries from the 2016-17 Marginal Syllabus include our work with open data as a proxy for learning analytics, and share descriptive statistics, interactive graphs, and data visualizations about:

  • Who participated in Marginal Syllabus annotation conversations throughout 2016-17;
  • How many times participants annotated a text during a Marginal Syllabus event, and how regularly participants also used annotation as a means of replying to one another and sustaining conversation; and
  • How groups of Marginal Syllabus participants connected with one another, focal texts, and partner authors during annotation conversations.

A few additional notes that provide helpful context about our data collection and analysis:

  • We are collecting and analyzing three types of data – participant interviews, digital artifacts (such as blog posts and planning documents), and publicly available web annotations (including both annotation content and metadata).
  • Prior to the 2016-17 annotation conversations, all authors agreed to have their texts publicly annotated; six of nine partner authors also contributed to these conversations using the web annotation platform Hypothesis.
  • The first six conversations (which occurred from August, 2016 through February, 2017) were organized as “flash mobs” that concentrated conversation during a single hour; perhaps not surprisingly, many people contributed prior to and after this set time. As a result, the final three conversations (March, April, and May of 2017) were organized as “annotathons” that spanned an entire week. Read more about the iterative design decisions that improved the structure of public annotation conversations.
  • Our analysis and visualization of open web annotation data about educator participation in public web annotation conversations is a proxy for learning analytics.
  • It is our hope that by examining individual conversations, and then by looking across the entire 2016-17 syllabus as a comprehensive dataset, these descriptive statistics and data visualizations will help our research team identify key moments, clusters of conversation, and salient texts and contexts to subsequently analyze using qualitative and discourse analysis methods.

A Note on Our Methodology

The Hypothesis web annotation platform offers a variety of ways for collecting public annotation data. Within the context of our research project, this open data – once collected, analyzed, and visualized – serves as one proxy for educators’ learning. The simplest approach to collecting open annotation data is by using the Hypothesis dashboard and the built-in search tools and filters. This method is helpful for performing quick single searches by: username, keywords, tags, URLs, or groups. However, in order to perform analysis on the Marginal Syllabus annotation sessions, we required bulk downloads of annotations by URL, so we decided to approach the problem by programmatically extracting the annotations through the application program interface (API).

Once we collected data about a specific Marginal Syllabus conversation, our exploratory data analysis was performed on the Jupyter platform using the Python programming language and the pandas library (pandas is an open-source library that structures data for analysis). Because Python is a general purpose programming language, it made it simple to build the ancillary work needed for this project, such as: collecting open data, formatting open data for visualization, and exporting open data and representations for sharing. Ultimately all these steps were packaged into a single Jupyter notebook and automated for processing each of the annotation sessions scheduled for the length of project.

Data Analysis and Visualization Tools

We’re using a variety of free, online tools to visualize and share open data about the complexity of educator participation in Marginal Syllabus conversations. We have initially decided to use three different chart types to visually represent educator participation in open annotation conversations: stacked bar graphs, time series line graphs, and network node graphs. We have used the following tools to publish and share these graphs more widely.

Tableau is a software company that seeks to “help people see and understand their data.” In our project, we’re using Tableau Public, a free service that allows our team to create and share interactive graphs. We used Tableau for publishing stacked bar graphs and time series line graphs. Tableau also provides a feature to embed visualizations into websites or online documents. This allows us to update the underlying data to the graphs, and to propagate the changes to the visualizations without needing to modify each individual embedded graph.

Onodo is an open source network visualization and analysis tool. Network node graphs represent unique elements in a data set as nodes, or circles, and the relationships between those elements as edges, or lines, that connect those nodes. In our research, each user, and the source document (i.e. a partner author’s focal text), is represented as a node. Annotations are represented as directed edges, or arrows, in the visualization. Parsing each Hypothesis annotation record for “sender” and “destination” (i.e. an original in-line annotation of the focal text, or a reply to an annotation) helped us to visualize how the educators interacted with one another and provided a sense of structure to their public annotation conversation.