Internet Learning Volume 3, Number 2, Fall 2014 | Page 80

Visualizing Knowledge Networks in Online Courses
Our research questions address fundamental
challenges of doing sophisticated
analyses of online discussions. Conversations
have structural and other non-content
attributes, but are also contexts where
unique individuals come together and
co-create a body of content. The problem
of identifying and quantifying individual
influence on conversational content and
structure is a complex one, as is the problem
of identifying how conversational structure
and content might arise as a combined expression
of the attributes and behaviors of
multiple individuals. In the following sections
we will describe our approach to each
question, and discuss our findings.
IV - Methodology
In this paper, we present the current state
of the qualitative, quantitative, and visual
research methodology that has emerged
over the past three years of collaborative
work. The Columbia and Pearson teams
adopted an iterative, grounded approach
to data gathering and analysis, beginning
with a thick, digital ethnography of discussions
in several online courses. Methods
included close readings of discussion texts,
analysis of conversational moves and strategies,
and detailed analysis of engagement
with assigned and unassigned resources. We
identified quantitative and qualitative attributes
as described above in the Analytical
Framework, which we then applied to the
data on successive passes over a period of
several months. The result was a set of rich,
augmented discussion data containing both
automated and hand-coded attributes for
each discussion response, along with detailed
digital-ethnographic field notes.
Then, in order to analyze the data
from network and visualization perspectives,
we employed a variety of software
tools and techniques. These approaches included
creation of a graph database with a
custom schema designed to model threaded
discussion data, a domain specific language
(DSL) for exploring that data, and
use of Natural-Language Processing (NLP)
tools, network visualization tools (such as
Gephi), graphic design software (such as
Adobe Illustrator), and spreadsheets. We
relied heavily on open source software, and
wrote our own code as well. We were able
to automate some tasks with custom scripts
and parsers, while others required hours
of painstaking, repetitive work. Thus, the
present work is presented as a practitioners’
progress report on the project of defining
a set of Social Knowledge Networking attributes
relevant to emergent digital pedagogies,
and of devising ways to measure
and reason about them. Our examples are
intended to be illustrative rather than definitive.
Our methodology is presented as
one of exploratory inquiry, rather than as a
proven, streamlined approach to answering
the kinds of questions we engage here.
We draw our data examples from a
single week of anonymized, small-group,
threaded discussion data, consisting of one
instructor prompt, seven individual thread
response trees, and a total of 64 comments,
over a period of four days. All names are invented
code names, applied without regard
to gender or course role. The seven students
are Alakel, Danen, Fesler, Loret, Viska, Renlit,
and Kerrad. Naya is the lead instructor,
and Jakata is a TA. Radsel, a participant
from another group, cross posts one comment
in Fesler’s thread.
For each research question, we provide
a brief conceptual overview of our approach;
a technical summary describing
the processes and technologies involved; a
situated example to illustrate an application
of the model to real data; and a discussion
where we explore Instructional Design insights
and implications for future work.
79