The Learning Object Context Profiling Model

Abstract

From the descriptions of the individual dimensions in Sections described in Strijker (2004), many of the dimensions relate to overlapping situations. This occurred through their evolution within the Why?, Who?, What?, How?, and Where? questions as a structure for the research. This structure may have served its purpose however, if a reduced set of dimensions can cover the same general ideas with respect to the Systems-Personal orientations.

The Learning Object Context Profiling Model

All the dimensions presented in Section 1.2–1.6 (Strijker, 2004: Chapter 9) can be combined to one model. Figure 2 shows the full set of dimensions.

		S					P
Why?
	Cultures within the context	The industrial world	The Domestic world	The Civic world	The world of Opinion	The Merchant world	The world of inspiration
	Level of learning objectives	Knowledge	Comprehension	Application	Analysis	Synthesis	Evaluation
	Learning scenarios	Acquisition					Participation
	Incentives for reuse	Organisational					Personal
Who?
	Quality of the object	Formal processes					Personal
	Work processes	Formal workflow					Personal habits
	Need for human interaction	Low					High
What?
	Purpose for creating the object	Created for learning					Not created for learning
	Nature of the course object	Specific					General
	Adaptability of the learning object	Fixed					Editable
	The role of the learning object	Replace the instructor					Supporting the instructor
How?
	Specifications for the learning object	Predefined					Defined when needed
	Personal control over learning	Low					High
	Tools for reuse of learning objects	Specific					General
Where?
	How learning objects are stored	Repository					Locally
	Structuring of learning objects	Organisational					Personal

Figure 2 Learning Object Context Profiling Model (all dimensions)

However, as was apparent from the descriptions of the individual dimensions in Sections 9.2 – 9.6, many of the dimensions relate to overlapping situations. This occurred through their evolution within the Why?, Who?, What?, How?, and Where? questions as a structure for the research. This structure may have served its purpose however, if a reduced set of dimensions can cover the same general ideas with respect to the Systems-Personal orientations. To make the Model as concise as possible (to support its use in practice), the following reductions of dimensions could occur:

·         One dimension relating to Cultures within the context: Remains as separate dimension

Eight dimensions relating to pedagogical/learning issues: Level of learning objectives, Learning scenarios, Need for human interaction, Purpose for creating the object, Nature of the course object, The role of the learning object, Specifications for the learning object, and Personal control over learning: Can be represented by the dimension Learning scenarios

·         One dimension relating to Incentives for reuse: Remains as a separate dimension

Three dimensions relating to processes for dealing with learning objects: Quality of the object, Work processes, and Structuring of learning objects: Represented by the dimension Work processes

·         And the remaining three dimensions, all having to do with tools relating to learning objects, Adaptability of the learning object, Tools for reuse of learning objects, and How learning objects are stored: Represented by the dimension How learning objects are stored

This process reduces the original set of 16 dimensions to a concise set of five dimensions, representing the main differentiating aspects of the set of 16. With these five, the simplified Learning Object Context Profiling Model takes the form shown in Figure 1.

 

	S					P
Cultures within the context	The industrial world	The Domestic world	The Civic world	The world of Opinion	The Merchant world	The world of inspiration
Learning scenarios	Acquisition					Participation
Incentives for reuse	Organisational					Personal
Work processes	Formal workflow					Personal habits
How learning objects are stored	Repository					Locally

Figure 3 Learning Object Context Profiling Model, short form

From Model to Tool

Two Learning Object Context Profiling Models have been developed, a long form and a short form. These have been derived from the results of the ten projects in the research. A next step is to use the Models to develop a toolset that can be used for the three main tasks of the research. Section 1.8.1 describes the simple transition from model to tool, the Learning-Object Context Profiling Tool. The Learning Object Context Profiling Tool can be used for the three different tasks that are also used in the research: A descriptive task (Section 1.8.2) to describe a certain context, an explanatory task to explain certain outcomes in a context (Section 1.8.3), and a prescriptive task to predict certain outcomes in a certain context and on that basis suggest guidelines for how to proceed (Section 1.8.4).

Description of the tool

The Learning Object Context Profiling Model has been deliberately presented as a set of parallel dimensions, each of which has a left-hand extreme value that corresponds to a Systems orientation toward learning objects and reuse, and a right-hand extreme value that corresponds to a Personal orientation. It is simple from a representational point of view to convert either the short or long form of the Model into a paper-based tool that can be used for different purposes in relation to the Model. The tool is no more than the same graphic representation of either the short or long form of the Model, but with each dimension labelled “1” to “5”, with “1” corresponding to Systems oriented, with a vertical grid running through all of the “1” values on each dimension, and similarly all of the “2”, “3”, “4”, and “5”values. These gridlines are used to plot the representation of a context involved with reuse and learning object, by marking each dimension on a scale of “1” to “5”. By plotting the scores (usually obtained through a researcher’s subjective assessment rather than a formal measurement) the characteristics of a course or curriculum or other setting can be placed on the different dimensions in the Learning Object Context Profile Model. The profile of the particular context can be found when all dimensions are filled in and connected with a line and can show if the particular context for learning objects is Systems oriented or Personal oriented. Thus the profiling process supported by the tool can be used to predict how learning objects can be specified in a certain setting and what type of learning objects can be expected to be effective and efficient for reuse. The model can also be used to observe, explain, or predict how dimensions interrelate. When a course object is analyzed and found to be mainly on the left side of the scale, it is expected that a specific specification of learning objects can be made including a various set of characteristics such as a predefined instructional model, time constraints, testing, tracking, structure, and interactivity within the learning object rather than with humans as they make use of the learning object. Reuse is expected to be on an asset level because of the specific requirements. If a context or object is analyzed and is found to be mainly on the right side of the scale the specification of learning objects has to be more general but is still possible with such descriptors as subject and description. Reuse can occur with assets, but also sets of objects with a larger granularity can be reused. Because of the general nature of the object it is expected that reuse will be interesting if course developers/instructors can change or add pedagogical annotations to make a learning object useful for their own contexts.

The profiling possible with the tool can be used for descriptive, explanatory, or prescriptive tasks related to reuse, as described in  following Sections.

Descriptive task

The Learning Object Context Profiling Tool can be used to describe a certain context by filling in the values on each dimension. Plotting the values for each dimension can give insight about a certain context and help to describe the characteristics of a given context. An example is given for the “Ranks” course from the military context and a university course. The Ranks course is represented in the model as a set of small triangles. The university course is represented with small circles. This reflects a course given in a blended-learning approach such as provided in the TeleTOP® projects in the university, corporate-learning, and military contexts. Figure 2 shows the Tool with the two different courses represented.

 

1

2

3

4

5

 

	S					P
Cultures within the context	The industrial world	The Domestic world	The Civic world	The world of Opinion	The Merchant world	The world of inspiration
Learning scenarios	Acquisition					Participation
Incentives for reuse	Organisational					Personal
Work processes	Formal workflow					Personal habits
How learning objects are stored	Repository					Locally

Figure 4 Learning Object Context Profiling Model, short form used for descriptive purposes

 

The differences in profiles of the two courses on the dimensions have a large impact on the specifications for the learning objects. For the “Ranks” course the profile is shown by a non-vertical line, indicating discrepancies in the orientation on the different dimensions. This can be seen via the graphic; the more a profile is vertically aligned, the more likely that aspects of reuse will proceed smoothly in a particular context. In the university TeleTOP® project reuse was occurring but in a Personal-oriented way. The profile shows a near vertical alignment. And conversely, the more the profile deviates from a vertical line, the less likely that reuse will proceed successfully.

Explanatory task

For the explanatory task the Learning Object Context Profiling Tool can be used to do more than describe but also to explain certain outcomes based on the alignment of values of the dimension. When the values for a given context are plotted on the dimensions and there is no alignment in the values, the tool can explain the reasons for failure of success with the dimensions that are out of line.

The Learning Object Context Profiling Tool tries to identify important aspects for a reuse strategy but when the tool is used not all dimensions may have the same orientation because of the complexity of organisations and the different blends in learning scenarios. The results of the profiling with the tool may be difficult to interpret when such complex contexts are analyzed. The tool can be used to give information about courses and curriculums in order to help explain why reuse may or may not be likely to take root. For example, problems may arise when the curriculum covers a very large cognitive domain whose objectives range from knowledge to evaluation. This means that for one sub-context in the setting, one dimension may be system oriented and for another, a dimension may have a personal orientation. Such complex profiles are likely to explain why reuse strategies fail to become embedded in an organisation.

A particular source of discongruency can come from a lack of vertical alignment between the underlying “worlds” or world view in a particular setting, and the values of other dimensions, or when a particular organisational context has within it different world views in different subsets of the organisation. Boltanski and Thevénot describe how opinions about the underlying values of a culture can be influenced by the culture which dominates one’s way of thinking. Table 3 shows an adaptation of Boltanski and Thevénot’s interpretation of how the cultures react to each other and what the criticisms are from one world to another.

 

Table 3 Criticism from one world to another (adapted from Boltanksi and Thevénot)

Criticism	Criticism of the World of Inspiration	Criticism to the Domestic World	Criticism of the World of Opinion	Criticism of the Civic World	Criticism of the Merchant World	Criticism of the Industrial World
From the world of Inspiration		Habits, inherited social norms and principles, fossilized institutions form a break to creativity and initiatives.	Vanity of appearance, personal rivalry, the higher attention paid to the image of the self, inhibit imagination.	Cold institutional frameworks freeze human warmth and affective relationships	Self-interested people and dependence on “money’ hijack invention and innovation to reroute them for business	Rigidity of routines, impersonality, methods and know-how can hinder spontaneity and creativity.
From the Domestic world	Disorder, carelessness, disorganized behaviour. Too much attention given to the emotional component		Good manners require discretion and caution. Exhibitionism is incompatible with common decency	The collective reinforces an underlying anonymity and obstructs individual responsibility	You can’t buy everything. Self-interest corrupts social bounds.	Assemply-line production brings low quality. Technical expertise sweeps away common sense and realism.
From the world of Opinion	Esoteric. False depth and elitism. Selfishness.	Domestic secrets, paternalism. Opacity. Lack of daring. Refusal to be compared and to be assessed.			The commercial focus of communication and information through self-interested advertising.	The esotericism of specialists
From the Civic World	Individualistic approach, irresponsibility, spontaneity, adventurism.	Paternalism, family secrets (corruption, etc.), authoritarianism, pollution of authentic human relations.	Public opinion is manipulated, does not reflect aggregation of interests.		Egoism of the wealthy and individualism in a merchant world puts democracy at risk.	Technocracy, attention paid to individual promotion more than to collective enrichment.
From the Merchant World	Lack of emotional distance and control of emotions, in business one needs to keep one’s self-control	Personal relations, traditions, prejudices, and routines hold back competition and opportunistic merchant relationships.	Deviousness, mass culture, snobbery	Collective processes inhibit action.		Rigidity of tools and methods, heaviness of organisations, mentality of engineers conflict with commercial principles.
From the Industrial World	The wastefulness of improvisation, uncertainty, unreliability	Tradition is not adapted to present times, the old is outmoded.		Inefficiency of administrative procedures. Costs of social policies.	Useless luxury goods, unjustified prices, market impulsive drives

The sorts of criticisms or scepticisms identified in Table 3 may be deeply submerged in a cultural setting, not acknowledged or even articulated. Yet a balance emphasizing a particular world view is likely to underlie any context and if not identified, may result in a subsequent lack of vertical alignment on the Learning Object Context Profiling Tool. This lack of alignment can help explain why a learning object sharing and reuse strategy fails to take root.

Prescriptive task

The tool can also be used to predict success or failure based on the alignment (or lack of alignment) of the profile mapped onto the five dimensions of the short form of the tool. For example, when three of the five dimensions are focused on a Personal orientation and the other two on a Systems orientation, it is expected that reuse in the context will not be very successful. However, suggestions can be made to improve the likelihood of success if the alignment is not severely non-vertical. The following guidelines relate to predictions made using the Tool, and are summarized from the guidelines expressed at the ends of Chapters 7, 8, and 9. They can be used for increasing the likelihood of success in implementing a reuse strategy:

·                     Guideline 1

An organisational-wide strategy for reuse is needed for different reasons such as critical mass, copyrights, rewarding, and expectations. A policy for how to deal with exchange for individual course developers is essential. Course developers need to be aware of what to expect and what to do in terms of reuse and exchange of learning material. For the Personal orientation the organisational policy can make individual persons aware of the potential of their own resources and provide rules for how to exchange material. A Systems orientation can focus on giving access to resources in repositories based on the organisational reuse strategy.

·                     Guideline 2

Research projects that focus on new systems-oriented developments should also have a strong implementation component to test different scenarios in practice. The outcomes of the research should be validated by practical use. For the Personal orientation the research results should be available to see what the benefits for personal use can be, within a Systems orientation the organisational strategy should use research outcomes for a more efficient way of working, providing course developers with the tools and infrastructure that were validated in the research.

·                     Guideline 3

An IT infrastructure should be available to all users such as course developers and participants to distribute and exchange available materials. For the Personal orientation the IT infrastructure is less important because most material is kept on a local hard drive. For a Systems orientation, an adequate network infrastructure is a prerequisite for efficient reuse and exchange possibilities

·                     Guideline 4

Because various types of learning scenarios may be in used in a certain context, different approaches regarding to reuse possibilities and definition of learning objects may be needed. Within a Personal orientation learning scenarios depend on the course developer, while within a Systems orientation designers can make use of predefined template-based authoring tools that express a certain learning scenario.

·                     Guideline 5

Exchange of material can be easily implemented when resources are based on structured data such as databases or XML. Standards should be implemented for future reuse of course material. Within the Personal orientation standards based on CMSs can support exchange of learning objects on a general level. An implementation of standards within a Systems orientation can support reuse on asset level.

·                     Guideline 6

A policy related to security and classified material should be developed in terms of accessibility to material. Encryption of material, network access, and export regulation are issues that need to be covered. Within a Personal orientation the security is organized by the individual who decides what to offer for reusability. A Systems orientation focuses on structured repositories and related database-access controls to protect classified content.

Reflections on the Model

Previous sections indicated how the Learning Object Context Profiling Model and Tool can be used for descriptive, explanatory, and prescriptive tasks relating to reuse of learning objects. In this section some reflections on the Model are given in terms of key issues that have been considered throughout this research. Does the Model help to understand and interpret those issues? Standards and metadata, granularity, and the lifecycle of a learning object will be discussed first, followed by the fit of the Model to the 3-Space Design Strategy (Moonen, 2002) and the 4-E Model (Collis, Peters, & Pals, 2001) introduced in Chapter 1 and used through the research to structure the observations.

·                     Standards and metadata

For the use of standards different approaches can be chosen for the metadata, packaging, and runtime specifications. The expected metadata are closely related to the results of the tool. When the results of the tool are Systems oriented, the assumption can be made that most LOM metadata are available for tagging the learning objects. These metadata can be based on a predefined vocabulary and build upon professionally structured taxonomies. When the results are Personal oriented, it is expected that only a minimal ADL SCORM™ set of mandatory fields is relevant. The metadata used are individually created for personal use, for identification of learning objects and for the individual’s own reuse purposes. Thus, standards and metadata map onto the Systems-Personal orientation, with different variations appropriate for settings near the endpoints:

	S					P
Metadata	Complete set					Minimal set

 

·                     Packaging

For packaging, a Systems orientation can be helpful for the specification of learning objects that are part of a larger, composite learning object. The packages can be constructed in such a way that learning objects are also editable in other CMSs. Using structured data for the content and full metadata descriptions makes it also possible to present content on mobile devices such as telephones and PDAs. In contrast, when the Tool indicates a Personal orientation, packaging may be more general in terms of fixed pieces of HTML. The use of a limited set of metadata may present barriers for exchange of learning objects outside of one’s own reuse. Thus, packaging maps onto the Systems-Personal orientations, with different variations appropriate for settings near the endpoints:

 

	S					P
Packaging	Strict defined					General

 

·                     Runtime orientation

Runtime interaction with an LMS is expected to be mainly involved in a Systems orientation. The human-computer interaction in CBT can be used to make learning more attractive when learning scenarios are focused on acquiring knowledge. In contrast, when the results are more Personal oriented, the runtime model is not used because interaction comes from instructors or other learners. Thus, the characteristics of the runtime model map onto the Systems-Personal orientations, with different variations appropriate for settings near the endpoints:

	S					P
Runtime model	Based on interaction					No interaction

·                     Granularity

The development of CBT-oriented learning objects in a Systems-oriented context may require smaller learning objects than in a Personal-oriented context. The use of assets such as pictures, text fragments, and videos are part of the development process of CBT for knowledge acquisition. In contrast, a Personal-oriented context is expected to reuse larger learning objects and use combined learning objects. Thus, granularity maps onto the Systems-Personal orientations, with different variations appropriate for settings near the endpoints:

	S					P
Granularity	Small					Large

·                     Lifecycle of a learning object

When the results of an application of the tool are Systems oriented, the lifecycle of a learning object can be static and formalized, as used for the description of several of the projects in Chapters 5, 6, and 7. In contrast, a hyperlinked and flexible use of the lifecycle as described in Section 8.3.2 is appropriate when the results of an application of the Tool show a Personal orientation for the context. Some lifecycle stages may not be relevant for development in some cases in a Personal-orientation setting. Thus, the lifecycle of learning object maps onto the Systems-Personal orientations, with different variations appropriate for settings near the endpoints:

	S					P
Lifecycle	Static formalized					Flexible

·                     3-Space Design Strategy

The tool can also be used to predict what kind of design strategy can be used for the development of learning objects. When the results from application of the Tool are Systems oriented, the development of learning objects can be structured or rational, and based on rationally expressed development strategies and models. When the result of an application of the tool shows a Personal orientation to the context, the development is expected to be associative and creative, not following predefined models for software engineering and instead a good candidate for methods such as user-centered design and rapid prototyping (Moonen, 2002). Thus, Moonen’s 3-Space Design Strategy can also map onto the Systems-Personal orientations, with different variations appropriate for settings near the endpoints:

	S					P
3-Space Design Strategy	Structural					Associative

·                     4-E Model

The 4-E Model (Collis, Peters, & Pals, 2001) was introduced in Chapter 1 as a tool for predicting the likelihood of an individual’s uptake of a technological innovation in his or her own working situation. It was used as the basis for the secondary research questions related to the human perspective throughout the research. How does it relate to the Learning Object Context Profiling Model? According to the 4-E Model, the likelihood is related to four clusters of variables: an individual’s perception of effectiveness, ease of use, personal engagement, and characteristics of his or her organisational environment. When the results of an application of the tool are Systems oriented, the 4-E clusters are likely to be underrepresented because they focus on human aspects. When the results of an application of the tool are Personal oriented, it is expected that all aspects from the 4-E model will be relevant and applicable. Thus, the 4-E Model maps onto the Systems-Personal orientations, with different variations appropriate for settings near the endpoints:

	S					P
4E Model	Not represented					All applicable

·                     Combining the issues

Reflecting on the overall results through the frame of reference of the Model has led to the following general observations. The results of the research show that reuse may not be focused on a wide exchange of all available material but on a small level within departments and particularly on the reuse of one’s own course material. The use of specifications such as ADL SCORM™ may not have the expected impact on adaptive learning and building courses based on learning objects from large repositories as expected by many. Also the runtime specifications for tracking and tracing may not be suitable for the required learning scenarios in a certain context. Also the complete set of metadata to select material from a large repository may not be required or efficient. Reuse of material is important for individual users or for knowledge management Reusing material from colleagues that move to another job can be very efficient and time saving. The fact that knowledge of instructors is stored in courses that can be (partly) reused can also been seen as a form of knowledge management in large companies such as Shell EP or the military.

The use of CMSs can be compared with the use of e-mail as a tool that can help work processes related to learning objects and reuse. Instructors use CMSs as a tool to provide course material in their own ways, supported by a Personal oriented system. The ease of use of the systems and the freedom offered to the instructors make such a widespread use possible. In contrast, the use of courseware-development tools such as Authorware™ and Easygenerator™ is very much limited to a group of specialized users, likely to represent a Systems orientation. The complete specification bundle of ADL SCORM™ seems to focus on this small group of courseware developers. The strength of the specifications will be found in the extent they become taken up in the frequently-used and flexible CMSs to make exchange and reuse of material possible under the control of the individual instructor and with a Personal orientation.

Reflections on the Research Methodology

As the dissertation draws to a close, it is appropriate to not only reflect on the insights and implications of the topic of the research, the reuse of learning objects, but also on the research methodology as a process in which the researcher, as an Action Researcher, spent nearly four years. This section reflects on how well the research tasks have been carried out (Section 1.10.1) and the limitations of the research despite the care of the researcher to maintain structure and consistency over ten different projects (Section 1.10.2).

How well have the three tasks set out for the research been carried out?

This question focuses on how well the descriptive task, the explanatory task, and the prescriptive tasks of the research have been carried out. It can be asserted that these tasks have been done, carefully and systematically (as well as, occasionally, from a Personal orientation). Within the dissertation the different tasks can be clearly identified. For each project in the three contexts the aspects are described using the common Why?, Who?, What?, How?, and Where? questions. The learning-object lifecycle and secondary research-question summaries explain what aspects are key for reuse strategies and the implementation of learning-technology standards for that particular project. The Learning Object Context Profiling Tool is part of all three of the descriptive, explanatory, and prescriptive tasks in the research. The five dimensions in the short form of the Learning Object Context Profiling Tool can be used to predict if a reuse strategy within a certain context will be successful. Thus yes, the three tasks set out in Section 4.1.2 were never forgotten, and guided the research from beginning to end.

Limitations of the research methodology

As earlier discussed in Section 4.1.8, Action Research has some limitations related to the personal over-involvement of the researcher, the limited amount of control that the researcher has on the environment where the research takes place, and the generalizability of the research. All of these remain limitations for the present study.

The personal over-involvement or the researcher in the research is a limitation of the research because the role of the researcher in the different projects was essential. The projects were based on the skills and knowledge of the researcher and could not be carried out by others.

The control of the environment was not determined by the researcher but given by the organisations which set the boundaries of the projects. Also the time planning for the research was determined by the projects. The evolving character of the projects was also important because every cycle within the project included learning aspects for the next cycle and thus changed the starting point for the new project and made inter-project comparisons different.. The specific character of the research also included restrictions on the sorts of projects available to the research and the number of organisations with an interest in the domain of the research who were willing to be test beds for the research.

The generalizability of the research is also a limitation because only a limited set of organisations and projects was used for the research. The three contexts give an overview of different types of organisations where reuse plays an important role, but the range of institutions within the original contexts was limited. The ten described projects give detailed information about the different organisations but they may not reflect projects in other university contexts, corporate-learning contexts, or military contexts. The University of Twente developed its own CMS in contrast with other universities that use CMSs such as BlackBoard™. The development of such a CMS (TeleTOP®) had special advantages for reuse functionalities within the organisation. Also the fact that the TeleTOP® CMS was used within the corporate-learning context gave opportunities for reuse tailored to the organisational needs that are not likely to be possible with a CMS that was not so much under the control of the researcher in terms of design innovations. Outside of TeleTOP®, within the military context a tailor-made LCMS was developed to support the users. Such tailored systems may not be used or not be available within other contexts.

Where Next?

With the reflection on the methodology of the research, this dissertation is effectively over. Four year’s of work and immersion in a topic have supported the researcher’s conviction that many barriers and difficulties confront the mainstream uptake of reuse and even use of digital learning objects in practice. Successes can be found, but often these are successes within a Personal orientation whereas the focus of research and industry development with metadata and standards and reuse typically represents a Systems orientation. Does this mean the problems will never be solved? New technological developments are occurring that many feel will stimulate the same step-change as occurred when the World Wide Web first became available to the broad public. These technological developments relate to the Semantic Web. During the last year of this research a rapid growth in articles, conference, workshops, special issues of journals, and Web sites sprang up, indicating that the Semantic Web would in fact be the real breakthrough for sharing, finding, and reusing resources. The researcher and promoter were asked in early 2004 to contribute to a special issue of a journal focused on the question of whether the Semantic Web was going to lead to mass-scale breakthrough with respect to the lifecycle of learning objects and in particular to the ontologies and taxonomies that serve as the basis for metadata. The reflection we wrote can also serve as a closing thought for this dissertation. We repeat the final portion of our reflection here[1]:

The Semantic Web and Ontologies: An answer? To what question?

In discussions of the Semantic Web, it seems that the focus is predominately on only two of the six lifecycle stages: “select”, and before that, “label”. The assumption seems to be (perhaps this is an unfair interpretation) that if these functions work well, then this is the key that will “forever change the shape and form of learning” (Hodgins, 2000a). However, our argument is that all stages are important, particularly the “use” stage; and also that context and learning philosophy give very different views of these stages. For many of the issues identified in the special issue, the Semantic Web and ontologies have little or no relation to the sorts of questions that are raised.

There are lessons already being learned from the current work with standards and metadata. All of the standard bodies are developing taxonomies for their metadata. While these taxonomies may seem appropriate from a systems perspective, in practice they may not reflect a personal orientation: the way human users think about learning objects if they go to find them, or have to label them. There are two major issues: Can a taxonomy be generalized across all potential users? How much detail is necessary and how much detail is it feasible to collect?

In terms of the first question, a number of groups have tried to define taxonomies for metadata based on pedagogical analyses of potential end users. In the CANDLE Project (2000-2003), sponsored by the European Union, considerable effort was put into the modelling of different user groups in order to provide input for the set of metadata to be used (Scott & Van Helvert, 2001). To help users in the CANDLE Project assign the metadata to a potential learning object, a software Wizard was created to guide assigners through each of the metadata categories (Liu, 2003). As far as possible, pull-down menus were available in the Wizard, and for each metadata category, an example and set of definitions were supplied. However, even with this level of detail, the use of the Wizard by an instructor intending to use an eventual object as a potential resource, particularly for a generative or contribution-type activity, turned out to be problematic in user trials (Brostoff & Kent, 2003). One reason is that with a generative or collaborative approach, the activity is not inherent to the learning object itself, but depends upon what the learner does with the learning object. It may be useful, for example, that a broad selection of learning objects be made available, so that the learner can decide for himself which are the most useful for his task.

Another problem is the selection of a taxonomy. Sets of tags that might appear generally appropriate in a university context would lack many elements that would be necessary in a corporate or a military setting. In a corporate setting, objects are likely to be labelled in terms of their relation to a competency framework (Mulder, 1999) where personal authorship is of little importance. More fundamentally, there is considerable debate about the possibility of developing taxonomies that involve the same ontologies for different groups of users. Kraan (2003) notes that objects are “best described by using multiple vocabularies. There is no way to determine which vocabulary will be relevant to either an author or user of a given object…What may be a learning object to you, is a news article, archive context or a use case for somebody else. An object’s meaning, in other words, depends on its context of use”.

Berners-Lee, Hendler, and Lassila, (2001) in their work with “The Semantic Web” see ontologies as one solution to this problem. “Ontologies are a shared and common understanding of a domain that can be communicated between people and application systems” (Davies, Fensel, & Harmelen, 2003, pp. 4-5). Much of the current research on ontology development follows a rational approach (see for example, Berners-Lee, Hendler, & Lassila, 2001). Engers and Lech (2003) however note that “within current approaches to the Semantic Web, it is debatable what should be central –the human using the Web or the possibility of performing machine processing on Web content. In the former case, logical representations are probably not the most intuitive for use with humans, and different, more ‘cognitive’ representations of such knowledge might be more convenient” (p. 114).

However, even with tools focused on ontology development and a relatively well-defined knowledge domain (ontologies about skills, job functions, and education in a knowledge-management setting), Reimer, Brockhauser, Lau, and Reich (2003) point out that many human problems occurred when trying to use a Semantic Web approach to ontologies. Problem areas were a lack of domain experts to build the ontology, difficulties with ontology evaluation beyond a certain range of core concepts, and user difficulties in selecting the right concepts. Doctorow (2002) anticipates these problems when he notes that “there is more than one way to express something”. Another difficulty is the problem of “ontological drift” (Fensel, Stask, Studer, Harmelen, & Davies, 2003).

The latter see the combination of peer-to-peer collaboration and ontology development as the future: “Only by bringing together Semantic Web (specifically ontologies) and P2P (peer-to-peer) technology can we fully realize the potential…by giving participants freedom to use their own ontology structures” (p. 264). User-tailored descriptions for metadata are a form of peer-to-peer collaboration being studied in a number of locations. Recker, Walker, and Wiley (2000) describe an approach similar to that used on the Web in public sites such as Amazon Books in which patterns of choices and responses of users are used to identify which objects might be of interest to which persons. Called “collaborative filtering”, the approach involves “developing and evaluating a collaborative filtering system, which enables users to share ratings, opinions, and recommendations about resources”.

However, if such a system would be taken up in widespread practice throughout an organisation is not clear. An incentive for content specialists to take the time to add comments about a particular object is likely to be lacking.

With regard to incentives for the labelling of learning objects with metadata, a major issue is the amount of metadata that is feasible to expect, given the time constraints of those who enter metadata and given the interests of those who make use of the metadata for the selection of objects. Bois (2002) says that “all” that is needed is that learned societies develop domain ontologies, authors use the new tag editing application to complete their texts with tags, and retrievers use the new browsers that allow the selection of documents by specifying tag contents and relations. However, she acknowledges that while “this is simple it doesn’t mean that there is no effort” (p. 343). The effort involved needs organisational embedding and incentives in order to occur.

All of these problems have been studied for many years within the domain of information retrieval. Swanson, in 1988, summarizing 30 years of fundamental research on information retrieval concluded that:

“Our relevance judgements and our thinking entail, among other things, artful leaps of the imagination unconstrained by logic, reasoning, or the clammy hand of consistency; more important, they entail knowing who we are, what kind of world we live in, and why we want what we seek. It is hardly imaginable that a mechanism other than a human could acquire such self-knowledge, be given it, or do the job without it.” (p. 95)

This insight is not out of date; it is the basis of a new research line at the University of Twente in The Netherlands (Huibers, 2003). Due to the insight of this research as well as our on-going analyses of the impact of context and learning philosophy on the lifecycle of learning objects (Collis & Strijker, 2001-2002, 2002, 2003; Strijker, 1999, 2000a,b, 2001, 2002a,b, 2004[2]) we remain sceptical about how a focus on the Semantic Web or ontology development will act as keys to change the way people learn.

It is not that we are sceptical about the power of improving agents to select objects from the Web based on semantic approaches. The site KartOO (http://www.kartoo.com/en/servlet/H) for example shows that currently available tools can help locate and select objects but also expose a network that you didn’t know existed in terms of who is linking to objects you find particularly useful, something that goes beyond finding a particular object. There are new efficiencies, new power, new ways of thinking and “new forms of intelligence and meaning being added to display and navigation of context in the current World Wide Web” (Anderson & Whitelock, 2003). We encourage continued development toward these ends, but we are constrained by two sets of concerns: (a) the process should not be over formalized; and (b) intelligence and creativity are more important during the use process than during the find and select processes, and intelligence and creativity will come from humans, individually or collectively, outside of the Web (whatever sort, Semantic or World Wide). In a participation or contribution approach to learning, learning objects are only a tool; human processes involving communication, sharing, and collaboration are more important.

With regard to procedural/conceptual difficulties and the dangers of over-formalization, it appears to us that the Semantic Web as now described depends too much on a pre-formed structure; maybe finding this will succeed in certain cases, but for this to happen, too much must be organized, too many people (user groups, etc) must be in agreement about the structure, and a clear description in a shared language of the domain is needed. Shanks, Tansley, and Weber (2003) note that ontology theory requires the following rules when modelling a domain: “Composites and aggregates should be modelled as entities, not relationships, Relationship should not be modelled with attributes, Entities should not be modelled with optional attributes, Conceptual models should clearly distinguish between classes and instances, and Things and their properties should be clearly distinguished in the conceptual model” (p. 88). What does all this mean? Shanks, Tansley, and Weber continue by noting problems in practice in carrying out these rules, such as misclassifications and dual classifications. Putting groups together to form the ontology may be possible but requires too much discipline to be feasible in practice. Ontological drift and human drift will be unavoidable.

With regard to the underlying learning model, we recognize that in many cases knowledge transfer is the goal and thus an acquisition-based learning model is appropriate. However we agree with Euler (2003) that this is the lowest level of learning. In the knowledge-building and sharing model represented by the right-side of the Systems-Personal Model, the essence of learning is not so much concerned with finding or being presented with objects but in learning situations where collaboratively creating and constructing the objects may be a larger goal. We see this kind of learning occurring in a setting where a great deal of formalism isn’t needed to make sense of objects, because humans are around to supply the sense and be aware of the tacit understandings involved. A human-to-human “ontology” that comes from personal shared understandings and communication is not likely to be simulated/paralleled by technology. Thirty years of attempts to model learners for intelligent tutoring systems shows us the limitations of trying (Park, 1996). Even if we can find objects more quickly and more accurately doesn’t mean a higher-quality learning experience. For many types of cognitive development, finding and deciding about the appropriateness of knowledge is a major learning goal in itself, and striving for a situation where an agent or system presents “what you need” without mental effort or responsibility on the learner’s part will not even be desirable. We also agree that the use of technology in the form of agents and their capabilities never will and can replace human-to-human communication. “Human-to-human communication will always be a important component of the educational experience” (Anderson & Whitelock, 2003). The promises of the semantic web are high but the costs to achieve such a kind of automatism may be unobtainable in practice. Even more fundamentally, the focus on content may not be the solution for the needs of a pedagogy based on a participation or contribution-oriented educational philosophy (Anderson & Whitelock, 2003).

 

Thus the dissertation is over. But the issues and challenges will remain, as well as controversies related to how to interpret and deal with them.

 

---

[1] From Collis, B. and Strijker, A. (2004). Technology and Human Issues in Reusing Learning Objects. Journal of Interactive Media in Education, 2004 (4). Special Issue on the Educational Semantic Web. ISSN:1365-893X [www-jime.open.ac.uk/2004/4]

[2] This dissertation