Category: Dublin Core

  • A Manifesto for Managing Metadata in an Open World…

    Metadata is produced and stored locally, published globally, consumed and aggregated locally, and finally integrated and stored locally. This is the create/publish/consume/integrate cycle.

    Providing a framework for managing metadata throughout this cycle is the goal of the Dublin Core Abstract Model and the Dublin Core Application Profile (DCAM/DCAP).

    The basic guidelines and requirements for this cycle are:

    • Metadata MUST be syntactically VALID and semantically COHERENT when it’s CREATED and PUBLISHED.
    • Globally PUBLISHED Metadata SHOULD be TRUE, based on the domain knowledge of the publisher.
    • PUBLISHERS of metadata MUST publish the semantics of the metadata, or reference publicly available semantics.
    • CONSUMERS of published metadata SHOULD assume that the global metadata is locally INVALID, INCOHERENT, and UNTRUE.
    • CONSUMED metadata MUST be checked for syntactic validity and semantic coherence before integration.
    • AGGREGATED metadata SHOULD indicate its PROVENANCE.
    • CONSUMED metadata MAY be considered TRUE based on its PROVENANCE.
    • Locally INTEGRATED global metadata MUST be syntactically VALID, semantically COHERENT, and SHOULD be TRUE based on local standards.

    The DCAM takes as its base the rdf data model because of its simplicity and flexibility of that model. The DCAM refines the rdf data model in order to support syntactic interoperability between the rdf data model and non-rdf data models.

    A DCAP defines a distinct subset of the set of all things and defines the domain-specific knowledge of the properties of those things, and the relationships those things have to other things. It expresses that knowledge through the DCAM and a set of related documentation (the Singapore Framework). A complete DCAP should provide the necessary domain-specific infrastructure to fully support the create/publish/consume/integrate cycle for any system, any domain, and any data model. A DCAP based on the DCAM should be able to be used by a machine to generate a valid data model in any modeling syntax, and any modeling syntax should be able to be expressed as a DCAP.

    We strongly recommend that metadata be published and consumed using the RDF data model. The strength of the rdf data model for publishing and aggregating metadata lies in its extreme flexibility of expression, it’s self-describing semantics, and its assumption of validity. These strengths become weaknesses when creating and validating local metadata that must conform to a set of local restrictions. An effective DCAP will provide metadata creators with the ability to create locally valid metadata, publish it as rdf, validate consumed rdf against local standards, and ultimately integrate global metadata into the local knowledge base that contains local descriptions of the things being described.

    There are many systems, many data models, many publish and subscribe models, many storage and validation models. There are many paths to integration. There are very few that provide a generic and neutral method for modeling and documenting inter-system metadata integration. The DCAM/DCAP specification has the potential to be one of those few.

  • Using the sub-property ladder

    I discussed the utility of the sub-property relationship in Getting to higher MARC branches, Netting more MARC fruit, and Adding MARC fruit to the cornucopia. Coincidentally, Bob DuCharme posted Simple federated queries with RDF which outlines the same technique and provides additional information on its use for resource discovery. Those posts are somewhat technical, and I tried to lighten up in my presentation Turtle dreaming at the recent Dublin Core Metadata Initiative (DCMI) seminar Five years on. This post is another attempt to demonstrate in a non-technical way (I hope) how useful and powerful the sub-property relationship can be.

    A metadata attribute, like ‘title’, that is to be used for linked data in the Semantic Web is usually represented in Resource Description Framework (RDF) as a property. A property can be used as the predicate part of a triple: “Subject – predicate – object”, where ‘Subject’ is what the triple is about (e.g. a resource), ‘predicate’ is the aspect of the subject, and ‘object’ is the value of that aspect for the specified subject. For example:

    “This resource – (has) title – ‘Using the sub-property ladder’”

    is a single metadata statement in triple format. We can think of this as conforming to the triple template:

    “Specified resource – (has) attribute – value”.

    Note that prefixing the predicate with ‘has’ turns it into a verbal phrase and renders the statement in (near) natural language.

    We can also make meta-metadata statements in triple format. These are ‘data about metadata’ rather than ‘data about data’, and are often referred to as ontological triples to distinguish them from data triples such as the example above. The triple template for one type of meta-metadata statement is:

    “Specified RDF element – (has) relationship – Other specified RDF element”

    Note that a relationship between metadata elements is also represented in RDF as a property. In particular, ‘sub-property’ is a pre-defined relationship between two RDF property elements, giving the ontological triple:

    “Property 1 – (is) sub-property of – Property 2”

    Furthermore, such relationships can embed semantic rules that can be processed automatically by software known as ‘semantic reasoners’ or just plain ‘reasoners’. The rule embedded in the sub-property relationship is: If “P1 – (is) sub-property of – P2”, then any data triple using P1 as its predicate can generate another data triple using P2 as its predicate, with the same subject and object. Let’s call this kind of ontological triple a mapping triple, because it effectively maps one property to another.

    Suppose we have two attributes ‘title’ and ‘varying form of title’. I can create the mapping triple:

    “‘varying form of title’ – (is) sub-property of – ‘title’”.

    If we have a data triple:

    “This resource – (has) varying form of title – ‘Pat presents cataloguing for beginners’”

    then a reasoner will automatically generate the data triple:

    “This resource – (has) title – ‘Pat presents cataloguing for beginners’”

    In a similar way, we can create the mapping triple:

    “‘title statement’ – (is) sub-property of – ‘title’”

    and from the data triple:

    “This resource – (has) title statement – ‘Cataloguing for beginners’”

    generate:

    “This resource – (has) title – ‘Cataloguing for beginners’”

    So what? Further suppose that the ‘title’ attribute is from the DCMI metadata terms, and the ‘varying form of title’ and ‘title statement’ attributes are from the MARC21 tags 245 and 246. So a MARC21 record for the resource might contain the set of data triples:

    “This resource – (has) 245 [title statement] – ‘Cataloguing for beginners’”
    “This resource – (has) 246 [varying form of title] – ‘Pat presents cataloguing for beginners’”

    A reasoner can generate the set of data triples:

    “This resource – (has) [DC] title – ‘Cataloguing for beginners’”
    “This resource – (has) [DC] title – ‘Pat presents cataloguing for beginners’”

    In other words, we have generated a DC record from a MARC21 record. Or we have generated a title index for the MARC21 record. Or both.

    Let’s add an RDA attribute and an ISBD attribute mapping to the mix:

    “[RDA] ‘title proper’ – (is) sub-property of – [DC] ‘title’”
    “[ISBD] ‘has title proper’ – (is) sub-property of – [DC] ‘title’”

    The data triples:

    “That resource – (has) [RDA] title proper – ‘Cataloguing for geeks’”
    “Another resource – [ISBD] has title proper – ‘Does cataloguing have a future?’”

    can generate the corresponding DC triples, and we end up with:

    “This resource – (has) [DC] title – ‘Cataloguing for beginners’”
    “That resource – (has) [DC] title – ‘Cataloguing for geeks’”
    “Another resource – (has) [DC] title – ‘Does cataloguing have a future?”
    “This resource – (has) [DC] title – ‘Pat presents cataloguing for beginners’”

    So now we have a title index to metadata from multiple heterogeneous sources. And the beginnings of a set of records in Dublin Core format.

    Note that the attribute which is the sub-property must be entirely narrower in its semantics than the related super-property. If we create the mapping triple:

    “‘title’ – (is) sub-property of – ‘varying form of title’”

    then we generate the data triple:

    “This resource – (has) 246 [varying form of title] – ‘Cataloguing for beginners’”

    which is incorrect.

    As a result, a data triple generated by a sub-property mapping triple is usually ‘dumber’ than the original data triple; detail is lost because the generated triple uses an attribute which is broader in meaning than the original. This ‘dumbing-up’ is necessary to produce interoperable metadata from different schemas – but data is not permanently lost because the original triple is still available for use in other applications. Needless to say, data triples created with broad attributes cannot be “smartened-down”, at least on their own.

    The sub-property relationship can be chained. We can create a new attribute property, MARC21 ‘title’, which could be used in an application for making a title index to MARC21 records, as already mentioned. This new attribute is a super-property of all the MARC21 title-type attributes, and is also a sub-property of the DC ‘title’ attribute:

    “[MARC21] ‘title statement’ – (is) sub-property of – [MARC21] ‘title’”
    “[MARC21] ‘varying form of title’ – (is) sub-property of – [MARC21] ‘title’”
    “[MARC21] ‘title’ – (is) sub-property of – [DC] ‘title’”

    Doing this does not affect the previous mapping triples relating each MARC21 title-type attribute directly to the DC ‘title’ attribute, although it  makes them redundant because this new set of mapping triples generates exactly the same data triples at the DC level from the MARC21 originals.

    Different application can therefore re-use and, if necessary, augment the sub-property chains for each of the high-level core attributes found in most bibliographic metadata schemas, such as title, author/creator/agent, subject, target audience, etc. The chains form a net(work) of mappings, or map, which can automatically dumb-up triples from any level of semantic granularity to any higher level.

    We should only have to publish such maps or part-maps once, openly so that anyone can use them and add to them. If the professional communities develop the maps first, much effort will be saved and much authority imparted. This requires collaboration and action real soon now – the ISBD Review Group and the Joint Steering Committee for Development of RDA have started with the development of a mapping between the ISBD and RDA element sets.

    These maps should remain valid forever, so the effort is worth expending. The original data triples use the original properties based on the schema attributes at the time and they will be valid “for their time”, in the same way that many catalogues are likely to contain records created under the Anglo-American Cataloguing Rules, with its ‘general material designation’ attribute long after the successor standard RDA: resource description and access has been adopted with its ‘content type’ and ‘carrier type’ attributes.

    And mappings from the MARC21 element sets will show, we hope, that it may not be necessary to convert the entire contents of every MARC21 record as a result of the Bibliographic Framework Transition Initiative!

    But the professional communities lack a framework to help them collaborate as a super-community. A network of mappings is more (socially) efficient than an aggregation of one-to-one mappings between pairs of schemas. We need (name)spaces to add intermediary attribute properties and publish the mappings; we need protocols for managing semantic change as schemas evolve; we need lightweight protocols for authorizing mappings; we need systems for ensuring the long-term preservation of RDF element sets and mapping triples.

  • What DCMI Should Be Doing

    One continuing theme of the recently concluded DC-2010 is that of the perpetual search for consensus on what the hell DCMI should be doing. I know this continual search for identity is a common phenomenon with this sort of organization, as it is for the human adolescent hovering around the age of 15 years. Like with the teenager, it should be seen as a healthy thing, and as most of us older than 15 know, it pretty much goes on for the remainder of life.

    For me the conference was preceded by a half day DCMI Advisory Board meeting, where one topic was the revision of the DCMI mission statement as well as the perennial topic of the conferences and how the conference series can be optimally funded and continued, and what exactly is its value for the organization. As usual, there was not much consensus, either at the beginning or the end of the discussion, though it must be said that the conference itself probably shifted some opinions about the value proposition. Generally the AB meeting has been scheduled after the conference, with the idea that this shift in perspective is a good thing for sparking discussion, but for logistical reasons the meeting was held prior to the conference this year.

    As it turned out, this change in placement of the AB meeting was unfortunate, given that Mike Bergman’s keynote on Friday morning contributed some important outside opinion to that basic question of mission. (Mike’s post about the keynote is here.) The fact that Mike arrived in time to sample a good chunk of the conference and to talk to a variety of participants gave his opinion the credibility that only exposure to the culture of the organization and the personalities that affect that culture can bring to a keynote. It was clear to me, in talking to him during the conference, that his view of DCMI was not the insider view of a contentious, financially strapped and sometimes dysfunctional organization, but instead one that included recognition of the experience, knowledge and potential there as well. In a nutshell, Mike Bergman was telling us that DCMI’s role in the emerging linked data world was critical, and should be focused primarily on expanding the presence of useful semantics available to the the web world, closing the ‘semantic gap’ he sees limiting the growth of linked open data.

    Later in the day, my task was to lead a discussion of the work of the DCMI/RDA Task Group, of which I’m co-chair with Gordon Dunsire. Gordon and I had both prepared slides for that meeting—mine covering the history and work of the task group, what we’d learned, and what remained, and Gordon’s covering the related important work he’d been doing in parallel with IFLA. I’ve been frustrated for some time with the lack of attention and traction we’ve received for this important work, both from DCMI and the Joint Steering Committee for the Development of RDA (JSC). We have found ourselves at the stage where DCMI is waiting for JSC to make some statement about the work done (in the form of approval), and the JSC is waiting for DCMI to endorse the Task Group’s assertions made about the work done and its usefulness for the Semantic Web—or at least this is the way it seems from the point of view of the co-chairs. It’s as if both groups are standing opposite one another in a middle school gym, the ‘boys’ and the ‘girls’ waiting for someone to move towards the middle. Nobody seems to want to make that first move, and though in the case of the TG, each conversation with representatives of both parties seems to be positive, resolution of the concrete issues moves at a frustratingly glacial pace.

    But as I spoke about this work to various people, continuing to think about the ongoing conversation about what the role of DCMI should be, particularly in the context of Mike’s keynote, it struck me that the DCMI/RDA Task Group was in some sense a model for what DCMI could do to fulfill the role Mike saw for us in the world. In essence, the TG came into being because Don Chatham at ALA Publishing took the initiative to bring DCMI and the JSC together, where the DCMI message was “How can we help?” The rest is history, but we seem not to have learned from that how powerful that simple question is, and where it could lead. DC-2010 brought a number of new communities to the conference, representing a variety of groups interested in moving into the wider web world of information, but lacking in-house knowledge and skills necessary to make progress. Helping them move forward requires much more than attracting them to the conference and talking to them in the hall or after tutorials. We need to offer more concrete help, like we did for the JSC, and move the knowledge and experience that the DCMI Community has assembled into the broader world of information.

  • Live (ish) from DC-2010

    This morning’s highlight was Stu Weibel’s opening keynote address to the assembled conference attendees (yesterday included primarily workshop sessions and tutorials). Stu was asked to talk about DC’s past and future, and he gave many of us food for thought.

    The first thing he did was typical Stu—he took a photo of the assembled group. I hope those will show up on his blog sometime soon (if so I’ll link to them). Will they look that much different from those he took 15 years ago, when Dublin Core began, or those taken at various points along the way? As is also typical, he asked how many people were returning participants in DC, and how many were brand new to the DC conference. Surprising to some of us, roughly one third of the group were new participants, a nicely healthy proportion of new people.

    Stu asked some interesting questions, and gave DCMI some letter grades for performance in a number of areas. His first question, “Why didn’t we just stop after the 15 elements” suggests the possibility that nothing done since then (the mid-nineties) was worth the effort. He points out that a number of the assumptions made at that time have since been repudiated by experience—the Web is more than just a collection of document-like-objects that can be described in much the same way that we’ve traditionally described library materials. He got a laugh when he reminded us (including the three of us who were actually there in Mar. 1995 when DC was born) that we thought we could solve the syntax problem fairly easily—but of course our notion of syntax in those days was HTML.
    Stu gave some personal assessments of the 15 years of DCMI in the form of grades:

    For providing an international basis for the effort (A)
    For including a diverse group of participants (B+)
    For becoming a sustained and solvent organization (D)
    Moving work from consensus to completion (C)
    Establishing objectives and completing them (C-)
    Documentation of decisions (B+) (I have to say I think he was somewhat over generous on this one, for reasons I’ll explain later)

    Stu also pointed out the places DCMI got stuck along the way, among those were ‘tarpits’ (his word) of our own making—for example, data models (for which he gave DCMI an ‘F’). Those that were not our fault or not entirely of our making were things like the aforementioned syntax confusion (C+), which he believes stemmed from trying to do too much and getting overwhelmed (probably the rapid evolution of syntaxes had an effect, too). The ‘tarpits’ created by others included LOM (learning object metadata, promulgated primarily by IEEE), and INDECS (a now dead effort that was touted some years ago as a business model based approach).

    Some other points Stu made which are hard to argue include the notion that cooperation with organizations with different business models is difficult, but such cooperation is critically important, for reasons around convergence of effort, identification of similar models and related technologies—all amplifying the network effect of what DCMI has done. Increasingly, cooperation is seen as an expensive value, particularly in terms of time and travel, and certainly DCMI has seen those issues having a big effect on conference planning and ability to build on past efforts. Stu also gave DCMI an ‘A’ for its standardization efforts, including the work to make DC a recognized standard via IETF, NISO, and ISO. He pointed out that these efforts were essential to allow DC to be adopted by government agencies and others that have requirements for such an endorsement.

    On the Singapore Framework, Stu is dubious, in particular thinking of the four levels of interoperability [link] that include the Description Set Profile (DSC) and the Dublin Core Abstract Model (DCAM). He pointed out that lots of metadata is still used primarily at the lowest, human level, and is not yet useful to machines at the moment. This issue is particularly relevant given that the DCAM is currently under review, with opinions flying around (in the halls, on twitter, etc.) in ways they didn’t even when that document was new, and pretty much nobody understood it. Understanding is still an issue, to a great extent because specification and user documentation are not the same thing (something that the DCMI technical folks don’t seem to understand). Stu contends that the DCAM has failed, and felt that its authors still don’t agree about its motivations or implications. It will be interesting to see whether that assessment is widely held throughout the conference.

    On linked data, Stu was clearly somewhat ambivalent, calling it “An aspirational target of great promise and unproven benefit”. Stu was around and in the fray when the RDF standard was still in diapers, and recalled for the audience how easy we all thought it would be then to bring its promise to fruition. At least a decade later, we’re still trying to do that.

    In Stu’s opinion, the Web is the data model, and we shouldn’t deviate from it. He pointed out that with the issues of flexibility vs. constraints we are drawn in by the Siren Call of Flexibility but would be better off with more constraints.
    On the positive side, Stu points out that the linked data bulge has brought us a strong commitment to identifiers, some useful conventions about vocabularies and syntax, some tools to build ontologies and models, and some expectations of utility due to broad adoption – network benefits, in other words. But we still need to worry about data quality, usefulness, and bridging the boundaries between the existing semantic communities.

    Stu is skeptical of linked data as the new grail, but still thinks we’re on an exciting threshold—we need metadata more than ever, but we’re drowning in it. He quoted Tom Baker: “Data that cannot speak for itself will be more vulnerable to becoming irrelevant”.
    Stu’s last points covered DCMI as an ongoing experiment in social engineering. He cited Malcolm Gladwell’s article in the Oct. 4 New Yorker where Gladwell asserts that social media are largely broadly disseminated, networked, weak tie activities, with low barriers, low commitment, low persistence. Gladwell contends that systemic change requires strong ties, hierarchical social structures, leadership organization, and f2f work. Stu believes that DCMI is a strong tie phenomenon, and its impact is amplified by this fact.

  • Why I Still Go to DC Conferences …

    When I was teaching my RDA course this past Spring, with my virtual students so very present in my thoughts for that time period, I found myself trying to explain to them why I still go to Dublin Core conferences, after all these years. I am one of those people who was around at the birth of DC, back in the dark ages of 1995, and aside from the conferences added retrospectively to the series and the one in Seoul last year, I’ve been to every one. This year, I’m co-chair of the program, something I’ve not done before, and more involved in the internal workings of the conference than ever. I’m sure that I didn’t make much sense to those students, who likely ascribed my long tenure to habit, loyalty, or some other factor.

    This year the conference is finally back in the US after many years absence, and the program is looking pretty good, if I do say so myself. As usual there’s a mix of tutorials, papers, and working meetings, and this year the conference is a bit shorter than usual, in an attempt to keep costs down in these hard financial times. It’s also being held in conjunction with ASIS&T, which is being held in the same hotel right after DC-2010.

    So why do I keep going? The other conferences I go to as a participant (as differentiated from the ones where I’m an invited speaker) are really down now to two: ALA Annual and ALA Midwinter. They’re big, sprawling, every-librarian-for-her/himself, with thousands and thousands of librarians taking over some hapless city, every restaurant in town, and just about every hotel. DC is a very different kind of thing—it’s small (usually no more than 200 people), generally not more than 50% librarians, if that, and the rest a mix of researchers, implementers and software folks. Quite a few people straddle more than one category, and the group provides an experience that I’ve never found anyplace else. The conference is very international in flavor, and the focus is metadata, and more metadata, unlike, say, JCDL or other conferences with a computer science or web focus, where metadata is a very small part of the program. It’s a place where the metadata geeks at the edges of other communities can feel at home, even when they feel marginalized in their own home communities. I remember one year when a guy I know who had never attended a DC conference before buttonholed me in a corridor to tell me this was the best metadata conference he’d ever attended. It still is.

    I always learn new things at the DC conference, always meet a least one new and interesting person (and usually more than one), and there’s more good talk in the hallways and bars than can possibly be taken in. I’m an extrovert, so conferences jazz me up—I relish the intensity, the ongoing conversations from year to year, and the real sense of community, no matter the strong differences of opinion and approach.

    I’ll be in Pittsburgh this year, and am hoping to blog a few times while I’m there, if I can manage to find time for it. I hope I’ll see a few of you there too.