Category: Futures

  • The RDA Vocabularies, Continued

    One of my favorite aphorisms is “Time flies, whether you’re having fun or not.” I’m not sure where I heard it, but for sure I’m not creative enough to make it up on my own. The truth of it has been reinforced by the realization that here it is the end of January, post-Boston Midwinter, and I’ve done so little blogging for the past six months that it’s a stretch to call myself a blogger. Time to reclaim the turf. So this post is an attempt to summarize what I’ve been doing all that time, some of which has come to a sort of fruition, but some still ripening.

    Last fall I participated as a speaker in a NISO Webinar “Bibliographic Control Alphabet Soup.” I decided for my topic to talk about some of the issues around building the RDA vocabularies from spreadsheets and ERDs (Entity Relationship Diagrams), which is what I had to work with on that task. (You can see the ERDs on the RDAOnline website). Part of my reason for trying to tackle those issues in the webinar is that the vocabularies had become a major focus of my working life for quite a while (does the word ‘obsessive’ sound too dramatic?) At the time, we (Jon Phipps, Karen Coyle, Gordon Dunsire and I) were also trying to write an article about what we’d done with the RDA elements and vocabularies, and that article came out last week in DLib Magazine. Starting the article last fall prompted me to create some diagrams in an attempt to try and convey the structure of these vocabularies, and to provide examples for folks to look at while they puzzle through the ideas. I used a subset of the diagrams in the webinar as well, but most of them are used to better purpose in the article. I’m not at all sure that the several hundred folks listening in to the webinar got much of what I was trying to convey—it’s pretty new stuff for most people, and I was trying to fit too much into my limit of 20 minutes (not nearly enough time). I hope that for those who might have been overwhelmed or confused by the webinar, the article will help to make the work we’ve done a bit clearer.

    The writers of the article had a number of purposes in mind, not least to document the decisions and rationale for the strategies taken by the DCMI/RDA Task Group charged with the work of building the vocabularies. Given that the library community generally has little experience with RDF and RDF vocabularies, it seemed particularly important to attempt to provide explanations that we hoped would be accessible to most librarians, and I hope we’ll have sufficient feedback to determine how close we came to that goal. Given that we expected the article to come out just after Midwinter, we did most of our presentations in Boston on the implications of the vocabularies rather than the mechanics. I talked about these at the Technical Services ‘Big Heads’ meeting (slides) and Jon, Karen and I included some of these expectations in our introduction to application profiles at CC:DA on Monday (slides).

    Just before Midwinter we started a dialogue with the JSC about next steps, and I hope that some of the issues that have come up will be open to public discussion, just as the vocabulary building was done in public. We’re all in a fairly intense learning space at the moment (at least it seems so to me), and keeping the process open and visible for all seems in support of that learning. We’re also continuing to update the vocabularies as errors are brought to our attention, and to complete portions where we need to add information. One such point in the property/subproperty hierarchies in the element sets—at the time the work was done there was a limitation in the Registry software that prevented us from including the proper hierarchies in both the general and FRBR-bounded portions of the vocabularies. That limitation is now removed, and the missing relationships will be added.

    Interestingly, we’re getting lots of help in finding errors from our friends in Germany, who are doing nifty things with the vocabularies. They’ve been particularly helpful with things like inadvertent spaces introduced in URIs and other things difficult for a human proofreader to find. Because we don’t have good ways (yet) to visualize the relationships, their help has been invaluable. We urge anyone else who spots an error or who has a question to use our feedback button in the Registry to communicate with us about their concern.

  • Bugs in Amber

    A few weeks ago I attended the opening of an amber exhibition at our wonderful Museum of the Earth which is only about 6 miles from my house. The exhibit had a little of everything: science, history, geography … and jewelry. I have to admit (and this will surprise no one who knows me) that the jewelry was a big draw, and I went laden (literally), with a varied selection of my own collection of amber. Hey, laugh if you will, but these days I work at home, and have very few opportunities to wear jewelry of any kind—so this opening was irresistible.

    But, enough about jewelry, I want to talk about bugs and bibs! As you might expect in a science museum, there was far more emphasis on amber as a carrier (so to speak) of bits and pieces of the past, particularly the biological past. As a preservation medium, amber is hard to beat, though, of course, there are limitations in terms of the size of the biological specimen. I didn’t realize it, but apparently fake amber is everywhere, and one way to recognize the bio fakes is that they include specimens too big to be slowed down by sticky tree sap. The exhibit had some nice fakes, including a small snake in plastic colored to look like amber.

    The interest of the scientist in amber is that it stops the process of decay for those creatures lucky (or unlucky) enough to be captured in its grasp. The amber captures a moment in a bug’s short life in a way that allows us to examine it closely and in detail in our own time, millions of years later. In much the same way, the Study of the North American MARC Records Marketplace by R2 Consulting captures a moment in time, very likely too late to have much of an effect on the future, but just in time to capture the state of the cataloging world before the tsunami arrives.

    But the R2 report is as fascinating to a metadata maven as a bug in amber is to a biologist. It describes in detail the current world of cataloging distribution, focusing on the “dysfunctional market” that has grown like Topsy around distribution of MARC records. It gets exactly right the disconnect between the librarian sense that “records want to be free” and the business approach that production costs must be recouped and profit margins maintained for there to be any point in participation at all, and comes down predictably in support for the latter view.

    It’s a fascinating read, particularly if the fact that LC commissioned the report is kept in mind—because this is hardly a context-free analysis. I was particularly interested in the description of the businesses outside of libraries supplying MARC records either as contractors or as part of a materials supply chain. As a former denizen of one of the large academics that R2 identifies as part of the “green tier” (more about that later), I was aware of the fact of that portion of the MARC marketplace, but had little contact with it.

    The gist of the report is that the MARC distribution network is a dysfunctional hybrid, partly librarianishly “free” and part commercial marketplace. The authors feel that it should be possible to increase the supply of MARC records from “the community” without relying on poor beleaguered LC to supply them, and they give us a multitude of statistics to support that assertion. They believe that there’s enough time to accomplish this and save everybody money before the promised changes come to pass, and all must be re-thought.

    My comments on this report, informed by my well-known biases, fall into a few convenient categories:

    Dysfunctional? Probably …

    Much of the first portion of the report is devoted to a description of the current “marketplace” and a discussion of the survey results that illuminate and inform the description. It’s here that R2 makes the case that LC is subsidizing the whole shebang, to the benefit of everyone else.

    “Both libraries and vendors (at least the good ones) rely on “service” to their respective clienteles to distinguish themselves, but there are important distinctions in their respective definitions of the term. In the commercial world, service must exist within a context of profitability, in which all costs are covered and some additional increment is contributed to the company’s continued growth and as a return on the capital initially invested. The library service ethic is much more open‐ended and less directly constrained by costs.”

    The report contains much interesting description of what the authors perceive as the bifurcated market, one which, in their view, inhibits the growth of useful marketplace incentives to increase output:

    “This tension ‐‐ between community values and commercial values, between idealism and pragmatism, between social responsibility and private benefit – has deeply affected some aspects of the library market. Cataloging, regarded by many as the heart of librarianship, is one of those areas.”

    It’s pretty clear where the authors come down in this conflict between “community” and “commercial” values:

    “The impulse to share records for which the costs have not been fully recovered may make sense as a form of community good, but is not sustainable without some form of subsidy or exchange. From the commercial viewpoint, it’s simply bad business.”

    And, perhaps more to the point:

    “It should not go unnoticed that LC itself provides open access to its MARC records via multiple channels. The prevalence of open databases is a key factor in the economic confusion that plagues the MARC Record Market … “

    The report goes on to a rather interesting and revealing categorization of the complex MARC marketplace into three tiers. The “Green Tier” includes the “ … oldest, most traditional segment of the market, in which nearly all MARC records originate.” This tier includes both libraries and businesses, as well as OCLC, and is, as such, a mix of the “community” and “commercial” as described earlier. The big thing is that they’re contributors to the marketplace, even if also consumers. According to R2’s statistics, this tier includes 97% of academic libraries, 63% of public libraries and a similar proportion of school libraries.

    The next tier down (and it’s clearly down, in this categorization) is called the “Blue” or “opportunistic” tier, including by the author’s definition “ … non-OCLC libraries and underfunded libraries without adequate cataloging capacity.” More interestingly, this tier “ … is also home to open database providers, and the pervasive (did they mean to say “pernicious”?) Z39.50 protocols used to locate and obtain MARC records free of charge.” But R2 makes note of the shifting borders between tiers: “Both in Canada and in the US, historically ‘green libraries’ are adopting ‘blue tier’ practices and expectation, as library budgets are cut and as Z39.50 targets proliferate. Nearly all libraries, regardless of size or type are strategically patient, periodically re-searching the ‘blue tier’ for certain records to become ‘available’; but for ‘blue tier’ libraries, this is the primary approach to cataloging … Open Access and Open Archives Initiatives reside in the blue tier, strongly supported by the basic philosophical stance that access to information should be free.”

    The “bottom” tier is the non-library “purple” tier, and this description clearly defines the real threat to the current MARC world, not just the fuzzy-wuzzy library community notion of sharing: “The non-library (purple) tier operates to a large extent without appreciation for or experience with MARC records, and without much regard for the library market in general. It is important to remain aware of activity in this segment, of course, because developments here pose the most significant competitive threats to the traditional values and economic structures of the ‘traditional green tier,’ and even the ‘opportunistic blue tier.’ This is the place where newer technologies and non-MARC data formats are used and developed.”

    Obviously, we have met the enemy of libraries, and according to R2 it happens to be us. But wait, there are some unexpected companions in the nasty “purple” tier. In addition to the usual suspects, like Google and Amazon, we find … “OCLC pro-actively operates within the “traditional green tier” and within the “purple non-library” tier. OCLC member libraries, however, are also very active in the “opportunistic blue tier,” sharing records in ways that may conflict with OCLC’s proprietary intent.”

    The battle lines seem clearly drawn here, with the “information wants to be free” crowd clearly the enemy, whether in sheep’s clothing as traditional librarians or explicitly displaying wolfish teeth as a member of that unappreciative crowd that cares little about the current MARC marketplace and would like to see the library data silo dismantled brick by brick. No matter that we seek these changes for the benefit of libraries struggling to live within their budgets and to innovate to serve their users as well–shame, shame!

    The R2 Solution

    The report’s authors actually manage to ask THE most relevant question that should be (and often is) on our minds, but only to dismiss it as out of scope:

    “The practice of cataloging has never before faced the level of scrutiny it now enjoys … or endures. Two types of question predominate. First, are traditional cataloging and the MARC record—even after modernization by RDA and FRBR—still necessary in an era of full‐text indexing, OpenURL linking, and other discovery options? While this is a worthy question, it is fortunately not within the purview of this report.”

    Leaving aside the odd assumption that RDA and FRBR represent the “modernization” of the traditional MARC record, they couch the issue only in the context of a limited number of technologies, never mentioning the gorilla in the room, the data being built by others outside our comfy and bounded silo. Then they go on to pose the questions they would rather address:

    “How do we as a profession understand and explain the costs and benefits of producing and distributing cataloging records? Where and by whom are most original records produced? What incentives exist to stimulate production? What are the barriers that discourage production? How does the library market assign value to the work of cataloging? What is the return on any organization’s investment in producing original catalog records? How does shared cataloging and free or low‐cost distribution of records affect the market? To what degree is market activity subsidized by LC and by the work of individual libraries?”

    The problem is, that without an answer to question #1, the other questions seem hardly relevant.

    “As noted there, the market is in need of adjustment, if it is to create an incentive for producers while retaining the community ethic of free sharing of data. The ethic of the cooperative can only be sustained if the full costs of production are borne by the community.”

    It seems to me that the market will be adjusted, and the recognition of the full costs of traditional cataloging and the plunging ROI as we address Question #1 will hasten that readjustment, but probably not in the direction R2 predicts or that those seeking compensation for their MARC record production might want.

    The authors provide some telling glimpses into their world view in their discussion about crosswalks:

    “ONIX to MARC record translations and fully operable MARC to non‐MARC metadata crosswalks could dramatically alter this three‐tiered landscape. To date, major players in the blue and purple tiers have failed to buy into the concept of shared bibliographic and authority data. While some efforts to encourage cross‐market cooperation are underway (notably the OCLC/NISO forum), fierce competition flourishes within and between each tier of the market. Even more problematic, each tier has distinctly different needs and incentives, making it difficult to establish an adequate degree of shared urgency and/or investment in new solutions.” [RIN]

    Clearly, in a world where the only relevant data one can see “out there” is ONIX, crosswalks seem a no-brainer, but to call this view “limited” seems far too kind.

    Ultimately, R2 thinks we still have time to tweak the marketplace and flog out more MARC records by identifying and marshaling unused capacity (e.g., hidden catalogers) and providing economic incentives. In my view, this is a flawed argument, and takes away from the need to plan for the transition to a much different future. I agree that MARC will indeed be used by libraries for some time, but as a lossy exchange format, not the lynchpin of the library data world. R2’s strategy prolongs the old world, jeopardizing the possibilities of moving forward in a timely manner.

    The Sacred Cow Effect

    Sadly, the whole report, interesting though it is as a biological specimen, fails utterly to examine the data activity outside libraries except to demonize it and its proponents. In making the Library of Congress into Poor Nell, they also deny the innovations in creating and reusing data that LC itself has accomplished, for instance, the American Memory Project, the LC Flickr Project, and many other digital initiatives that have proactively (and openly) pushed the metadata envelope in ways that inspire and engage us. The report fails also to understand that the changes they fear, the ones that they rightly expect to undermine the current marketplace completely, are already nibbling ravenously around the edges of MARC and its traditional marketplace in ways that will hardly take the 5-10 years to make change become real that R2 predicts.

    Last summer at ALA in Chicago, a small group of us pulled together a linked data program, hearteningly well attended, where Eric Miller persuasively predicted that the return on investment for integrating “free” metadata from “the cloud” will trump traditional concerns about quality. [Miller] Mainstream entitles like the New York Times are moving aggressively into the linked data space, seeking to merge their data with the likes of DBpedia and FreeBase. [Sandhaus]

    Consider this from MMA partner Jon Phipps: “The future cataloging marketplace will have to compete with ‘free and more than good enough’. Like the people who initially sneered at Google for being too simplistic and ignoring metadata when it came to searching, the professional cataloging community ignores (or tries to fend off) the enormous future output of Linked-Data-enabled systems at its peril. By opening up a clear relationship between the semantic web and library data sets, the RDA vocabularies represent a threat to the hegemony of catalogers. The RDA vocabularies are a a disruptive, game-changing technology.” [Phipps]

    The reality is that it’s not just the marketplace that’s changing, it’s also the profession. As part of the analysis of why the numbers of catalogers reported in their survey doesn’t lead to the expected output levels, R2 speculates that “These data lead us to ask what catalogers are doing. Bob Wolven and others suggest that catalogers are being called upon to apply their knowledge of cataloging principles to new initiatives; and specifically to creating metadata for digital and archival collections.” [Wolven] R2 seems to imply that this is a bad thing, taking away resources from the business of actually churning out MARC records, but certainly these newer roles are critical to the survival and renewal of libraries, far more than shoring up current MARC record production.

    The solutions the R2 report poses, from paying more attention to recouping cataloging costs and re-centralizing creation of cataloging records, if taken up, would actively undermine a transition to participation in a more open, linked data world. They represent a step backward, in a community that has already internalized the values of sharing and decentralized data critical to seeing value in the world of openly accessible data lying on our doorstep.

    Oddly enough, the report ends with a quote from my old friend Sherman Clarke (unattributed, so most likely as a comment to the survey):

    “We collectively need to have a model that allows us to do some of the building of BIBCO records mechanically or through accretion of metadata from institutional records or other record loads. OCLC already does considerable building of the master record from incoming records; what we need is something more like the metadata that is becoming usual in NewGen environments. If someone adds a tag or review or picture, that becomes available in the master cluster. Not a BIBCO record, but a BIBCO cloud of metadata for a particular manifestation of a work/expression.”

    Yup, you got it, Sherman. The change we need is not really about records, or catalogers; it’s a new way to think about information and added value.

    [Miller] Miller, Eric. “Linked Data and Libraries: Grassroots Program: From Legacy Data to Linked Data, Preparing Libraries for Web 3.0. Available at: http://zepheira.com/talks/ala-em-lod.pdf

    [R2] Study (for the Library of Congress) of the North American MARC Records Marketplace, October 2009, R2 Consulting LLC, Ruth Fischer, Rick Lugg. Available at: http://www.loc.gov/bibliographic-future/news/MARC_Record_Marketplace_2009-10.pdf

    [RIN] Research Information Network. (2009). Creating catalogues: bibliographic records in a networked world. Available at: http://www.rin.ac.uk/files/creating_catalogues_REPORT_June09.pdf

    [Sandhaus] Sandhaus, Evan. “150 Years of Semantic Technology.” Presentation at the Cornell University Libraries Metadata Working Group Forum, Nov. 13, 2009. Slides will be available from: http://metadata-wg.mannlib.cornell.edu/forum/index.php?date=2009-11-13

    [Wolven] Wolven, Robert. (2008). In search of a new model: Columbia University Libraries: Robert Wolven reflects on what’s next for cooperative cataloging. netConnect, 1/15/2008. Available at: http://www.libraryjournal.com/article/CA6514925.htm

  • FRBR vs. FRBR-ization

    ALA Annual in Chicago has been a blur—I did three presentations (which I hope to talk about and link to slides as time permits). But one issue has been rolling over in my mind ever since I blurted something about it at my first presentation on Friday of Annual, when I was last up on a panel about “The Future of MARC.” Rebecca Guenther of LC spoke about the efforts to keep MARC relevant and Ted Fons of OCLC covered similar topics from the viewpoint of “The Big O” (thanks to Karen Schneider for that wonderful appellation!) A feature of both talks was the idea that reorganizing MARC records into “FRBR-ized” views was really all that was needed to take advantage of FRBR. I argued at that time that this was not the case, and as I think more about it I’m more convinced it’s true: FRBR-ization is not the same as using FRBR in native RDA.

    Part of my view is based on the differences between RDA and MARC semantics (not syntax, which is where the conversation usually goes). One of the most overlooked aspects of RDA in general is the rich vocabulary of relationships that it brings to the table for use in bibliographic description. Most people who’ve focused on RDA as a textual guidance or set of rules have overlooked this, because the relationship vocabulary appears in appendices, and most of us don’t consider appendices the most important part of anything. But consider this: in the RDA Vocabularies, each of these relationships has an identifier, is part of a hierarchy that allows expression of bibliographic relationships at several levels, and gives us the ability to use these relationships to navigate the bibliographic landscape without having to delve into records and interpret the text notes we’ve used for the same purpose in MARC. For instance, using RDA you can say that ‘Resource X’ is an abridgment of ‘Resource Y’ (and that ‘Resource Y’ has an abridged version in ‘Resource X’) in a way that a system can expose to the user with no muss or fuss. The relationship is specific, identified and explicitly defined if anybody needs that to apply or interpret it.

    In contrast, FRBR-ization only exposes what we can assert based on a mapping from MARC to FRBR (or RDA), which is at best the relationships between the FRBR Group 1 entities: the Work, Expression, Manifestation and Item. With the RDA array of identified relationships, we have a whole lot more. I suppose one could say that these are not necessarily part of the FRBR panoply, but if you consider them the “horizontal” relationships that fill in between the “vertical” relationships that Work, Expression, Manifestation and Item provide, then it’s possible to see how these relationships are enabled by the way the FRBR model has allowed us to rethink our world.

    This is one of the issues that makes my head hurt when I think about the RDA “testing” regime that we keep hearing about. Are we wedded to the notion that if it can’t be crammed into MARC we aren’t going to use it? Can’t we start to think about MARC as a fairly lossy output format and move on to something that expresses the relationships we know will help us maintain some important functionality and credibility in the broader data world? As Jennifer Bowen and the eXtensible Catalog folks have discovered as they build the services to transform MARC into RDA (see my post on Jennifer’s paper for more about that) transforming MARC to RDA represents a fundamentally different set of problems and trade-offs than going the other way. [By the way, the XC Project was everywhere at Annual–go Jennifer!]

    And as more vendors step up to the RDA plate and begin to build applications that start with RDA rather than try and transform MARC into something that could be mistaken for RDA only in dim light, we’re going to have to accept the fact that like any other metadata mapping, there is no such thing as a free lunch or a round trip.

    The Registry has some of these relationships registered already (see “RDA Roles” for the relationships between FRBR Group 1 and Group 2 and “RDA Relationships for Works, Expressions, Manifestations, Items” for the relationships between Group 1 entities), but be aware that these are not yet the final versions. I haven’t gotten the information yet about the final changes to allow me to make those updates, but when I do I’ll make an announcement to that effect.

  • Testing! Testing?

    Today I got a very disappointing note in my inbox, from the US National Libraries RDA Test Project. I guess I’d call it a “ding” letter, and I have to say it was more than a bit surprising. I had volunteered to help with the testing, not by creating records, mind you, but in analyzing the records other people create. Given the fact that I’ve been the co-chair of the DCMI/RDA Task Group, done the major part of the work in registering the RDA schemas and vocabularies, and have been involved in building the XML schemas that will be the basis of much of the data creation for many early RDA implementations, I figured my experience might come in handy. But apparently not …

    Dear Diane: Thank you for your interest in the US National Libraries RDA Test project. The RDA Test Steering Committee regrets that you could not be selected as a formal test participant. Interest in the project was much greater than the Steering Committee originally anticipated, and it was necessary to select test partners from more than 90 applications. Every applicant had a great deal to offer to the project, and each was carefully considered. The Steering Committee based its final selections on the goal of ensuring that the RDA Test will reflect a cross-section of US cataloging agencies balanced by size, type of organization, OPAC and cataloging systems used, and areas of specialization in cataloging and collection development.

    The Steering Committee will share the methodology for the test on its Website at URL . If you are interested in conducting your own test of RDA, we encourage you to produce records following this methodology and to share the results with the Steering Committee during the test period.

    Thank you again for your interest in the RDA Test.

    So, exactly what are they testing that makes my knowledge and experience useless? Darned if I know. But I can’t get beyond the notion that the testing regime I see described on the website is pretty limited, and it’s hard to imagine what the results can really tell us, aside from the obvious difficulties people will encounter in attempting to cram a FRBR-based structure into any one of our current flat MARC-based library systems.

    Much more interesting, to me anyway, is the idea of what RDA records might look like in straight XML or RDF, without the necessity of the contortions involved in making it all “fit” into a MARC system. Without the layer of MARC contortion we might really be able to figure out whether catalogers could adjust to RDA and create FRBR-based records. It would be nice to think that some of the open source systems would find a way to play with these records and test some more forward-looking, rather than backward-looking implementation issues.

    Any volunteers for an alternate testing regime?

  • On the Road at NJLA and the Five Colleges

    This week I’ve been on the road, doing presentations at the New Jersey Library Association meetings on Wednesday, and Five Colleges in Western MA Friday morning. I’ve been doing all this travel in the faithful MetadataMobile, increasingly an object of interest, amusement and (almost) veneration by those who have heard about her. It’s been a great trip, with wonderful audiences, and as always, I learn a lot about what’s on the minds of those in the trenches.

    I arrived later than I’d expected at NJLA, due to missing the exit I was supposed to use on the Garden State Parkway. I was very busy looking for a coffee and a rest stop, and it probably whizzed right by me. By the time I realized I was way too far down the Parkway, I decided to get off and use my newly acquired New Jersey Official Map to get me to my destination. I know, it’s very retro as a strategy, but maybe I was feeling that a little self-punishment was in order, or maybe I just needed to get off the big roads for a while.

    As a child of the fifties, my first stop was a gas station as I needed to gas up and figured I’d confirm my location on my handy-dandy map and get some advice on my route. I had forgotten that New Jersey doesn’t do self-service gas pumping, and I’d also forgotten that virtually all the NJ gas stations I’ve visited in the past decade are staffed by (and possibly owned by) persons from somewhere else. When shown a map and asked for directions, they invariably act like people who have been lately dropped into their current location from the sky with no idea how to locate themselves in the world they inhabit (pretty close to reality, probably). I knew pretty much where I was, but wanted to figure out how to get to a big secondary East/West road that would take me to where I needed to be. It was hopeless—these guys (they’re always guys) couldn’t even locate the town they were in on my map so I thanked them for their trouble and got back on the road. I ended up following my nose (and the MM’s internal compass, which tells me whether I’m going N, S, E, W or any rational combination thereof). Thankfully, in NJ if you go east you find the Atlantic eventually.

    It was a gorgeous day to be taking the long way, and I passed by what remains of the agricultural areas in NJ, including a whole bunch of horse farms (one was called “Due Process Stables” and you could pretty much guess how that was funded). I got to the conference venue in Long Branch (right on the beach) in time to take a nice walk on the boardwalk and decompress from those long hours behind the wheel.

    The program was about FRBR and RDA for real people, and Rhonda Marker from Rutgers presented first. Rhonda and I have presented together before, and generally have fun with it, since we sometimes disagree and are very open about it, to the general amusement (or maybe consternation) of the audience. We only had two hours, which wasn’t enough, but the audience was very receptive and had some good questions. The slides for this presentation are available here.

    From Long Branch, NJ, I pointed the MetadataMobile north, through the horror of Big Apple driving (BIG trucks!), though Connecticut and up to Amherst, Mass., for a confab with a group from the Five Colleges. This had been in the planning works for some months, but was made more interesting by the announcement a few weeks ago of some pending reorganization of technical services operations in the colleges in response to the same kinds of financial pressures being experienced everywhere. I’d been forewarned about this by the organizers of this session, and included some of the issues most relevant to the group in my presentation (though the slides, available here don’t always reflect this directly). My underlying point was that they were not, in fact, dealing with only one crisis—the financial meltdown—but two, if you include (as I do) the pending changes in how we do business as data creators and managers, changes that are absolutely necessary to avoid the continuing marginalization of our efforts to provide information to our users.

    One thing that impressed me about this group was that it included a number of the library chiefs, systems folks and others not part of the cataloging cohort that normally predominates in my audiences. This is good—very good, in fact—because it signals to me that the issues I talk about are being seen as crucial to the discussions around a change in mission and strategy essential to creating positive change over the longer term, not just arguing against cuts in budgets for the short term. But, like President Obama points out, the one-crisis-at-a-time approach doesn’t work in a context of multiple crises related to one another in fundamental ways. Attempting the financially necessary reorganization without a refocus on mission, in the face of the huge challenges we face in remaining relevant in the current information environment, is self-defeating at best, suicidal at worst. I spoke to one of the chiefs after the presentation, and he said that he wished his public services folks had been there, too, and I agreed. Certainly the demise of the library catalog as we know it will affect their work hugely, and it behooves us all to break down those barriers of specialization as we face issues of our own survival as information providers.

    What this also suggests is that the focus of the testing being done by the US national libraries cannot be limited to cost benefit analysis. If we fail to look at the issues of most importance to libraries, as the LC Working Group on the Future of Bibliographic Control certainly did, we risk our future entirely. Of course, in the current environment particularly, we need to pay attention to costs and efficiency as we move forward, but they cannot be our sole criteria for decision making.

    For those of you knitters among my readers, I also made a pilgrimage to Webs and spent a goodly portion of my speaker’s fees to support the local economy. It was a great, though at this point I’m going to have to do a “real” retirement sooner than anticipated to use up all the accumulated yarn and patterns I’ve amassed. Either that or recruit a chauffeur on these trips so I can knit on the way.

  • Letting go

    Some of you may know that a few years ago (2005 actually, as I look at my files) I worked with the ALCTS Continuing Education group to set up a “Metadata Standards and Applications” workshop, as part of the series of workshops they were developing with LC funding. At the time, I was intrigued by the set of course series objectives I was given by the committee:

    • To equip catalogers to deal with new types of resources and to recognize their unique characteristics
    • To equip catalogers to evaluate competing approaches to and standards for providing access to resources
    • To equip catalogers to think creatively and work collaboratively with others inside and outside their home institutions
    • To ensure that catalogers have a broad enough understanding of the current environment to be able to make their local efforts compatible and interoperable with other efforts
    • To prepare catalogers to be comfortable with ambiguity and being less than perfect
    • To enable practicing catalogers to put themselves into the emerging digital information environment and to continue to play a significant role in shaping library services

    All these seemed really important to me, and I was eager to try and set up something to meet those objectives. At the time, the notion was that LC would hold rights to the work, and I would contract to deliver the workshop materials. It wasn’t a lot of money, but I was sensible both of the honor of it and the challenge. After a great deal of back and forth with the committee and others, the first workshop was held in July of 2006, as a “train the trainers session.” The idea, as in most of the other successful ALCTS efforts, was that a group of trainers would be able to spread the effort around, and provide a variety of face-to-face efforts for both general groups of catalogers and specialized groups. At the time, workshop organizers needed to work with LC CDS to arrange for materials to be printed for trainers and participants—but last year LC got out of the business of printing materials (and attempting cost recovery from the workshops) and made the materials available for free.

    I think I presented the workshop somewhere between 6-8 times, usually with another trainer (two full days is a long haul for one person, even one who knows the materials as well as I do). I’ve no real idea how many times it was presented by all the folks who were in the trainers group, but it was my understanding that it was one of the most popular workshops in the series. In fall of 2007 I did a full revision of the slides, and sent them to CDS, but I found out later that the updates I did were never integrated with the workshop materials. In April of last year I was contacted by one of the participants in an earlier workshop, who wanted me to present it to a group of Canadians in the fall. We agreed on dates and a co-trainer and everything was going along as usual when I got an email from someone I knew slightly at LC who’d taken over the management of their training efforts.

    At that point I was told that the materials had been entirely revised by two LC staffers (both of whom I knew) and that the version they’d created was now the “approved” version. I’d had no idea this was in the works, hadn’t been consulted or even informed, so needless to say I was pretty unhappy about this news. I was reassured that this was inadvertent, that everyone thought that someone else had “informed” me of this, but it seemed to me that if there had been a good faith effort to consult with me about suggested changes, the question of “being informed” would never have arisen.

    When I was able to view the slides, I was even more unhappy about the situation. Not only did they now have LC slide themes, but the whole thrust of the workshop had changed. There were many more slides about MODS, MADS and METS (accompanied by more LC examples), and a new emphasis on PREMIS (including what I felt was an unrealistically complex exercise). There were also errors introduced about OAI-PMH (that section was cut drastically), and misstatements about Dublin Core, and a number of other worrying changes. I told my contact at LC that I would not present the Canadian workshop with the “approved” slides but would update the slides I had to reflect some of the changes in those slides that I thought were improvements. Thankfully I had help from Rhonda Marker, who’d presented with me before and was booked for the Canadian adventure, and we finished a new improved version of the slides in time for the Canadian workshop. We tweaked them a bit more, and shipped them to LC.

    There were a few more interchanges, but it became increasingly clear that what was at issue was not the examples, or the specific content of slides, but a very basic difference of approach. My approach was not necessarily to provide solutions for librarians, but to survey the landscape with them and try to affect how they thought about their choices and how they make decisions. When I presented the workshop I spoke frankly about the advantages and disadvantages of the available metadata formats, and particularly how I thought they might hold up in the future. I tried to provide perspectives that were not just library perspectives on the world, recognizing that they were probably working with technical people with whom they had no shared language or frame of reference. It seemed to me that they needed to understand what was going on outside libraryland, and to think differently about how technologies being discussed in that world might affect their work.

    I didn’t see that in the LC slides. I was told that the changes made were in response to comments by other trainers and participants, but yet, the evaluations from the workshops I gave were quite positive, and this sense was confirmed by several other trainers I knew, who did the workshop frequently and had made their own modifications in the slides and exercises to suit their styles and experience (this was always fine with me, by the way). One statement in the back-and-forth between me and LC stands out—the LC revisers of the slides felt that the workshop should be more “practical,” which their version indeed was, if one bought into the notion that what librarians really needed was more about MODS, METS, MADS and PREMIS.

    As I pointed out to my contact at LC, I disagree pretty strongly with that approach. Librarians need to look beyond standards (whatever your definition of standards) created within the library community, based on the ways we’ve always done things in libraries. New standards are coming down the pike: RDA, FRAD, etc. Unlike MODS and MADS, these are based on the FRBR model, not the MARC model (if there is indeed such a thing) and have formal structures that can be used easily by libraries and by others outside libraries. We do our colleagues no favors by promoting solutions based primarily on how comfortable and familiar they seem, and what institution is responsible for maintaining them. Will MODS and MADS survive the transition to RDA, FRAD and FRBR-aware data? I don’t think so, and I can’t therefore recommend those solutions to librarians looking for answers to their digital project questions. As I said to my contact at LC:

    “[Librarians] need to be able to THINK about these standards in a broader context, ‘get’ the issues they present in an implementation environment, and understand how they can make decisions today that will still seem sensible five or ten years down the road.”

    If that’s impractical, so be it.

    So, the upshot is that I’ve asked that my name be taken off LC’s version of the slides, and I’ve made it very clear that they should take me off the trainers list too, because I won’t present their version of “Metadata Standards and Applications.” I’ve also told them that they are welcome to use any of the slides I’ve sent them for any version of the workshop. My slides will continue to be available on my website at http://managemetadata.org/msa_r2/ and I’m hoping to continue to develop them to include RDA and other new standards as they become available. So, for those of you planning to present the workshop, expect to be asked to agree to give the workshop as LC has revised it (I hear that the LC slide themes will be removed, but the content will remain the same) or you won’t be able to advertise it as a Cat21 workshop.

    I’ll be doing my workshop, or nothing; but that’s okay. I’ve got places to go, many interesting things to do, and I’m done arguing about this one.

  • Denver Midwinter: All RDA, all the time …

    Yes, it was laughable to imagine that I might blog from Midwinter—silly me. My schedule was insanely full and I didn’t even get to peek into the exhibit hall, much less visit and pick up some swag for the grandchildren. As things move along with RDA, and as I become even more convinced that despite its flaws we need to have it out and used, my conference program became even more ridiculously focused on anything RDA. So there I was at both CC:DA meetings, the RDA Testing meeting, the RDA Implementation meeting, and on, and on.

    I posted briefly before I left for Denver on the work Metadata Management Associates (me and Jon) will be doing with ALA Publishing in relation to the RDA Online product. Some of this will involve integrating the RDAVocab registered elements, roles, relationships and value vocabularies, with the online product, thus ensuring that catalogers using the online product and applications using the Registry output will be linking to and/or referencing the same information (no synchronization issues!) Other tasks involve building XML schemas so that those seeking to build data—whether in the tool or in another application—have somewhere to start. We’ll be working on other bits and pieces to enable support for those trying to see their way forward from where they are to a world where RDA with a FRBR foundation is the way we do description. This is a tremendously exciting project for us and it was nice to see that when I explained it to others they thought so too!

    The RDA Testing meeting, led by Beacher Wiggins, was an interesting one—more so than I had anticipated. The idea is that the effort will be led by LC, NLM and NAL, but it will be open and participants from a variety of venues (including library students and recent graduates) are encouraged to sign up. Beacher emphasized that testers will use the environments they have available, which for most means MARC systems, but if RDA Online can produce records as well, that’s an added opportunity, particularly for systems eager to figure out how these records might behave in newer systems. Testing will last about six months and include a training phase, an active record creation phase, and an assessment phase. After that, the three national libraries will make individual “go” or “no go” decisions. There will be a website at some point, most likely a subset of the website of the LC Working Group on the Future of Bibliographic Control.

    There were a lot of questions about this testing effort, only some of which were asked or answered at the meeting. One of my concerns (which was too fuzzy and formless to ask at the meeting) has to do with the usefulness of testing how much time it takes to create a record under the old regime (AACR2 and MARC21) vs. the new regime (RDA and MARC21? or RDA and the RDA schemas and tools?)—you see part of the problem here. The deck seems pretty stacked towards the old and familiar, despite some attempts to create more balance by ensuring that some of the anticipated participants—the library students and recent grads primarily—will have as little experience with the old regime as they do with the new one. In all cases there will be “subjective” assessment solicited from participants as well as the “objective” results (the time invested in record creation). Part of the plan is that there will be a 20-30 item list of resources to be cataloged, and most participants will do one regime or the other (not both) for a particular resource, with hopes that aggregation of a large-ish number of results will provide a more reliable measure. I’m not sure the “timing test” is going to be particularly useful, given the number of uncontrollable variables likely to be introduced along the way—particularly in whatever technical environment is used—but I’m sympathetic with the notion that looking for a useful objective test in this situation is extremely challenging. It’s obvious to me that we’ll learn something from the objective and subjective portions of the testing, but whether what we learn is going to be clear enough to support either option with sufficient clarity for those who prefer evidence-based reality is quite another story.

    Given the fact that the testing results are likely to be subject to various interpretations and caveats, it’s hard to imagine how any of the national libraries could justify a “no go” decision, and what that would actually entail, if one or more might be inclined in that direction. One hears rumors, which naturally can’t be attributed, that one or two are so inclined but it’s something I’m really curious about. Would a “no go” decision be something like “no go until 2011” or “we’re sticking with AACR2 and MARC21 until Hell freezes over,” and how long would that be supportable? Two years, five years, maybe? It all depends on how long and how messy one thinks the transition would be, and how many would be inclined to lag far behind the early adopters waiting for some undefined
    tipping point. I tend to think that there will be enough of the early adopters, particularly in the open source development community, that by the time the “go”/”no go” decisions are made, the “go” decision would be almost inevitable. But I’m an optimist on this, as I’m told fairly frequently.

    Of course, the timeline for testing is dependent on RDA Online being released on time (3rd quarter, 2009) complete with vocabularies. I’m an optimist on that, too, which is a good thing, since some of it depends on me continuing to work relentlessly on some of my task list. More on that topic in other posts …

  • Collaboration between ALA Publishing and the NSDL Registry

    Some of you will already have seen the blog post by Patrick Hogan on the ALA Tech Source blog. This is an exciting new collaboration for us, and in some ways represents the culmination of the DCMI/RDA Task Group efforts to register the RDA Elements, Roles, and value vocabularies. We figured that seeing the vocabularies in the Registry would highlight the importance of the formal representation of those vocabularies, and lead to good ideas for embedding them into actual products of use to libraries. This is certainly what seems to be happening.

    Because this announcement came out so close to Midwinter, much discussion on the implications of this collaboration will happen in the context of RDA programs, meetings and hallway chats. I will certainly be reporting on some of those discussions from Midwinter or shortly thereafter.

    Stay tuned.

  • The XC Project White Paper–well worth reading …

    … and I say that not just because Jon and I have been working with the Project on their metadata planning. It’s an excellent paper, and we’re delighted it’s out, so we can tell people about it, and encourage them to read it and talk about it. We’ve been talking up the XC Project as one that has been doing important work and moving forward in ways that reward our close attention. But until now we’ve found it hard to find something to which to refer interested parties, given that so much about what XC has been doing has been happening under the surface of the pond. Now we won’t have that problem. Go look at the paper here.

    Jennifer generously mentions the inspiration of our ideas—as reflected in one of our grant proposals (which wasn’t funded and isn’t publicly available). We’d like to point out that many of the ideas in the proposal are available in previously published papers, all of which are linked from this site.

    Takeaways from XC

    Jennifer rightly and effectively presents the practical decision points in the XC project and documents them well. But there are bigger issues underlying those decisions, and it’s important to note how they fit into the larger task of building services in an environment as fluid as the one in which we find ourselves.

    “It is also important to understand that the XC Schema represents a purely practical rather than a theoretical approach to using metadata. While much interesting work is taking place in the library community to develop theoretically consistent metadata frameworks, models and carriers, the process used to develop the XC Schema should not be confused with these start‐over‐from‐scratch‐and‐do‐it‐right approaches. In some ways, the XC Schema represents exactly the opposite. For XC, we have attempted to manipulate and make use of the metadata in various schemas that is available to us, and to use this metadata in a way that it can be shared with others. While ours is not a theoretical approach in itself, we believe that the work that we are doing can inform the work of those who are developing more theoretical approaches to the future of library metadata. Our experiences should provide a reality check regarding the feasibility of retrofitting legacy metadata into a new theoretical framework. Much of the effort needed to retrofit library metadata for use in new applications alongside metadata that originated in other schemas is likely to require considerable experimentation. The XC Schema is designed to embrace and facilitate that experimentation.”

    Despite Jennifer’s emphasis of the practical solutions (and the fact that she has a notion of experimentation that is foreign to most librarians), there are some important theoretical underpinnings that she illustrates in the paper, and I wanted to hit on a couple that I think are important.

    The benefits of “dis-integration”

    It’s ironic that libraries have been urging their vendors for years to integrate their platforms better, and now we’re taking a quite opposite approach. Jennifer nicely describes how this works in XC, and the rationale behind those decisions. Many in libraries have come to the conclusion that discovery services needn’t be bolted to data management services, but XC goes several steps further, and unbolts data management from the ILS as well.

    “Dis-integration” is critical in bringing libraries out of the current environment of tightly bound library systems built on MARC and depending only on one another for data. Libraries today have too many silos of data for that strategy to endure: IRs, projects of all kinds, commercial partnerships, etc.

    Iteration vs. Perfection

    I was talking to a colleague new to libraries recently and describing how data was moved from one platform to another in the days when I was still working in academic libraries. Back then, we spent almost a year planning the “migration” from NOTIS to Voyager, and had contractually specified two test loads of our data into the Voyager test system we had up. These loads were then tested extensively, using specifically chosen test conditions and records (we had our own test plan, which was extremely detailed and fine tuned—we didn’t trust the vendor at all). After each load we would specify changes that had to be made before we could do another test load. The third load was the real one, it was final, and all the data had to be as perfect as we could make it, as we had no other chances. For months before the loads we had students sitting in front of computers making changes we couldn’t make programmatically and programmers slogging through the ones we could, knowing that it would take years to re-create the knowledge and tools to do them in a new system. They did the last changes about 20 minutes before the data was extracted for the “real” load. Stress, anyone?

    Consider the difference in XC. The normalization and transformation services are independent, they maintain their own data stores, and they’re designed to operate independently. Data services can be tweaked as experience and changed goals dictate, and data can be moved through the services any number of times, until the result is optimal, without interruption of any user delivery services at all. It’s an ideal strategy for an environment where nothing is stable, change is perpetual, and nobody can afford to wait until things settle down (they may never, you know).

    The XC strategy also supports the construction of an environment where many entities can build usable services and the expansion of overall service capability (and the cost of creation and maintenance) can be shared broadly.

    Taking the Leap

    One of the most important things that Jennifer’s paper illustrates is how libraries can make the leap from legacy data and systems into the future of library data. A lot of people are scared by this — they figure that there’s no way to make the leap unless we know more, standards are more complete, agreements are made. XC has provided real leadership here, by showing the value of jumping in and trying things out. They are showing far more leadership than the institutions (you know who they are) we have always depended upon to lead us forward.

    The only possible response to this effort is close attention and engagement. Take a look at the supporting documentation Jennifer provides, engage her in conversation about this work, and take inspiration from her willingness to try out new ways of doing things.

    We’re all very proud of you Jennifer—keep up the good work!

  • LCSH.info–gone but not forgotten

    The library blogosphere is reacting with shock [1, 2], anger [3], and disappointment [4, 5, 6, 7] at the news that LC has asked (as in “ordered”) Ed Summers to take down his wonderful experimental LCSH.info service. It’s not entirely clear why they did this, and Ed has been carefully circumspect about the reasons. LCSH.info was a project that Ed had accomplished largely on his own initiative, and despite the fact that he certainly had some important support within the institution for building it, he too readily takes responsibility for not gathering the broad institutional support that appears to be a necessity for sustainability at LC. Still, it’s easy to speculate that others were embarrassed by it (as in “embarrassed that Ed could do it, and yet there’s still no official service”), particularly given the fact that the domain was not “loc.gov.”

    LC has been “planning” (or so they say) to create just such a service since well before the Working Group on the Future of Bibliographic Control but we’ve seen no real evidence of it, except for what Ed has done. I was one of the people who spoke to that group during its fact-finding year, making the point that the vocabularies that LC manages for the library community (LCSH, LCNAF, LCSAF, etc.) should be available as web services for the benefit of the that community (I was assured then that such plans were afoot.) LC could have wisely allowed Ed to maintain the site until such a service was in place and then Ed could have made a simple change that would have redirected lcsh.info URIs to the official service, and all would have been well. Instead, LC has let the world know that its management is technically and politically clueless as it destroys, without even an official announcement, a service that many organizations, regardless of disclaimers of its experimental nature, had welcomed and some had come to depend on. This had roughly the equivalent impact on the confidence of the Linked Data community in LC that letting Lehman Brothers fail had on the investment markets.

    Although LC frequently cites the legal necessity to do cost recovery on the vocabularies, their poorly-supported subscription services are a constant source of annoyance for those who must rely on them, and if they were really expecting to recover significant costs it’s probable that the group that supports that effort would be adequately staffed, which it isn’t. (This same lack of adequate support was the likely reason that the ALCTS continuing education workshops are now freely available, but that’s another story.) In short, we’ve seen little evidence that there are serious efforts to replace these outdated services with the web services that the Working Group and the library community have called for—except, of course, the late LCSH.info service.

    I’ll make the point again that I made to the Working Group in 2007—these are vocabularies that were not created solely by LC—particularly the Name Authority File (LCNAF) which has seen significant investment by the library community through the NACO Project (Name Authority Cooperative, please note), and the Series Authority File (LCSAF) now being built entirely by the community since LC unilaterally withdrew from the creation of series authorities in 2006. It was that decision, clearly made without adequate consultation with anyone outside LC (or many within—I happened to be visiting a colleague there a day or so after the announcement, and most of the staff were surprised, too), which resulted in the furor that led to the creation of the Working Group in the first place. But obviously, lessons were not learned from that debacle, and we’re back at the barricades, frustrated yet again.

    Contrast this with the recent post on the LC blog (posted on 12/11) announcing the report on the Flickr Pilot, one of the most innovative projects LC has done in a very, very long time. The report, ironically entitled “For the Common Good” (summary available) describes the overwhelmingly positive response to the project, and how LC used the information provided by viewers to enhance their information on the photographs. Most telling is this snippet from the report on the project taken from the LC staff newsletter and included in the blog post:

    “The popularity and impact of the pilot have been remarkable,” said Michelle Springer, project manager for digital initiatives in the Office of Strategic Initiatives, who said total views reached 10 million in October. The site is averaging 500,000 views a month, she said, adding that Flickr members have marked 79 percent of the photos as “favorites.”

    “The report recommends that the Library of Congress continue to participate in The Commons and explore other Web 2.0 communities.”

    Clearly by “other Web 2.0 communities” the writer could not mean the library community, which has been stymied for far too long by LC’s reluctant and too often less-than-competent leadership, and are still some distance from being a “Web 2.0 community.”

    What we see here is an institution with no coherent strategy for moving forward, despite a year long study of the issues, and clear recommendations for action. This is an institution where decisions are still being made by one hand that the other hand may not know about or have been consulted about—decisions that are ill-considered and fray LC’s credibility and its relationship with the rest of the library community (not to mention the Semantic Web community). This decision may well draw attention away from the other supposed library leadership institution—OCLC—caught making its own self-serving decisions without reference to its members real needs and interests, though members pay for the records that make the monopoly possible.

    As the New Year fast approaches, perhaps we need a resolution that says: “Stop expecting the 8000 lb. Gorillas to lead us; start leading ourselves.”

    [1] Jenn Riley
    [2] Tim Spalding
    [3] Karen Coyle
    [4] Andy Powell
    [5] David Bigwood [Corrected]
    [6] Richard Wallis
    [7] Dan Chudnov