This case study introduces the Occupational Folklore Project, a multi-year oral history initiative that seeks to capture a portrait of America’s workforce through interviews with workers across the United States. The project, designed by the American Folklife Center at the Library of Congress, is still being beta-tested. The progress report presented here provides information about how the project is using an innovative application of online digital technology to collect and process documentary materials from geographically dispersed fieldworkers.
This case study introduces the Occupational Folklore Project, a multi-year oral history initiative that seeks to capture a portrait of America’s workforce at a time of economic transition through interviews with workers across the United States. Designed by the American Folklife Center (AFC) at the Library of Congress (LOC) and originally titled America Works, the Occupational Folklore Project is still in its early stages. We present this progress report now, because we want to share information about how the project is using an innovative application of online digital technology to collect and process documentary materials from geographically dispersed fieldworkers. If all continues to go as smoothly as indicated by the initial beta-test results, fieldworkers across the United States will be able to submit their interviews and accompanying metadata directly to the AFC/LOC through a custom-designed, user-friendly online application that reflects and reinforces contemporary archival and oral history practices and protocols. This article focuses specifically on the technical aspects of the larger Occupational Folklore Project, which is being developed by AFC staff.
Launched in 2010, the Occupational Folklore Project is a joint collecting project and education initiative cosponsored by the AFC and the Institute of Museum and Library Services in cooperation with the American Folklore Society (AFS) and the National Council for the Traditional Arts. The Occupational Folklore Project engages with folklorists, oral historians, librarians, museum professionals, independent scholars, and other researchers to record interviews with American workers in all sectors of the economy and in communities across the United States. Eventually, we anticipate that community libraries, regional archives, museums, historical societies, cultural associations, universities and university libraries, trade unions, and professional associations will serve as host sites for a majority of Occupational Folklore Project activities. Guided by a list of suggested core questions, interviewers ask workers about their jobs and workplace experiences, their education and training, their attitudes towards their present jobs, and their hopes for the future. After obtaining a signed release form, the interviewer then uploads the digitally recorded oral history interview to the AFC/LOC via web-based storage and simultaneously submits the interview’s accompanying metadata to the AFC/LOC through the easy-to-use online cataloging template, described below. The AFC will serve as the primary repository for the resulting oral histories, but local libraries, regional archives, museums, and/or other participating organizations will be encouraged to retain copies of their Occupational Folklore Project interviews and the accompanying metadata for their archives. By participating in the Occupational Folklore Project, local organizations and researchers will gain experience with oral history techniques and familiarity with twenty-first-century digital archiving skills. We hope that interviews generated by the Occupational Folklore Project will also enrich local and regional organizational holdings, and enhance public programming activities.
The AFC’s interest in the digital future is firmly rooted in the LOC’s predigital past. Few oral history or occupational folklore collections are better known or more frequently cited than the Library’s American Life Histories. Compiled between 1936 and 1940 under the direction of Benjamin Botkin and the staff of the Folklore Project of the Federal Writers’ Project for the U.S. Works Progress (later Work Projects) Administration (WPA), and later deposited at the LOC, the collection includes 2900 documents created by over 300 writer/interviewers from 24 states. The interviews they collected, which are typically 2000–15,000 word summaries, vary in form and quality and cover a wide variety of topics but are arguably most valuable for the information and insights they provide on the experiences of American workers at the height of the Great Depression.
In winter 2009, during a period of economic uncertainty reminiscent of the 1930s, the board and staff of the AFC began discussing the importance of documenting occupational lore and life in contemporary America through recorded oral history interviews. The American Life Histories were inspirational, all agreed, but the feasibility of launching a collecting project approaching the size and geographic breadth of the WPA project seemed unrealistic, especially given the current funding climate. Thus, we began to consider whether an entirely new model might be in order. In the digital age, our thoughts were drawn to the possibility of some kind of a digital collecting project. Was it possible, we wondered, to design a website with an online data-submission application that would allow us to coordinate a nationwide oral history collecting project, one that was both cost- and labor-efficient? Could we build a user-friendly, public-facing application that would simultaneously reinforce best practices in folklore and oral history collecting and also provide an easy way for electronic documentary materials to be transmitted to the AFC? If so, what data and metadata should we request from interviewers and partnering organizations? How would we maintain the project’s focus and quality? What would be the legal and security implications? And how would we manage the resulting information to ensure that it would be discoverable and accessible by future researchers? And, while we were at it, could we ask fieldworkers to submit their data using consistent, easy-to-manage protocols that would assist us in cataloging and processing the incoming materials?
Envisioning such an innovative application also raised many more prosaic questions about the larger project, for example: How would such a project be funded? What would be the best way to select and train fieldworkers? How should we approach networking and partnership arrangements? However, all these important issues would be academic if we could not build a workable online platform. The following pages detail how we custom-designed an Oracle Application Express (APEX) platform for the Occupational Folklore Project, along with our assessment of how our application is functioning. At the time of this writing (December 2012), the application and an accompanying website have been launched and are being actively beta-tested. To date, we have received 150 interviews from fifteen organizations and individual researchers in ten states and the District of Columbia. During the current beta-test period, we are also turning our attention to addressing other, non-technical questions related to the project. We look forward to reporting results, challenges, and (we hope) creative solutions, in greater detail in future publications. For now, here is where we are with our online presence.
The Technical Platform
We were fortunate that the AFC’s interest in exploring an online approach to acquiring documentary materials on contemporary occupational folklore coincided with the LOC’s decision to test the functionality of Oracle APEX as an enterprise tool for LOC divisions to use in organizing and managing non-critical, internal data sets. Oracle APEX, aside from offering a robust object-oriented database platform, provides users with workspaces for developing and customizing database forms and reports that can be combined into web applications for distributed data collection. Oracle APEX also provides authentication and access controls. All of these parts have been combined to provide an easy-to-use, secure online application.
For the Occupational Folklore Project, Oracle APEX allows the AFC/LOC to collaborate on electronic collecting and documentation projects with partners outside the LOC’s walls. By creating a web-based, customized database application that is available through the Internet to selected users, this tool minimizes the work of translating and mediating data that accompanies acquisitions and/or library outreach projects. For example, Oracle APEX allows our partners to contribute to the categorization and description of the materials they submit to the library (in other words, to assist us by precataloging their own materials to expedite LOC processing and shorten the time required to make their materials accessible to researchers). Because the tool is extremely flexible, AFC catalogers can set the parameters and values for descriptive fields, which allows for controlled social cataloging. This is one step beyond the popular, but often problematic, practice of social tagging. This approach eliminates the need for packaging and mailing paper records and also prevents idiosyncratic, non-standard electronic data from being submitted. Hosted on LOC servers, the application and all data collected through the application are backed up according to the LOC’s IT protocols.
Over the past three years, AFC staff have given a great deal of thought to what data and metadata we would like to obtain via the online Occupational Folklore Project portal. In recent years, the AFC has been involved in several large field collecting projects: in 2000 it initiated the Veterans History Project to collect and preserve oral histories of American wartime veterans, and since 2003 it has served as the repository for interviews collected by StoryCorps. Both of these projects have produced tens of thousands of interviews; however, neither was entirely a born-digital project, and problems have developed over the years in the collection, processing, and transmission of data and metadata. Our goal for the Occupational Folklore Project was to draw on lessons learned from past experiences, while taking advantage of recent technological advances in order to build a better, smarter, more flexible platform that allows us to easily retrieve and share collection data.
At the heart of the Occupational Folklore Project is a digitally recorded interview with an individual about his/her occupation, work day, professional training, work-related experiences, attitudes, and expectations. The recommended interview length is forty-five to sixty minutes. Interviewers are provided with a short list of suggested core questions to provide general guidance and stimulate discussion, but they are also directed not to worry if they do not cover all of the questions, or if other work-related issues arise and are discussed during the interview. Interviewers are also free to substitute other questions that are more relevant to a specific interview. During the beta-test period, we are strongly encouraging audio recordings, hoping to add video documentation once the Occupational Folklore Project protocols are established and successfully tested. Interviews must be recorded and submitted as WAV audio files. We will not accept any other audio format (MP3, FLAC, OGG, AIFF, etc.). We are also requiring that interviews be recorded at a resolution of 24 bits/96 khz, the current international archival standard for digital audio files.
After an interview has been recorded, and a permission and release form has been signed by the interviewee, the recorded interview and interview data form must be submitted to the AFC online. In most cases, the interviewer/fieldworker will be the person to submit the interview file and metadata, although we have already received some excellent materials submitted by an organization’s curator, director, or properly trained support staff.
We have divided the Occupational Folklore Project online instructions and protocols into two parts: the first consists of publicly accessible pages on the AFC/LOC website; the second is the custom-designed online catalog template, which we call the Interview Data Form (IDF). On the home page of Occupational Folklore Project’s publicly accessible website, the project is introduced and its goals and requirements are outlined. Subpages provide an overview of the submission process, offer a short list of suggested core interview questions, and provide downloadable PDFs of the permission/release forms that can be printed out and shared with interviewees.
To maintain control on the size and direction of the project as it develops, a direct link from the website to the IDF is not included. Rather, organizations and individuals interested in participating in the Occupational Folklore Project are asked to contact project director Nancy Groce, who will provide them with the URL for the IDF. The protocols for participation in the project are still being formulated and may change over time. The Occupational Folklore Project website itself will not be posted publicly on the AFC/LOC website until the beta-testing period has been completed.
The Interview Data Form
The key to the management of the Occupational Folklore Project is the Interview Data Form (IDF), designed by Bertram Lyons and others, using the Oracle APEX platform, as detailed above. We sought to design an easy-to-use IDF that garnered sufficient data to adequately catalog each interview, while also satisfying the needs of present and future researchers. At the same time, we tried to streamline what was being requested so that the IDF would not be off-putting to our fieldwork partners—in short, to design a form that did not require a daunting amount of time and effort to complete. We assumed that most of those submitting Occupational Folklore Project interviews would be the interviewers themselves: oral historians, folklorists and folklore students, librarians, museum professionals, and other researchers, many of whom were not computer experts nor even very comfortable with online technology.
We also thought it was important for the IDF to incorporate the crucial step in which the interviewer’s digital-audio interview file is uploaded to an online storage site. Folding this step into the IDF makes it less likely that the audio file, as well as any accompanying digital images or ancillary files, will become disassociated from its metadata. We also hoped that providing interviewers with a step-by-step guide to uploading files to the cloud site would familiarize our colleagues and fieldworkers with the latest protocols and procedures in digital-file delivery.
We will not provide extensive details about each facet of the IDF here, only a quick overview. We also touch on a few of the many challenges we faced in designing this application. The Occupational Folklore Project’s IDF is a web application. It cannot be downloaded and researchers must be online to enter data. However, the form can be completed over a number of visits, and the data entered can be saved from one visit to another. For reading and review purposes, interviewers are able to download IDF pages as PDFs, Word documents, and/or web pages (html). In addition, submissions can be downloaded as delimited files that preserve the database fields: comma-separated values (CSV), or extensible markup language (XML).
The IDF consists of three guided sections designed to walk the interviewer through the submission process. Each requested data field is provided with a pop-up help box that explains the rules or constraints we have imposed on the data field and/or helps the user better understand any options that exist for the data field by providing examples. Additional instructions, tips, and examples are contained in help boxes located along the right-hand page margin. Longer entries, specifically the required timecoded “Interview Log,” can be written off-line and pasted into the form. A bread-crumb trail, along the top of each page, assists with navigation, and the top page allows easy access to interviews that are in the midst of being edited, as well as read-only access to previously completed and submitted interviews. (The fieldworkers’ ability to make online changes ends once an interview has been finalized and submitted, although AFC staff can still modify the record at the LOC.) A “View/Browse All Submitted Interviews” button, also on the front page, permits those with access to the system to review data from all submitted interviews.
Step 1, “Register Organization,” requests information on the organization that is sponsoring or supporting the interview. Users are asked first to check a list on the “Organization” page that contains previously registered organizations. If their organization is included, they are directed to Step 2. If not, they select the “Add an Organization” button and are taken to a data-entry page. There, customary information is requested by a series of data fields: the organization’s title, short description, and contact information. There is also a short drop-down list requesting “type of organization”: library, museum, humanities council, union/trade association, professional associations, independent researcher, other. This information assists us in assessing what types of entities are participating in the Occupational Folklore Project. Optional data fields accommodate the submission of additional information, such as the recognition of cosponsorships, acknowledgement of funders, or other metadata that the user would like included in the record. When the “Register Organization” page is completed, the user selects “save,” which adds the information to the Occupational Folklore Project database, and then proceeds to Step 2.
Step 2, “Register People,” guides users in creating a separate information record for each individual (both interviewer(s) and interviewee(s)) to the database. (As with the “Register Organization” page, a list of previously entered individuals appears on the top page. If records have already been created for the individuals involved, the user is directed to Step 3.) The information requested includes name, contact information, year of birth, and birthplace. A help box explains that this information is requested in order to help catalogers establish a unique record for each individual in the Library of Congress Authorities. Personal contact information, such as home addresses, phone numbers, and e-mail addresses, is accessible only to LOC staff and the user creating the record; users are not able to see sensitive data or edit records established by other fieldworkers.
For logistical reasons, we decided that Step 2, “Register People,” was the most logical place to ask for information on an individual’s occupation and occupational history. With the cooperation of AFC catalogers, we reviewed lists of occupation titles used by the U.S. Department of Labor and the LOC, and winnowed a list of hundreds of specialties and sub-specialties down to a manageable list of 150 of America’s more common jobs and occupations. These job titles were then included, in alphabetical order, in a scrollable shuttle-list on the “Register People” page. The user is asked to select those jobs/occupations that have been held by the individual whose record is being created; they may select as many as are relevant. In cases where job descriptions overlap (for example, a travel agent could be listed under “tourism workers” or “sales personnel”), users are encouraged to select both.
If an occupation discussed during an interview is not listed, users are instructed to use the “Other Occupation(s)” field immediately below the shuttle box to enter the occupation(s) that best reflect the interviewee’s experiences. An optional “Self-Description” field is also provided, since people sometimes identify themselves with occupations that they do not presently hold or work at only part-time. (For example, this is a way of identifying a poet who also works in sales, or a band leader who is also a full-time farmer.)
Once users add and save the information to “Organization” and “People” databases, the name(s) and entities appear on the catalog template’s drop-down lists and are available for use when submitting future interviews. Throughout the IDF, some fields are required; these are noted with a red star (*). Users are not able to advance to the next step without providing required information. The form reminds them if any essential information is missing.
Step 3, “Create New Interview Record,” is the central step in the submission process. This page draws on the “Organization” and “People” database records that have been created previously during Steps 1 and 2. Date, location, and a short summary (up to five hundred characters) of the interview are requested, and additional information on circumstances and venues relevant to the interview can also be entered.
One of the most important components of Step 3 is the timecoded “Interview Log” that interviewers are asked to prepare for each interview. The log is essential for locating and retrieving topics discussed in the interview. Examples of a timecoded interview log are provided in the pop-up help box as well as on the web page. Since writing an interview log is the most time-consuming part of the submission, we suggest that interviewers prepare the log off-line and then paste it into the form. This text field has a limit of thirty thousand characters, which can accommodate most of the submitted interviews. Although a full transcript of an interview is not required, we are delighted to accept a transcript, in addition to or instead of an interview log, as long as it is also timecoded. (Digitized transcripts can either be included in “Interview Log” field in the IDF or, if too large, can be attached as a text file and submitted via the cloud site, as described below.)
The “Interview Log” text field is followed by several multiple-choice shuttle lists that request basic information on the language(s) used in the interview, performance genres, and topics. Of these, the “Topics” list presented the greatest challenge. Early in our planning for this project, we determined that having a list of key topics discussed during interviews was essential for quick and efficient cataloging. Having interviewers provide this information seemed like the most logical thing to do, since they would be most familiar with specific interviews and their contents. But how could thousands of possible topics be reduced into a manageable number of online options? To address this challenge, we again worked with our archiving and cataloging colleagues to develop a controlled vocabulary, a list of preselected words and terms designed to limit ambiguity and aid the search and retrieval of information.
We knew from previous oral history projects that asking interviewers to develop their own topic lists often led to inconsistencies and variations that made cataloging and retrieval of specific data almost impossible. Therefore, we decided to draw on the Library of Congress Subject Headings (LCSH), a list of approximately 265,000 items that has been actively maintained since 1898 to catalog materials held at the LOC. Using this database, we compiled a long list of occupation-related topics that were likely to arise in an interview about work and the workplace. As with the occupations list, the resulting topics list was far too long; thus, after much discussion, we winnowed the list down to 140 topics. Even using the LCSH presented challenges. The list is based on topics in books that have been cataloged over the years at the LOC, and it is somewhat inconsistent. For example, there is no LCSH entry for “boss” or “bosses,” so we elected to include the closest topic heading we could find: “managing your boss”; likewise, there were no entries for “layoffs” or “downsizing,” so we used the closest available topic: “layoff systems.” Nevertheless, we think that the resulting list, which presents topics alphabetically in an easy-to-use shuttle list, will be manageable and useful. Researchers are instructed to select as many topics as are relevant; if there are overlaps, users are urged to select more than one. An “Additional Notes” text box is provided for users to fill in additional topics that are not included in the shuttle list.
In beta-testing, selecting topics from the topics list, as well as providing other information requested in Step 3, seems to go quickly and smoothly for the researchers. By precataloging the interview before it arrives at the LOC, the fieldworker enables the LOC staff to vastly decrease the amount of time it takes to process incoming information and make it available to researchers. However, we look forward to additional feedback from the beta-testers before declaring it a total success.
Uploading Interview Files to the Cloud Site
After the new catalog record has been created, the user continues on to the penultimate part of the submission: uploading the digital interview file to an external cloud site so it can be transferred to the LOC. During the beta-test period, we are using ADrive, an inexpensive commercial service. Although ADrive is low cost and easy to use, it is also slow. It can take several hours for a fieldworker with a slow Internet connection to upload a single large sound file to the ADrive cloud site. In several cases where a project was submitting numerous large sound files, we found it more expedient to transfer these files to the Library on an external hard drive. We are also beginning to use DropBox and will continue to explore other options.
In Step 3.1, “Interview Item,” users are asked to rename their digital file(s) to conform to the AFC’s file-naming protocols. This is a straightforward procedure, and instructions and examples are provided on the page. Once a user reaches this point in the application, the database has already generated a unique catalog number for his or her submission. Users are given instructions on how to incorporate this number when renaming their file(s) and then how to enter the resulting title in the appropriate data field. Users are also asked to confirm which format (WAV, TIFF, JPEG, PDF, DOC, etc.) is about to be uploaded, as well as the date of the interview and the upload date, to designate who created/prepared the file, and to provide a short description of the file (e.g., “interview with Raymond Jones”), all of which will assist AFC staff in harvesting this file from the cloud site.
After entering an item’s file name in the designated data field and selecting its format, the user then clicks on our cloud-site icon, which takes them to an external Occupational Folklore Project cloud site, where they enter the provided password and then follow simple instructions for uploading their interview file(s). Digital images, transcripts, scanned release forms, and ancillary materials related to an interview (all optional) and labeled according to the prescribed file-name protocols provided on the same page can also be uploaded during the same visit to the cloud site in which the user uploads the digital interview file(s). When uploading is completed, the user is instructed to close the cloud site and is automatically returned to the IDF for final review of the materials they have entered (Step 3.2: “Summary”), after which they can check out.
Users are now ready for the “Finalize and Submit” step, but first both the interviewer and interviewee must grant the LOC permission to add the interview to its collection. We consulted with the LOC’s Office of the General Counsel to determine whether the Occupational Folklore Project permission and releases could be streamlined through the use of an “I have read the conditions and I agree” check-off option, like those commonly found on commercial websites. To our delight, the LOC’s lawyers agreed that this option was acceptable for interviewers, but felt it was still necessary to receive a signed release from interviewees.
Permission and release forms for both the interviewer and interviewee are posted on the website and linked to the IDF. Interviewers are urged to read both forms before beginning their fieldwork and to print out and take the interviewee’s release form to the interview in order to obtain the interviewee’s signature during their visit. Any restrictions the interviewee wishes to specify can be noted on the signed release form or on an attached note; however, we urge interviewers to explain to interviewees prior to their session that a primary goal of the Occupational Folklore Project is to record interviews that will be publically accessible to researchers and members of the general public. The signed form itself must be returned to the AFC via mail or fax or digitized and uploaded to the cloud site. (Digitized signatures are acceptable.) To date, the majority of interviewee’s permission and release forms have been scanned and returned as attached digital records.
On the IDF form, the interviewer is asked to select “Yes” to confirm that the interviewee’s signature has been obtained and that the resulting signed form is either attached or being sent, and also “Yes” to confirm that that he or she has read the interviewer’s rights and permission form and agrees to its stipulations. Both these selections have been set up as required fields; thus, the IDF cannot be completed unless the user grants his or her approval and lets us know that the interviewee’s consent has been obtained. This step completes the IDF. The user can either save the assembled information for additional editing on a return visit, or select the “Submit” button. If the user selects “Submit,” an e-mail notice (bearing the unique, automatically assigned catalog ID for that specific interview as well as the names of the interviewer and interviewee) is immediately sent to the AFC notifying us that an IDF is ready for processing and that its associated interview file(s) is (are) waiting to be harvested from the cloud site. A copy of this confirmation e-mail is simultaneously sent to the interviewer who submitted the IDF form.
As soon as possible after receiving the e-mail notice, a member of the AFC archives staff processes the IDF and harvests the associated file(s) from the cloud site. The data and the associated file(s) are then verified and linked, and the files are moved to the LOC’s servers for long-term preservation and management. The cloud site is used only for transmission of interview file(s); once an interview and associated material have been harvested, copied to LOC servers, and verified and validated, the file is deleted from the cloud site by AFC staff. (Fieldworkers are advised to retain copies of their interviews as a backup in case of problems with the transfer process.) In addition to notifying the interviewer that his or her submission has been received, we plan shortly to set up protocols that will generate thank-you letters for interviewees after their materials have been processed.
After an interview has been submitted, users with access to the system are able to read and download a copy of the catalog record from the website; however, they will no longer be able to edit their submission. AFC staff can assist users if any post-submission edits are necessary. In fact, AFC staff are available to assist participants via e-mail and/or phone with all aspects of the submission process.
The Occupational Folklore Project model has great promise as an online oral history collection tool, and early feedback from beta-test sites has been encouraging. As part of the pilot program, cultural organizations and individual researchers in several states have kindly volunteered to serve as beta-testers. Beta-test sites include local libraries, arts and cultural organizations, universities, professional associations, and several independent researchers. Trained folklorists and oral historians at these beta-test sites are conducting in-depth oral history interviews with members of their communities and have begun to submit the resulting interviews and metadata through the procedures detailed above. Students enrolled in several college-level folklore courses have also participated in testing the system.
The AFC is maintaining close contact with the beta-testers during this pilot project in order to evaluate our methodologies and use testers’ feedback to refine our procedures and improve our protocols. An internal report will be submitted at the close of the pilot test, with the goal of securing full Library approval of the project. The AFC hopes to make the Occupational Folklore Project available to additional libraries, museums, and cultural organizations throughout the United States in the near future.
We expect the online platform developed for the Occupational Folklore Project to result in cost-efficient protocols for the acquisition of oral history materials from partners in geographically dispersed locations. The web-based, customized database application proposed here is expected to minimize the work of translating and mediating data that accompanies these acquisitions. The Occupational Folklore Project model is expected to facilitate this project, and possibly other LOC outreach projects, by allowing the creators of the content (the interviewers who are doing the fieldwork) to contribute to the categorization and description of the materials they submit. If the proposed application of this Oracle APEX platform proves to be as successful and adaptable as early results indicate, the approach may also provide a viable model for other online archival projects.
Many details have yet to be addressed. In-house workflow and processing protocols are continually being refined, and public-facing interfaces for serving the Occupational Folklore Project interviews and associated metadata to researchers and members of the public must be improved. How, or even if, the Occupational Folklore Project data will be posted online is still being reviewed by senior management at the Library of Congress. Finally, if the Occupational Folklore Project expands, as we hope it will, we think it will be necessary to enhance the LOC’s present online training materials and develop better interactive programming in folklore and oral history.
For now, we believe we have developed an important first step: an efficient and flexible online oral history collection tool that will enable us to effectively conduct a multi-faceted collecting project involving colleagues and collaborators throughout the United States. We look forward to advice and suggestions from our fellow folklorists and oral historians as we continue to explore new options in the digital age.