How Often are Librivox Books Listened To?

[TedL]

Thank you Peter. It would definitely be helpful to have someone who is up-to-date on AI give an estimate as to when it will be possible for machines to do cataloging. I would think librarians are watching AI closely, so I'd guess it is already being discussed in their professional journals.

Meanwhile I'll move ahead with the proposal that I'm working on; a step-by-step procedure for doing this with volunteers. Yesterday I found that Gutenberg.org already has LOC subject headings in most of their books. Since most of our books were originally Gutenberg books, this looks very positive.

I'll put my proposal on my website so I can use images to illustrate the process, and I'll estimate how long it would take to do each book. I'll post a note here when its ready.

Best regards,

[TheBanjo]

For what it's worth, I just looked up the Library of Congress record for the last book I read for Librivox, Horace Barnett Samuel's translation of "The Red and the Black", which is certainly one of the most famous French novels written in the last two hundred years. As far as I can see, the LOC catalogue record contains no tags at all for this book (https://catalog.loc.gov/vwebv/staffView?searchId=1636&recPointer=1&recCount=25&bibId=8736555).

It would be interesting to know how many of, say, a random sample of 100 texts in our collection similarly have no LOC tags. Perhaps nearly 100% do, and "The Red and the Black" is a rare exception — or perhaps the tagless rate is higher than that. This is surely a relevant consideration with regard to the approach TedL is proposing here.

[TheBanjo]

As per my previous post, there appear to be zero LOC tags for "The Red and the Black", translated by Horace Barnett Samuel.

Out of interest, then, I just asked ChatGPT the following question:

If you had to generate Library of Congress MARC tags for "The Red and the Black", translated by Horace Barnett Samuel, what 650 and 651 tags would you suggest?

ChatGPT responded:

As of my last update in January 2022, I don't have access to specific publication data like the publisher, edition, or ISBN of "The Red and the Black" translated by Horace Barnett Samuel, which would be needed to generate a Library of Congress MARC record. However, I can provide a general template for these tags that you can fill in with the appropriate details.

MARC 650 is used for subject headings. For "The Red and the Black," some possible subject headings might include:

French literature -- 19th century
Ambition -- Fiction
Social classes -- Fiction
Love stories
Historical fiction
Please note that these are just examples and may not be the most accurate or appropriate subject headings for your specific edition.

MARC 651 is used for geographical headings. If the book is set in a specific location, you could use a geographical heading for that location. For example:

France -- Social life and customs -- 19th century
Again, these are just examples and may not be the most accurate or appropriate geographical headings for your specific edition.

If you have more specific information about the edition of "The Red and the Black" you're working with, I can provide more accurate suggestions for the 650 and 651 tags.

To get this result, I did not even need to feed ChatGPT the text of "The Red and the Black". It clearly already "knew" a lot about this text. That would almost certainly NOT be the case with some of the more obscure texts our readers turn into audiobooks, but as I've made clear already, the latest development in this rapidly changing field, called "Gemini 1.5", is capable of ingesting a large amount of text, audio or video material on the spot (ie, material is has never previously seen or been trained on) and analysing it most "intelligently".

[TheBanjo]

I have a couple more issues for Ted to consider.

First, how are upi proposing to create topic tags for audiobooks that derive from multiple texts, such as themed or sometimes not particularly themed collections of poetry (potentially from different times and places) or fiction? [This is a special case, I guess, of the more general scenario where a Librivox 'audiobook' can have no corresponding LOC entry.]

Second, have you considered a programmatic way of extracting tags from such source materials as do have LOC tags via the LOC API? It might, in theory, be possible for someone with direct access to the database underlying the librivox.org collection to write a script that would iterate through the whole collection, then for each entry interrogate the LOC database via its "JSON/YAML for LoC.gov" API (see https://www.loc.gov/apis/json-and-yaml/) and extract any topic tags that might exist in the LOC catalogue. I haven't played around with the LOC API myself, but it would be pretty surprising if it didn't allow for the extraction of the kind of catalog info we're looking for here. The main technical difficulty would be handling the situation that arises where there are multiple 'hits' in the LOC database for a given book title/book author/language/translator(if any) combination. It would probably be necessary, in such cases, to extract the LOC topic tags for every possible 'hit', and decide later, manually, which of the various 'hits' represented the best match to the book in the Librivox catalog. Any half-competent second year computer science student should be capable of writing such a script.

[TedL]

Peter,

Your ChatGPT enquiry was an eye-opener. I didn't realize you could simply ask it for LOC subject headings and receive good ones. A fantastic idea.

As to a mass transfer of records from LOC - that's going way beyond my technical knowledge. I don't yet even know what metadata our Admins submit to Internet Archive. My searches on the LOC catalog often turn up records with no subjects listed. I suspect that its because the books are old, and so are the entries. However, I'm not denying it might work for records with subjects, but we would need feedback from people who work on the development side about doing mass transfers of data from LOC's database to IA's database.

I have not done enough searches to say for certain, but it looks like the the best source of subjects for books is Gutenberg.org. The second best is OCLC/WorldCat, and the third best is Open Library / Internet Archive. (It appears that Open library creates the records that Internet Archive uses, but I'm not certain.)

Open Library seems to be trying to use a 4-part "faceted" approach that OCLC is working on, which provides separate fields for subject, people, places, and times. It results in terms that don't necessarily exactly match LOC terms. However, it doesn't appear that either organization has 4-part subject terms on many of these old public domain books. Open Library says that using the 4 parts helps subject searches, but such subject searches look complicated, and I question whether most users would know how. Anyway, this is something we should decide about.

I have looked at a lot of Internet Archive book records as I work on my own website, and its clear that the list of subjects for each varies a lot in quality. Whatever we do, it will fall short of perfect. But if we keep in mind that the objective is mainly to lead users to more books on their favorite subjects, that will bring many of our audiobooks released in previous years back into circulation.

Finally, I want to point out that, despite 1,300+ views of this thread so far, you are the only person to reply in a positive way about this proposal or to make suggestions about how to accomplish it. No one from Librivox management has indicated any interest in updating audiobook metadata to enable subject searches. It would be helpful to know up front if such a proposal will even be seriously considered.

Regards

[TriciaG]

Posting my own opinion as an admin; I'm not speaking for LV as a whole or for the other admins.

I see some value in this, but I don't know if it would be as straightforward as you're implying. I'm awaiting your workflow proposal for more information. But then, since you don't know the structure of the LibriVox catalog/workflow system, it would be harder for you to make an accurate proposal since you'd be working off assumptions.

Here's how it works from our end:

Have you ever started a LV project (filled in the New Project Template Generator)? The key words are entered there by the soloist/BC. It's a fairly free-form field, where pressing Enter sets each keyword/phrase, and anything can be entered. (Sometimes BCs give the admin the keywords right before cataloging; they get entered into the database then, but it's essentially the same process.) These keywords are stored in our database for the project, and are also sent to Archive. Once the project is set up, only the admins (MCs) are able to change them; BCs, soloists, etc. don't have access to the database.

In our database: When you go to Advanced Search on our site (go to, say, librivox.org and click on "Advanced Search" in the upper right corner), you can search projects by keyword. So they do serve some limited functionality on our own site. However, they're not visible on the project's catalog page.

On Archive: When the project is cataloged, the system automatically includes "LibriVox" and "audiobook" as you have observed, plus adds the keywords, if any, entered into our database.

Originally, if I recall correctly, these keywords were meant to be somewhat like those "keyword clouds" or "tags" that were helpful in the early 2000s and 2010s. I don't know that they're popular anymore (and we never meant to display them as a cloud). The genres were meant to be more like the LOC-type categories, although much less detailed. (The genre pick lists, especially for non-fiction, are a pain to work with now; I can't imagine how painful it would be if they were more elaborate.)

Yes, we let the BCs choose their own key words, and some are quite verbose and detailed while others (IMHO) are inadequate or even silly. But I'm not going to try to police that, as it's a minor part of our process. If a soloist or BC gets overwhelmed on even starting a book, they might not record the book, which hurts our objective. So it's a balance between making the process simple enough for the newer BC/soloist and making them do all sorts of paperwork and research to make the project's database information as comprehensive and accurate as possible.

It comes up in a lot of suggested side projects that we need to make the books more accessible, prettier, better quality, or whatever else, for the listener. While that would be nice, our main objective - our prime directive - is to make audiobooks. If a change to our processes or a side project draws us away from making audiobooks, we'll choose the audiobook production over the prettier, more accessible, more professional-quality, etc. change to our processes.

All that said -- and as I say, I'm not speaking for other admins; they might overrule me on this -- I see value in adding more standardized keywords to the database. But I don't know how much sanctioned support there would be for it. If an admin or two decides it's worthwhile, they may choose to help on the side, as they have time.

And there's the problem of new projects. While it would be helpful to encourage soloists/BCs to add LOC categories as their keywords, we wouldn't require it (as I said before, it would quickly overwhelm a new soloist who's barely keeping up with the learning curve on all this LV stuff to start with, and who is barely able to start a new solo project of their own as it is). So that means that we'd have a constant flow of new projects without the LOC keywords, and so the keywords would have to be maintained as well going forward. Asking the admins to do it wouldn't be feasible, as even some of us are guilty of not adding keywords to our own projects - you can't expect even us to accurately fill them in for others' projects.

I probably have more thoughts bouncing around about this and more information helpful to your development of a proposed solution, but for now I'm tapped out.

[ej400]

In our database: When you go to Advanced Search on our site (go to, say, librivox.org and click on "Advanced Search" in the upper right corner), you can search projects by keyword. So they do serve some limited functionality on our own site. However, they're not visible on the project's catalog page.

Would it be possible to create some sort of "guideline," then, for BC's when creating keywords? Obviously, I wouldn't want newer BC's to feel discouraged and have multiple steps, or new soloist's BCing their first project have to know the ins and outs of the book, but perhaps there is something outside of the database before catalogue we could do? Another question is, though, how necessary is this?

For example, after a DR is completed, BCs create a cast list. So perhaps, before cataloguing, is there a potential for a small amount of "policing" in putting some or more keywords to help with the advanced searching?

I'm not exactly sure how folks would go about searching for new books. Typically, if I wanted to listen to something new, I'd search genre specific, looking at genre/subject line on the librivox page. Otherwise, it would be the advanced search function, but fwiw, that is only me.

FWIW,
Elijah

[TriciaG]

Would it be possible to create some sort of "guideline," then, for BC's when creating keywords? Obviously, I wouldn't want newer BC's to feel discouraged and have multiple steps, or new soloist's BCing their first project have to know the ins and outs of the book, but perhaps there is something outside of the database before catalogue we could do? Another question is, though, how necessary is this?

There's a line in the wiki about it: https://wiki.librivox.org/index.php?title=Project_Template_Generator

Keywords:
you can enter as many keywords as you like, but more than 6-8 is probably excessive.
Hit return after each keyword or phrase.
Keywords will be added on the archive.org page and will be also now be searchable in the new LV catalog.

This could be expanded/clarified.

For example, after a DR is completed, BCs create a cast list. So perhaps, before cataloguing, is there a potential for a small amount of "policing" in putting some or more keywords to help with the advanced searching?

Some BCs do this already (wait until cataloging time, then supply the keywords). That's fine; the MC enters those keywords into the database at that point.

[redrun]

Finally, I want to point out that, despite 1,300+ views of this thread so far, you are the only person to reply in a positive way about this proposal or to make suggestions about how to accomplish it. No one from Librivox management has indicated any interest in updating audiobook metadata to enable subject searches. It would be helpful to know up front if such a proposal will even be seriously considered.

I apologize again for side-tracking us on detail earlier. My intention was to help clarify your proposal and the estimates you mentioned, and several other admins have chimed in with potential issues so you can address or steer clear of them. That doesn't seem to have been particularly helpful to you, so as I mentioned then, I've since stepped back to wait for the more detailed proposal.

Regarding scripting and LLM "AI", mentioned since then: I'm in favor of automation where it makes sense to use it. Without getting too far into the weeds:
It would take a lot of volunteer work to update the back-catalog, no matter what tooling we use, but scripting could certainly be helpful. If someone has both the expertise and the time to develop and test a script like this, I'll be glad to talk specifics. LLMs are unlikely to fly, as aside from ability and cost, we also have concerns that most personal-scale or private projects do not.

[DACSoft]

I've been following this discussion closely, although I'm a volunteer at DP primarily as a hobby which I enjoy, and to help fulfill LV's mission of recording all public domain books (or as many as we possibly can). It doesn't bother me if I'm the ONLY ONE who enjoys listening to the books in which I participate, mostly solos. That being said, I use the keyword function on ALL my solos to describe the content and subject matter of the books I record.

While I'm not against a proposal of the ability to search on "standardized" subject searches, I would like to see that as an addition to our current keyword searches. I've looked at the LOC subject classifications attached to many of the books I've submitted to PG. In many cases, some of the "subjects" linked have nothing (IMHO) to do with the subject matter and content of the book; others being way too broad.

I wouldn't want to see this as a replacement to our current keywords (which become "topic" or "subject" searches at TIA and keyword searches at "LV." Who better than the soloist or BC (possibly with the assistance of the DPL) to know what their project is about, and more accurately describe the book's content?

And above all, the cost/benefit analysis of the change is important as it applies to the LibriVox mission and goals.

FWIW,

[TedL]

Posting my own opinion as an admin; I'm not speaking for LV as a whole or for the other admins.

I see some value in this, but I don't know if it would be as straightforward as you're implying. I'm awaiting your workflow proposal for more information. But then, since you don't know the structure of the LibriVox catalog/workflow system, it would be harder for you to make an accurate proposal since you'd be working off assumptions.

Here's how it works from our end:

Have you ever started a LV project (filled in the New Project Template Generator)? The key words are entered there by the soloist/BC. It's a fairly free-form field, where pressing Enter sets each keyword/phrase, and anything can be entered. (Sometimes BCs give the admin the keywords right before cataloging; they get entered into the database then, but it's essentially the same process.) These keywords are stored in our database for the project, and are also sent to Archive. Once the project is set up, only the admins (MCs) are able to change them; BCs, soloists, etc. don't have access to the database.

In our database: When you go to Advanced Search on our site (go to, say, librivox.org and click on "Advanced Search" in the upper right corner), you can search projects by keyword. So they do serve some limited functionality on our own site. However, they're not visible on the project's catalog page.

On Archive: When the project is cataloged, the system automatically includes "LibriVox" and "audiobook" as you have observed, plus adds the keywords, if any, entered into our database.

Originally, if I recall correctly, these keywords were meant to be somewhat like those "keyword clouds" or "tags" that were helpful in the early 2000s and 2010s. I don't know that they're popular anymore (and we never meant to display them as a cloud). The genres were meant to be more like the LOC-type categories, although much less detailed. (The genre pick lists, especially for non-fiction, are a pain to work with now; I can't imagine how painful it would be if they were more elaborate.)

Yes, we let the BCs choose their own key words, and some are quite verbose and detailed while others (IMHO) are inadequate or even silly. But I'm not going to try to police that, as it's a minor part of our process. If a soloist or BC gets overwhelmed on even starting a book, they might not record the book, which hurts our objective. So it's a balance between making the process simple enough for the newer BC/soloist and making them do all sorts of paperwork and research to make the project's database information as comprehensive and accurate as possible.

It comes up in a lot of suggested side projects that we need to make the books more accessible, prettier, better quality, or whatever else, for the listener. While that would be nice, our main objective - our prime directive - is to make audiobooks. If a change to our processes or a side project draws us away from making audiobooks, we'll choose the audiobook production over the prettier, more accessible, more professional-quality, etc. change to our processes.

All that said -- and as I say, I'm not speaking for other admins; they might overrule me on this -- I see value in adding more standardized keywords to the database. But I don't know how much sanctioned support there would be for it. If an admin or two decides it's worthwhile, they may choose to help on the side, as they have time.

And there's the problem of new projects. While it would be helpful to encourage soloists/BCs to add LOC categories as their keywords, we wouldn't require it (as I said before, it would quickly overwhelm a new soloist who's barely keeping up with the learning curve on all this LV stuff to start with, and who is barely able to start a new solo project of their own as it is). So that means that we'd have a constant flow of new projects without the LOC keywords, and so the keywords would have to be maintained as well going forward. Asking the admins to do it wouldn't be feasible, as even some of us are guilty of not adding keywords to our own projects - you can't expect even us to accurately fill them in for others' projects.

I probably have more thoughts bouncing around about this and more information helpful to your development of a proposed solution, but for now I'm tapped out.

This is really helpful Tricia, and I'll consider all of it as I move ahead.

Other "Admins" have also said that your focus is on producing more audiobooks. If so, you should be reviewing the process by which you bring on new volunteers. We can get bogged down in the day-to-day details of work and forget to stop now and then to review how the organization looks to new "employees".

Librivox got 640 new volunteers last year, and Librivox claims a total of 13,600 volunteers total. Tremendous numbers! But Librivox produces just 1,000 books a year, and it appears to me that most of the work is done by several dozen dedicated people. So what are all these other volunteers doing? And what is being done by Librivox to maximize their contributions and make them feel like part of the organization?

The Librivox main page, https://wiki.librivox.org/index.php?title=Main_Page , largely devoted to explaining how to record, has links a couple of dozen other articles. The "Instructional Videos" page has links to over 60 videos, with another link to Adrian Stephens's 18 additional videos. And the Audacity software program has its own webpage with dozens more articles. I think most new volunteers are simply overwhelmed with the complexity of learning to record, which in fact does not have to be that hard. I think you could be surveying volunteers to find out how the onboarding process could be improved. Then do a package that just covers the basics for new readers. Clean out the rest, or put them into a separate "Advanced" section.

I've noticed that Book Coordinators and Listeners are very encouraging and helpful with new volunteers, but I'm not sure all of them are providing the best advice. For example, I've never seen one recommend that a reader use Punch and Roll recording in Audiovox, which greatly eases the burden of editing. Some don't seem aware of the loudness setting that automatically modifies the volume of a recording to 90 db. Are Book Coordinators getting refresher training?

A short checklist for readers to print out and have at their side would avoid frequent simple errors that are time-consuming to fix. The checklist would include things like "One-half second silence at beginning" and "5 seconds silence at end". Repeatedly making simple mistakes like that can make a new reader feel embarrassed about their own competence, driving them away. Help them to help themselves.

And while I'm getting things off my chest, one final point. Clean up these Forums by archiving the old stuff. Going into the forums section is like walking into my basement storage area.

Enough said. Regards,

[TheBanjo]

And while I'm getting things off my chest, one final point. Clean up these Forums by archiving the old stuff. Going into the forums section is like walking into my basement storage area.

Enough said. Regards,

Hi Ted,

I'm afraid I can't let this pass. You've just been asserting, in effect, that administrators on this site have been paying your proposal less attention than it warrants. Along comes TriciaG and gives you a very generous, detailed and helpful response, and you conclude your reply to her by addressing her in this abrupt and demanding way. I wouldn't accept that treatment from anyone, face to face, and I don't see. why anyone should have to accept it on this online forum.

For what it's worth, I've found every single one of the administrators with whom I've worked on this site to be unfailingly helpful, generous with their time — and also invariably civil and polite, including in circumstances where I've been guilty of some stuff-up or another. If you want to have any hope of forming effective working relationships with others on this platform (and your mooted project suggests that you might like to) can I suggest that in future you address others with the kind of civility with which you yourself, presumably, would like to be treated?

With regards to this so-called "basement storage area", I would observe only that this "basement" would be more easily navigable if users started new threads for new topics of conversation. So far, under "How Often are Librivox Books Listened To?" you have initiated discussion on three completely different topics. This doesn't help to make this "basement storage area" any more navigable.

[TheBanjo]

For anyone who doubts that AI is being used already to address this kind of problem, can I suggest they enter the following search in Google "automatic book tagging with library of congress categories using ai"? There's a lot of work being undertaken right now in this space, and some of it is already open source (for example, https://github.com/ahmad-PH/auto_lcc )

Is there anything we can use right now off-the-shelf at no charge that will do an adequate job for us? No, probably not, particularly given that we don't have ready access to raw text versions of all audiobooks — but given the rapidity of progress in this area, it seems to me there's a realistic possibility that a fully open source solution, even to LOC-cataloging pure audio resources, will be available within five years' time.

[TedL]

Please see my proposal for creating a volunteer team to add subject headings to Librivox audiobook records at the Internet Archive, at my website. https://centurypast.org/librivox-proposal/

[Winnifred]

I've been trying (not terribly successfully) to follow this discussion. In theory, it sounds good (the end result seems like it would be a net positive), but it does sound like a lot of work and most of it would fall on our Admins (the only "management" that Librivox has). There are only 29 of them, and I'm not even sure that all of them are currently active. Plus, some might not be interested in this kind of work. The ones who are active are often coordinating multiple book projects of their own, reading for group and solo projects, and PL/DPLing, in addition to serving as Meta Coordinators for a lot of projects. I don't see how they'd have time to take on entering the requisite data into the New Project Generator and Internet Archive (possibly dumb question: do they even have access to do this last on IA?)

Below are my <queries/comments> on one section of your proposal:

"How LOC Subject Headings will be Entered into Audiobook Records
For new books (still in process), Team Leaders will copy the LOC subject headings into the tags box in the New Project Template Generator before the book <each book individually?> is completed. Book Coordinators will no longer enter anything into the tags box. <This would require Team Leaders to have access to the "New Project Template Generator" for each project, after the Book Coordinator has completed their part of the form, but before the BC posts the results to the New Projects Launch Pad. You'd have to reprogram the generator to allow for a multi-stage process like that. Wouldn't Team Leaders then have to be given a separate level of access or be Admins themselves? This additional step would add a layer of back and forth that would make setting up a solo or group project take longer and be even more complicated.>

For existing audiobooks already in the Internet Archive (IA) Librivox collection, Team Leaders will submit the list of LOC subject headings with their book title to one of the designated Librivox ‘Admins’ who are authorized to update book records. <So the Team Leaders would need to comb through (or direct their team to comb through) all books on IA, and set up a list of subjects for each book? That makes sense (though it's a lot of work).>

The Admin will update all versions of that book in the collection, by adding the subject headings to the ‘Topics’ field. Any subjects already in the ‘Topics’ field will be deleted. <This is rather tedious data entry work, isn't it? I have to wonder whether it'll be easy to get enough volunteer Admins to do it.>

Management needs to revise the instructions above the Tags box in the New Project Template Generator, and may need to revise Help documents referring to that form. <There is no "Management" here. The Admins are our management. They're all volunteers. So this would fall on them too.>

If there's a way to have AI generate the subject headings automatically somehow, that would reduce the amount of work involved, because manually assessing each of the books on IA to determine which subject heading applies seems to me to be a lot of work that could take an awfully long time. Most of the volunteers on here are doing what they do because they enjoy it. I'm not sure how many librarians and indexers we have around who would embrace this kind of task, and I know from experience that volunteer management/engagement is not an easy feat.

Sorry to rain on your parade, Ted, but I think the process might need some more refining to be workable.

Cheers,