SUBTITLES? proposal for an alternative recording method

tshirt · Post by **tshirt** » January 5th, 2006, 6:53 pm

Please go to this thread for discussions on this topic:
http://librivox.org/forum/viewtopic.php?t=945

Our goal is implementing an audio software that is suitable for recording
and listening audio reading of written text. The text will be paired with
audio synchronization information, so that it can be displayed as subtitles
for the audio at listening time.

Preliminary list of more advanced features:
* GPL
* Automatic scrolling of text as you read and listen.
* Noise cancelling with software (so that you don't have to invest in expensive mics)
* Help for intonation of the text (e.g. high pitch words displayed in bold)
* Automatic evaluation of your intonation as you record
* Manual evaluation of the intonation as you listen to a book (i.e. to be given as feedback to the person that recorded the book; or to be incorporated into automatic evaluation of other recordings)
* Timestamping of audio and text at recording time.
* Synchronised navigation of text and recorded audio (so that you can find a particular portion of the recording easily for post-editing)
* Ability to run the software on mobile phones and pdas
* Ability to run from the web (e.g. from librivox.org)

Hello All,

I am new to librivox.

I was wondering if everybody uses a computer to record their books. If so
I have a question/proposal:

Would you consider using a program to read each line of the book from your computer?

The program will show one line of the book at a time, and as you read you will
press a key to proceed to the next line. If you would like to read a line again you
will be able to press another button, and the program will take care of the
bookkeeping.

Meanwhile the program will be recording your voice and keeping track of which
line corresponds to which portion of the recording. This way we will be able to
make the audio recordings searchable using their text, and easy to process by
many other applications. Then people can write other programs to make the
recordings more accessible.

Please let me know if you would be willing to use such a program for your
recordings. If people think it will be useful, I will invest some time to build such
a program. Please vote to share your opinion or write a follow-up.

Once this tool is available:
*audio books will be searchable with their text
*people can record their blogs to podcasts as subtitles
*one day the whole web and all digital text will be accessible to visually
impaired!!!
* dyslexic people can use the subtitles to read books.

ChipDoc · Post by **ChipDoc** » January 5th, 2006, 7:08 pm

Welcome to LibriVox, tshirt! It's great to have fresh voices in the mix!

I'm certain that this is something which many folks would find useful, but it just wouldn't work for me. As I speak the words I'm already looking ahead for what's next. I need a fair amount of text on the screen to be able to do this - more than a single line would provide.

On the other hand, I do read directly from the computer screen. I can size the text any way I like and the thumbwheel moves it silently up and down as I read along.

But everyone's different and we each have our own comfort zone. I'm sure that a lot of folks would be delighted with such a program.

tis · Post by **tis** » January 5th, 2006, 7:18 pm

Agree with ChipDoc, but I wonder if there might not be some way to work with this. Perhaps if a couple of paragraphs at a time were shown? Then the search could still be accurate to a paragraph, not just a line?

One other issue is that the program would have to support editing and post processing (or there would need to be some way to use a tool such as audacity to do the editing).

tshirt · Post by **tshirt** » January 5th, 2006, 7:19 pm

ChipDoc wrote:Welcome to LibriVox, tshirt! It's great to have fresh voices in
the mix!

I'm certain that this is something which many folks would find useful, but it just
wouldn't work for me. As I speak the words I'm already looking ahead for what's
next. I need a fair amount of text on the screen to be able to do this - more than
a single line would provide.

I think this can be solved really easily. Just think of the window will have one
line (or a few lines) in the middle that is clear, and the lines above and below
that line will be under a transparent shadow. The lines under the shadow will
still be readable.You will just need to press the cursor to make the next line
scroll into the clear area. And you don't need to hit the cursor at the exact time
when you start reading the next line; it's just an approximate time stamp. And
even that approximate time stamp will be invaluable!

tshirt · Post by **tshirt** » January 5th, 2006, 7:36 pm

Here's the description of an interesting annotation tool:
http://www.plasticbag.org/archives/2005/10/on_the_bbc_annotatable_audio_project.shtml
However this tool is about pre-recorded audio. It does not record audio, but
annotates existing audio really well. Oh, and it's internal to BBC

There are really nice ideas like version control and wiki-like annotation pages.

tshirt · Post by **tshirt** » January 5th, 2006, 7:54 pm

tis wrote: One other issue is that the program would have to support editing and post processing (or there would need to be some way to use a tool such as audacity to do the editing).

I've already sent an email to audacity authors to ask whether they
would be willing to invest some time to write a plugin to their tool.

But isn't it easier to do the post-editing on a line/sentence/paragraph basis? ie.
replacing the whole recording for a sentence with an alternative. This will
probably simplify the post-editing and save a lot of time. Please correct me
if I am wrong.

Post by **hugh** » January 5th, 2006, 7:58 pm

if it's free software (gpl) I'm all over it. actually it sounds like a truly fantastic sort of development I could imagine being useful in spades in all sorts of applications. gpl or not

...

tshirt · Post by **tshirt** » January 5th, 2006, 8:16 pm

hugh wrote:if it's free software (gpl)

Of course it will be GPL! yay!

Peter Why · Post by **Peter Why** » January 6th, 2006, 12:22 am

Because of computer noise being picked up, I record as far away from my computer as possible, so *have* to use a printout. I'd have a go at the program if I get a quieter system.

Post by **hugh** » January 6th, 2006, 6:31 am

tshirt, how far along are you on this software? if you are getting along I'll pledge that we'll find a couple of Test project - at least for a couple of chapters of books.

I think it could be an amazing addition to the usefulness of our texts, esp non-fiction. So yes, we'll join in a testing project whenever you're ready.

GordMackenzie · Post by **GordMackenzie** » January 6th, 2006, 7:48 am

It's funny ... I was thinking of something similar prior to your post, tshirt!

The concern I had was what format the final content would be in? How would listeners experience the audio ... would they need a special "player" that would show text as the audio plays?

Would the output be a set of mp3 files (one for each sentence/block of text cataloged) or a single file that is tied to some sort of index... could it output multiple formats?

Many of us use Audacity for its audio editing tools. How would such editing be done with the tool you propose? How would flubbed readings be replaced, re-edited... and continue to be synced to the text...

btw, I like the idea and think it has great potential... I'm just throwing out questions for thought-generation.

Post by **hugh** » January 6th, 2006, 7:57 am

Jon Udell is looking at "bookmarking" mp3 files ... but I don't know much about it, see:

http://www.oreillynet.com/pub/a/network/2004/09/03/primetime.html

GordMackenzie · Post by **GordMackenzie** » January 6th, 2006, 8:39 am

I don't know much about the MP3 file format, but as you can stick ID3 tags before or after the audio content without problem, that it should be possible to stick the entire text of the audio there as well. Most players would simply ignore this additional text(?)

The text could be marked up with time codes in a format that would allow a player/reader application to display the correct text at the correct time... I suspect that the DAISY system may do something similar, but I haven't been able to find solid information on precisely what DAISY does...

http://www.daisy.org/

DAISY is a format used to produce talking books for the blind and those with reading disabilities.

DAISY is an ANSI specification, and if anyone wants to plow into the documentation, you should be able to find out how it works. I don't know how compatible it would be with MP3 players...

I'm surprised that they don't allow free access to the production tools and readers that they clearly already have...

GordMackenzie · Post by **GordMackenzie** » January 6th, 2006, 9:29 am

Just so folks know, I have contacted the Daisy Consortium and AHEAD to see if they have advice on how we can easily (and cheaply!) create DAISY standard ebooks.

I can't believe that there isn't already an Open Source Project for the creation of such works.

tshirt · Post by **tshirt** » January 6th, 2006, 11:28 am

GordMackenzie wrote:I can't believe that there isn't already an Open Source
Project for the creation of such works.

I checked everywhere I know to see if there is, but couldn't find any.

I don't know how it works quite well, but avi video files can be added subtitles by
using a text file with time annotations:
http://www.doom9.org/index.html?/sub.htm

This could be a better choice than going to a proprietary ebook format, because
the audio will still be in the mp3 or ogg format.

LibriVox Forum

SUBTITLES? proposal for an alternative recording method

Would you use this program to record your books?