voting?

Comments about LibriVox? Suggestions to improve things? News?
Post Reply
phthoggos
Posts: 1
Joined: December 19th, 2005, 6:42 pm
Location: Portland, OR
Contact:

Post by phthoggos » December 19th, 2005, 10:37 pm

Listening through a mixed assortment of the files already recorded, I've noticed a wide variety of quality in the performances, as is only natural in an open-submission project. Unfortunately, all the recordings are sort of in an undifferentiated mass, and there's no way to tell the great performances from the weak ones. Could a user rating system be implemented so that particularly outstanding readings could be celebrated?

Peter Why
Posts: 4200
Joined: November 24th, 2005, 3:54 am
Location: Chigwell (North-East London, U.K.)

Post by Peter Why » December 19th, 2005, 11:22 pm

I can understand the need, but it might be difficult.

Two sorts of criteria: objective and subjective. Examples:

Objective: background noise, volume.

Subjective: tonal and emotional range (where appropriate), speed, understandability.

We'd need to decide on what they are to be compared against, too.

I'd tend to grade only on the objective, as in the subjective decisions, a) there would be too much variation in opinions, and b) we don't want to put people off reading because they've been down-graded in the past.

Gesine
Posts: 14185
Joined: December 13th, 2005, 4:16 am

Post by Gesine » December 20th, 2005, 2:57 am

phthoggos - yes, I also understand where you're coming from (and thank you for posting a suggestion; it's always good to be reminded of the listener's point-of-view), but...

Like Peter said, 'quality' can be such a subjective thing... for instance, people like having things read to them in different ways. One of my nieces really likes it when the reader puts on voices and accents to portray the various characters, the other niece prefers a 'straight' read... so they might vote completely differently on two 'good' readings. Also, person A finds a certain voice very attractive, person B dislikes fairy tales recorded by male readers, person C is irritated by American accents in Shakespeare readings... you see what I mean.

Recording quality is of course an issue, but then this is a volunteer-based project, and one cannot, nor should one, expect professional sound set-ups. We are lucky to have volunteers to fit in readings in their busy schedules, and if occasionally one hears a dog barking or a child shouting in the background, I think one ought not to complain too much about it.

I think all the volunteers here are doing their very best (some people may be 'better' readers than others, but we will all improve the more readings we do), and we shouldn't discourage anyone from participating. Not that there isn't any room for constructive criticism!

In the future, a search by reader will be implemented (as part of another development on the catalogue page), and this should allow listeners to find the works of their 'favourites.' [Edited to add: this is now in place] Also, as we grow, the same thing may be recorded by different readers, which will give listeners more choices.
Last edited by Gesine on April 23rd, 2010, 3:03 am, edited 1 time in total.
[size=92] "Imagination is more important than knowledge. Knowledge is limited. Imagination circles the world." Albert Einstein[/size]

GordMackenzie
Posts: 601
Joined: September 26th, 2005, 5:50 am
Location: Troy, MI

Post by GordMackenzie » December 20th, 2005, 12:31 pm

Actually, while it is not very user friendly, you CAN write reviews and rate files at Archive.org.

Do a search for "Librivox" in the Archive.org search field and you'll pull up all the Librivox related audio files stored there. If you register on the site, you can write reviews and rate files.

The advantage is that these ratings (if good!) may encourage new listeners that have never heard of Librivox to give some of our recordings a listen.
Gord Mackenzie
gord[dot]mackenzie[at]gmail.com
Librivox Wiki Page: [url=http://librivox.org/wiki/moin.cgi/GordMackenzie]GordMackenzie[/url]

pberinstein
Posts: 604
Joined: September 26th, 2005, 5:47 pm

Post by pberinstein » December 20th, 2005, 6:44 pm

Gesine wrote:
person C is irritated by American accents in Shakespeare readings... you see what I mean.
My understanding is that English people in Shakespeare's day sounded more like Americans than like today's English.
Paula B
The Writing Show, where writing is always the story
http://www.writingshow.com

geneven
Posts: 6
Joined: December 20th, 2005, 1:44 pm
Location: Agoura Hills, CA
Contact:

Post by geneven » December 20th, 2005, 9:56 pm

Why not invite listeners to give their ratings on a 1-5 scale? It would be subjective, but it would give other listeners a rough idea of whether they might be pleased by the recording.

LibraryLady
Posts: 3148
Joined: November 29th, 2005, 5:10 pm
Location: St. Louis, Missouri

Post by LibraryLady » December 21st, 2005, 12:40 am

geneven wrote:Why not invite listeners to give their ratings on a 1-5 scale? It would be subjective, but it would give other listeners a rough idea of whether they might be pleased by the recording.
I really understand the interest in rating recordings becauses there is such a wide variance in style of recording and quality due to the equipment being used. However, this is a project that could not function without volunteers, most of whom are not professionals at this and could easily be deterred from recording if they see they received a low rating. I know that if one of my recordings got a poor rating, I probably would quit volunteering. And, as mentioned earlier, what is pleasing in a recording is very different for different people. We will have in place sometime in the future a mechanism to search the catalog by reader and that will make it easier for listeners to find recordings they like. Until then, however, I'm afraid the general concensus of the administrators is not to implement a voting or rating system.
Annie Coleman Rothenberg
http://www.anniecoleman.com/

"I hear the sound I love, the sound of the human voice." ~Whitman

pberinstein
Posts: 604
Joined: September 26th, 2005, 5:47 pm

Post by pberinstein » December 21st, 2005, 9:37 am

I agree with Annie. You can overdo this ratings thing. I don't think Librivox is in the same category as Amazon.com.
Paula B
The Writing Show, where writing is always the story
http://www.writingshow.com

Stephan
Posts: 1607
Joined: December 18th, 2005, 9:38 am
Location: Leverkusen, Germany

Post by Stephan » December 21st, 2005, 11:21 am

This aspect wasn?t mentioned:

With just 10 ratings, rating might be pretty skewed.
If you?d have 1000 ratings the outcome would be pretty fair and true!
If its a bad rating, then the recording WAS really bad. One could argue that the author would just have to live with it, be open for helping critics and suggestions that at best would accompany a rating-system, and try better next time.

But i agree too, it would always be destructive in a bad case. Not a good system to encourage - so harsh, mathmatic, robotic.
"There is your receit. Black on White. People think your work sucks. Try better next time." Wouldn?t encourage me for sure.

kayray
LibriVox Admin Team
Posts: 11843
Joined: September 26th, 2005, 9:10 am
Location: Union City, California
Contact:

Post by kayray » December 21st, 2005, 11:33 am

Most of our volunteers (even me) were very nervous about submitting their first recordings. If they had to worry about being rated, I think most would not bother.

A few readings that we've collected would probably get harsh reviews -- yet those readers did their best, and everyone gets better with practice. Perhaps some of those readers will become some of our best! A harsh anonymous review would stop me in my tracks.

So, no reviews :)

But -- thoughtful, _constructive_ criticism is probably welcomed by most people, and can be channeled through the admins here or through our librivox contact email.

Kara
Kara
http://kayray.org/
--------
"Mary wished to say something very sensible into her Zoom H2 Handy Recorder, but knew not how." -- Jane Austen (& Kara)

hugh
LibriVox Admin Team
Posts: 8039
Joined: September 26th, 2005, 4:14 am
Location: Montreal, QC
Contact:

Post by hugh » December 21st, 2005, 2:17 pm

hi all I agree with the consensus.

Right now our objectives are more focused on supporting reader volunteers, more than our listeners. That may sound funny, but we are at the very beginning of what we hope will be a huge and important project, with a long-term vision of having thousands of books available for free in audio. In order to achieve that objective, we need to make this place as friendly for volunteer readers as possible, hence to have negative reviews coould be devastating.

One of our early volunteers is a podcaster, and I remember a posting he made about feedback. He'd gotten tons of positive emails from people saying: great show. Then he got one really viscious attack (probably one out of tens of positive stuff) and considered quitting podcasting. I and others sent him positive emails, and he kept on - thank goodness.

In a way LibriVox is a very personally revealing sort of project, people love the work they are recording, and to be told: you did a crappy job reading your favourite author - well that could be very difficult, and would encourage many to stop recording.

Maybe we could consider allowing ratings a year after project completion - but that's a future discussion.

so for now: no official librivox ratings - unless they are glowing :)

Jon Ingram
Posts: 58
Joined: December 6th, 2005, 4:29 pm

Post by Jon Ingram » December 21st, 2005, 2:35 pm

In other areas, I've seen fairly successful attempts to avoid the possibility of hurtful feedback by removing the possibility of anything but positive voting. It's as simple as having an 'I liked this!' link next to all the recordings -- you can then record the number of times the link was clicked, and in some sense this serves as the number of votes for the work.

I suppose you could argue that people may feel disappointed if the work they post isn't as liked as some others, but it would still feel much friendlier than allowing something like marks out of ten.

hugh
LibriVox Admin Team
Posts: 8039
Joined: September 26th, 2005, 4:14 am
Location: Montreal, QC
Contact:

Post by hugh » December 21st, 2005, 3:21 pm

oh, that might work. a specific button to say: "I liked this recording." interesting.

kayray
LibriVox Admin Team
Posts: 11843
Joined: September 26th, 2005, 9:10 am
Location: Union City, California
Contact:

Post by kayray » December 21st, 2005, 3:25 pm

Yes, interesting... something to look into in our spare time ;-)

Kara
Kara
http://kayray.org/
--------
"Mary wished to say something very sensible into her Zoom H2 Handy Recorder, but knew not how." -- Jane Austen (& Kara)

thistlechick
Posts: 6246
Joined: November 30th, 2005, 12:14 pm
Location: Michigan

Post by thistlechick » December 21st, 2005, 3:36 pm

hugh wrote: Maybe we could consider allowing ratings a year after project completion - but that's a future discussion.
*rotflmao* ... project completion?!??! yea, right, like that will ever happen hehehe
~ Betsie
Multiple projects lead to multiple successes!

Post Reply