LibriVox API Discussion Thread

Comments about LibriVox? Suggestions to improve things? News?
williamjones
Posts: 2248
Joined: April 26th, 2016, 7:47 pm
Location: Florida

Post by williamjones »

QUASI-DISASTER: THE API SEEMS TO BE SHOT TO HELL!!

Most API calls which I was using as recently as YESTERDAY without encountering errors, are now failing:

Example 1: Detecting what the End Of Book Data will look like:
https://librivox.org/api/feed/audiobooks/?offset=15500&limit=1000&fields={id}
Example 2: simple call
https://librivox.org/api/feed/audiobooks/?&offset=49

Both of these calls and a significant number of others yield:
=============================
A PHP Error was encountered

Severity: Notice

Message: Undefined index: url_other

Filename: libraries/Librivox_API.php

Line Number: 142
=============================
A PHP Error was encountered

Severity: Notice

Message: Undefined index: url_other

Filename: libraries/Librivox_API.php

Line Number: 142
=============================
A PHP Error was encountered
-- Bill Jones

When you think that you have exhausted all possibilities, remember this: you haven't.
--- Thomas Edison
annise
LibriVox Admin Team
Posts: 38542
Joined: April 3rd, 2008, 3:55 am
Location: Melbourne,Australia

Post by annise »

Just before we panic - were you trying during the "shut down" when only the forums were running ? the global announcement is taken down now, but it started about 12:33 hours from the date on this post and ran about 3 hours

Anne
williamjones
Posts: 2248
Joined: April 26th, 2016, 7:47 pm
Location: Florida

Post by williamjones »

Nope, I tried to do some testing about 30 minutes before I posted my Quasi-Disaster message.
-- Bill Jones

When you think that you have exhausted all possibilities, remember this: you haven't.
--- Thomas Edison
TriciaG
LibriVox Admin Team
Posts: 60512
Joined: June 15th, 2008, 10:30 pm
Location: Toronto, ON (but Minnesotan to age 32)

Post by TriciaG »

This is related to the updates yesterday. I'll report the error.

Since both guys who know anything about it have full time jobs, it won't be fixed until this evening at the earliest. :)
Serial novel: The Wandering Jew
Medieval England meets Civil War Americans: Centuries Apart
Humor: My Lady Nicotine
williamjones
Posts: 2248
Joined: April 26th, 2016, 7:47 pm
Location: Florida

Post by williamjones »

TriciaG wrote: April 6th, 2021, 5:40 am This is related to the updates yesterday. I'll report the error.

Since both guys who know anything about it have full time jobs, it won't be fixed until this evening at the earliest. :)
Thanks for the update, Tricia. I'm in contact with Anne, too.

This "upgrade" seems like a downgrade for the API.

Everything else seems to be fine, tho.
-- Bill Jones

When you think that you have exhausted all possibilities, remember this: you haven't.
--- Thomas Edison
TriciaG
LibriVox Admin Team
Posts: 60512
Joined: June 15th, 2008, 10:30 pm
Location: Toronto, ON (but Minnesotan to age 32)

Post by TriciaG »

This "upgrade" seems like a downgrade for the API.
It's just a bug that will get squished. The newer software version has more constraints, so it generates more errors. We squished two bugs already (one during the outage, which was why it took longer than scheduled, and one shortly after it went live dealing with authors in collections) - bugs that if we had had a more robust or formal testing than we did, we may have caught beforehand. This was simply another one that snuck through. :)
Serial novel: The Wandering Jew
Medieval England meets Civil War Americans: Centuries Apart
Humor: My Lady Nicotine
williamjones
Posts: 2248
Joined: April 26th, 2016, 7:47 pm
Location: Florida

Post by williamjones »

TriciaG wrote: April 6th, 2021, 6:09 am
This "upgrade" seems like a downgrade for the API.
It's just a bug that will get squished. The newer software version has more constraints, so it generates more errors. We squished two bugs already (one during the outage, which was why it took longer than scheduled, and one shortly after it went live dealing with authors in collections) - bugs that if we had had a more robust or formal testing than we did, we may have caught beforehand. This was simply another one that snuck through. :)
Sadly, I have to live with the reality that the API doesn't bother a lot of people, nor, as Anne would say over and over: it's not our prime directive.

Alas, this outage means I'll have to go back to a steady regimen of recording and PLing.

If you happen to think of it, when the fixes have been made, drop me a line, please.
-- Bill Jones

When you think that you have exhausted all possibilities, remember this: you haven't.
--- Thomas Edison
CSCO
Posts: 393
Joined: April 6th, 2010, 10:48 am
Location: Toyokawa, Japan

Post by CSCO »

I'm not CodeMaster, but (ex-)CSCO. I've been a non-tech worker for 12 years. So I would say to myself...
Has_Wiki_Page_But_No_LV_url.xlsx - list of books which have a wikipedia page but no LV_URL page (157 books)
A good PHP script retrieved a bad data, an inconsistent URL record, from a good DB?

If someone didn't define the error's severity as Notice in the script, the error's real severity is Notice as it was output. (Notice is a notice. The error is not a severe one. Although PHP built a catastrophic page, server down couldn't be occurred. Charlie Brown sighs. The server sighed at that moment: It's not a handsome page. *SIGH*)

Oh, I must go to bed.
Last edited by CSCO on April 6th, 2021, 11:41 am, edited 1 time in total.
!!!!!!.!!!!!!.!!!!.!!!!!!!!!..!!!.!!!!!!!!!!!...!!!!!!!!!.!!!!!!.!!!!.!!!!!!.!!!!
No way. He stole away a pretty thing, you know.
That's your heart.
!!!!.!!!!!!.!!!!.!!!!!!!!!..!!!.!!!!!!!!!!!...!!!!!.!!!!!!.!!!!!!!!.!!!!!!.!!!!!!
TriciaG
LibriVox Admin Team
Posts: 60512
Joined: June 15th, 2008, 10:30 pm
Location: Toronto, ON (but Minnesotan to age 32)

Post by TriciaG »

The error is severe enough that no data gets through. :)
Serial novel: The Wandering Jew
Medieval England meets Civil War Americans: Centuries Apart
Humor: My Lady Nicotine
TriciaG
LibriVox Admin Team
Posts: 60512
Joined: June 15th, 2008, 10:30 pm
Location: Toronto, ON (but Minnesotan to age 32)

Post by TriciaG »

williamjones wrote: April 5th, 2021, 6:22 pm Most API calls which I was using as recently as YESTERDAY without encountering errors, are now failing:

Example 1: Detecting what the End Of Book Data will look like:
https://librivox.org/api/feed/audiobooks/?offset=15500&limit=1000&fields={id}
Example 2: simple call
https://librivox.org/api/feed/audiobooks/?&offset=49

Both of these calls and a significant number of others yield:
=============================
A PHP Error was encountered

Severity: Notice

Message: Undefined index: url_other

Filename: libraries/Librivox_API.php

Line Number: 142
=============================
A PHP Error was encountered

Severity: Notice

Message: Undefined index: url_other

Filename: libraries/Librivox_API.php

Line Number: 142
=============================
A PHP Error was encountered
I think this is fixed.
Serial novel: The Wandering Jew
Medieval England meets Civil War Americans: Centuries Apart
Humor: My Lady Nicotine
williamjones
Posts: 2248
Joined: April 26th, 2016, 7:47 pm
Location: Florida

Post by williamjones »

TriciaG wrote: April 6th, 2021, 5:17 pm
I think this is fixed.
Yaaaaay! it appears so!
Hats off to the backroom boys!
-- Bill Jones

When you think that you have exhausted all possibilities, remember this: you haven't.
--- Thomas Edison
knotyouraveragejo
LibriVox Admin Team
Posts: 22067
Joined: November 18th, 2006, 4:37 pm

Post by knotyouraveragejo »

and girls! :wink:
Jo
williamjones
Posts: 2248
Joined: April 26th, 2016, 7:47 pm
Location: Florida

Post by williamjones »

knotyouraveragejo wrote: April 6th, 2021, 6:29 pm and girls! :wink:
OH, yes! Mustn't forget them. Hell, who COULD forget them?!?!? Grace Hopper, Ada Lovelace, Adele Goldberg, Mary Lou Jepsen, etc. etc. I might add to this list a stunning blonde working with me in the sub-basement under the Louisiana State Capitol building. Barbara Something. She was so darned smart!

AND, in almost every corporation that I've worked in, there was one secretary or keypuncher or aide who could have run the whole company as well as or better than the guys in the big offices.

My recipe for World Peace: put a woman in charge of all the big countries for 10 years and Voila! No war, no strife. But, we'd have to learn to put the toilet seat down.

OK, that's enough pro-woman stuff. I mean every bit of it, too.
-- Bill Jones

When you think that you have exhausted all possibilities, remember this: you haven't.
--- Thomas Edison
mlepage
Posts: 10
Joined: July 1st, 2021, 11:11 am

Post by mlepage »

Hi, I have a few questions regarding the catalog API at https://librivox.org/api/info

Is it possible to request the JSON details for a selection of ids? Suppose I want the info for books 1, 2, 3, 5, and 7, returned in one response?

How can I tell the current maximum book id?

When I look at the "overall list" at https://librivox.org/api/feed/audiobooks
I see it starts with book 47, then 52 to 57, then 59 then 64...
In fact, the 50 books returned have id from 47 to 141, so roughy half are absent.
Are some ids invalid or missing or obsolete/removed?

Are the books always returned in id order? If not always, which cases do?

There is a query that returns books by genre; is it possible to query the list of genres? How do I know which genres I can query by?
(Note, API info says "genre" is a query parameter but that returns an error; genre_id=9, which from Advanced Search I determined to be Erotica, returns results but just the first 50 books as above, it seems any genre_id is not working, is this a bug?)
Also, the API doesn't return the genres a book is in? (The API info says it's a field, but I haven't seen it returned.)

The API doesn't return who the book is read by? And cannot be queried by reader?

The API doesn't return the URL of the cover image?

The API doesn't return the URL of the individual tracks?

Also it would be helpful to have a field indicating whether the first chapter is a "zero chapter". (E.g. David Copperfield has 64 chapters, the initial "preface" brings the count up to 65, but it's difficult to tell if section 10 is chapter 10 or chapter 9.

Has it been considered to remove the "(version X)" from the title? Those are not actually part of the title, they should rather be in a separate field for reading_version or similar.

Some meta questions:

Am I looking at the correct API? Is the API being actively worked on? Is the source available, e.g. on github? Is the library's database schema available? Are online interfaces or downloadable versions of the database (e.g. a binary sqlite .db file) available? Are there docs explaining how it works, like when a book gets an id, when it's removed, etc.?

I suppose my most basic question is, if I programmatically want to get the best info for books, to display title/author/readBy with a cover image and a total book duration, and then get the track URLs to play, where should I be looking? The RSS (iTunes) XML feed? Is the another location?

Thanks in advance!
Last edited by mlepage on July 29th, 2021, 8:38 pm, edited 1 time in total.
annise
LibriVox Admin Team
Posts: 38542
Joined: April 3rd, 2008, 3:55 am
Location: Melbourne,Australia

Post by annise »

Perhaps you could read the thread ?
To respond to the version question - we are makers not distributers and we make the catalogue work for us , not for those wanting to use our free volunteer database to make money.

And the version as part of the title works for us (and any vaguely competent coder would have no difficulty handling it). Free advice to help our prime directive is always welcome. And our prime directive is to make all public domain books in the USA available as an audio version

Anne
Post Reply