LibriVox API Discussion Thread

Comments about LibriVox? Suggestions to improve things? News?
Post Reply
Kuranes
Posts: 3
Joined: November 5th, 2012, 12:41 pm

Post by Kuranes » September 16th, 2013, 1:36 pm

Hi, I'm trying to use the new API to effectively consume the whole catalog - I'm using this request:

http://librivox.org/api/feed/audiobooks/?offset=0&limit=20000&fields={id,title,authors,language,genres,url_rss}

This works, but the response is lacking genre information. A partial response is here:

<book>
<id>47</id>
<title>Count of Monte Cristo</title>
<language>English</language>
<url_rss>http://librivox.org/rss/47</url_rss>
<authors>
<author>
<id>431</id>
<first_name>Alexandre</first_name>
<last_name>Dumas</last_name>
<dob>1802</dob>
<dod>1870</dod>
</author>
</authors>
</book>
<book>
<id>52</id>
<title>Letters of Two Brides</title>
<language>English</language>
<url_rss>http://librivox.org/rss/52</url_rss>
<authors>
<author>
<id>86</id>
<first_name>Honoré de</first_name>
<last_name>Balzac</last_name>
<dob>1799</dob>
<dod>1850</dod>
</author>
</authors>
</book>

Am I doing something wrong?

Thanks,

Andrew

dcsohl
Posts: 22
Joined: March 11th, 2013, 7:46 pm

Post by dcsohl » September 16th, 2013, 6:44 pm

Hi,

The "audiotracks" listing seems to be missing links to the actual audiotracks. For example, http://librivox.org/api/feed/audiotracks/?project_id=685 shows me:

Code: Select all

<xml>
  <sections>
    <section>
      <id>131328</id>
      <section_number>1</section_number>
      <title>Stave 1</title>
      <listen_url/>
      <language>English</language>
      <playtime>2315</playtime>
      <file_name/>
    </section>
 ...
</xml>
I would expect either "listen_url" or "file_name" to have a value. I'd like to use the API to programatically retrieve the audio files from archive.org...

Also, playtime is in seconds, I assume? (It's not 23 minutes 15 seconds in my example above, it's 2315 seconds or 38 minutes 35 seconds, right?)

Also also ... technically, an XML tag named "xml" is in violation of the XML spec (no element or attribute name is allowed to start with 'xml'). I doubt very much that anything will actually break as a result, but I thought I'd bring that to your attention.

duke
Posts: 9
Joined: September 9th, 2013, 4:42 pm
Contact:

Post by duke » September 17th, 2013, 8:33 am

I'm having a really hard time getting the new API to work (especially compared to the old API). Perhaps more examples on the API page (http://librivox.org/api/info) might help.

Problems:
Search by Genre returns a database error:
...
<p>Error Number: 1064</p><p>You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')
LIMIT 50' at line 4</p><p>SELECT p.*, l.language
FROM (`projects` p)
JOIN `languages` l ON `p`.`language_id`=`l`.`id`
WHERE `p`.`id` IN ()
LIMIT 50</p><p>Filename: /librivox/www/catalog.librivox.org/librivox/application/libraries/Librivox_API.php</p><p>Line Number: 122</p> </div>
...

Did the genres change with the new API? Can I see an example?

Search by title:
Can you only search from the beginning?
For example, trying to find "Pride and Prejudice"...
The only thing that seems to work is ".../feed/audiobooks/title/^Pride" you can't search for "Prejudice".
It seems like you have to provide the anchor caret ^ at the beginning to get anything.

When search results are empty it returns a 404. That's sorta weird.

There are other problems, but if I could get some info on these, I'd be a happy camper.

-Duke
Duke Browning -- Freshmowed Software -- Bookmobile Audiobook and Podcast Player

etienne
Posts: 1
Joined: September 16th, 2013, 2:12 pm

Post by etienne » September 17th, 2013, 10:24 am

duke wrote:I'm having a really hard time getting the new API to work (especially compared to the old API). Perhaps more examples on the API page (http://librivox.org/api/info) might help.

Problems:
Search by Genre returns a database error:
...
<p>Error Number: 1064</p><p>You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')
LIMIT 50' at line 4</p><p>SELECT p.*, l.language
FROM (`projects` p)
JOIN `languages` l ON `p`.`language_id`=`l`.`id`
WHERE `p`.`id` IN ()
LIMIT 50</p><p>Filename: /librivox/www/catalog.librivox.org/librivox/application/libraries/Librivox_API.php</p><p>Line Number: 122</p> </div>
...

Did the genres change with the new API? Can I see an example?

Search by title:
Can you only search from the beginning?
For example, trying to find "Pride and Prejudice"...
The only thing that seems to work is ".../feed/audiobooks/title/^Pride" you can't search for "Prejudice".
It seems like you have to provide the anchor caret ^ at the beginning to get anything.

When search results are empty it returns a 404. That's sorta weird.

There are other problems, but if I could get some info on these, I'd be a happy camper.

-Duke
I would be interested in more examples and the answer to the above questions also. Plus, is there a way to search the whole catalog for a search term the way it was possible in the old API with

Code: Select all

http://librivox.org/newcatalog/search_xml.php?extended=1&simple=searchterm
Thanks!

tbook
Posts: 77
Joined: May 12th, 2012, 7:01 am

Post by tbook » September 18th, 2013, 2:53 pm

It would be great to know if the genres field is coming back. I haven't been able to import new books since the old API went down, and am working on re-building my import code to match the new API. I would hate to drop support for genres unnecessarily. Thanks!
Working on iOS and Android apps for LibriVox. You can see the comments from the apps on our web site: LibriVox Audio Books

Cori
LibriVox Admin Team
Posts: 12029
Joined: November 22nd, 2005, 10:22 am
Location: Great Britain
Contact:

Post by Cori » September 18th, 2013, 2:55 pm

I sincerely hope it's coming back. In general, that is. The genres were updated with the new site (although should have been visible in the API a while before that.)
There's honestly no such thing as a stupid question -- but I'm afraid I can't rule out giving a stupid answer : : To Posterity and Beyond!

dcsohl
Posts: 22
Joined: March 11th, 2013, 7:46 pm

Post by dcsohl » September 19th, 2013, 7:52 am

No answer to my questions? Ok, let me restate: How do you get the actual location of the audio files from the API? Going back to my example, if you look at http://librivox.org/api/feed/audiotracks/?project_id=685 there's no way to know where exactly the files are located. How would this be done?

Thanks,
-=-Dan

tbook
Posts: 77
Joined: May 12th, 2012, 7:01 am

Post by tbook » September 19th, 2013, 12:48 pm

Dan: The way I get them is to use the RSS feeds. I don't know if there will be a way to get them through the new API. Of course, Archive.org also has its own API, so you can gather info from there without using the Librivox API at all.
Working on iOS and Android apps for LibriVox. You can see the comments from the apps on our web site: LibriVox Audio Books

ekzemplaro
Posts: 2030
Joined: December 31st, 2011, 7:17 am
Location: Tochigi,Japan
Contact:

Post by ekzemplaro » September 19th, 2013, 2:03 pm

Hello Dan san,

Why don't you use audiobooks API instead of audiotracks API?
dcsohl wrote: if you look at http://librivox.org/api/feed/audiotracks/?project_id=685 there's no way to know where exactly the files are located. How would this be done?
In this case,
http://librivox.org/api/feed/audiobooks/?project_id=685

As tbook san mentions, you can also use archive API.
curl 'http://archive.org/details/letters_brides_0709_librivox&output=json'

Cheers,
Masa

dalewking
Posts: 11
Joined: September 1st, 2010, 6:03 am

Post by dalewking » September 22nd, 2013, 8:38 pm

Having trouble figuring out how to use the new API to give me new books since the last time I checked. With the old API you had to page backwards until you got to a book you had already seen. It was a pain, but worked.

I figured that the books were in order by when they were completed, but that does not seem to be the case. I assumed if you had already seen x number of books you could just use an offset of x and you would get books you had not seen. However the end of the list if you use offset=7650 bears no resemblance to the most recent books added and instead has a bunch of foreign language books.

It would seem that using since would be the way to go. One problem with that is that the records returned do not actually have timestamps in the returned data. I have also found discrepancies with the results for the 4 most recent books currently.

The most recent books at the moment were:

* Bull Hunter
* The Devil's Bridge
* Ruth Fielding at Briarwood Hall
* The Getting of Wisdom (Version 2)

Trying some since queries I get some inconsistent results:

* since=1379803278 gives me the first 2 books
* since=1379703278 gives me the first 3 books
* since=1379601278 gives me the first 2 books and the last book, but not the third book

After running those queries apparently Tom Swift Among the Diamond Makers got added as the most recent book, rerunning the queries with this new book added gives me:

* since=1379803278 gives me just Tom Swift
* since=1379703278 gives me Tom Swift and the first 2 books
* since=1379601278 gives me Tom Swift and the first 2 books

Something is definitely not working with the since query.

I really need a way to just query for new books that I don't already have in my copy of the data.

dcsohl
Posts: 22
Joined: March 11th, 2013, 7:46 pm

Post by dcsohl » September 23rd, 2013, 8:16 am

The "audiobooks" API gives information about the books and links to ZIP files of the entire contents of the book. What I was looking for was individual URLs to each file segment so they can be downloaded piecemeal instead of all at once.

But the RSS feeds do provide such links, so thank you very much, tbook.

bart
LibriVox Admin Team
Posts: 7515
Joined: February 16th, 2009, 9:17 am
Location: Utrecht, the Netherlands
Contact:

Post by bart » September 23rd, 2013, 11:41 am

Thanks everyone for the responses.
I am currently summarizing all the requests in this thread for our developers. They will work through this as soon as possible.
I already know we cannot realise everything, but as soon as we know more we will communicate about it.

Bart
Alle Nederlandstalige projecten op de Librivox Boekenplank
Lees mee met Alles Zingt

tbook
Posts: 77
Joined: May 12th, 2012, 7:01 am

Post by tbook » September 23rd, 2013, 1:23 pm

Thanks, Bart! Please keep us posted. Since the old API went down, a lot of us have been holding our breath waiting for something we can use.
Working on iOS and Android apps for LibriVox. You can see the comments from the apps on our web site: LibriVox Audio Books

tbook
Posts: 77
Joined: May 12th, 2012, 7:01 am

Post by tbook » October 7th, 2013, 12:57 pm

Hi Bart,

I thought I'd ping you and see if you had a sense of what additional changes might be coming down the pike (particularly, whether we will have access to genre data and a sortby=recent field.) I'm trying to figure out if I should work with the API as it is, or hold off for some more changes. Thanks!
Working on iOS and Android apps for LibriVox. You can see the comments from the apps on our web site: LibriVox Audio Books

bart
LibriVox Admin Team
Posts: 7515
Joined: February 16th, 2009, 9:17 am
Location: Utrecht, the Netherlands
Contact:

Post by bart » October 7th, 2013, 1:39 pm

I think the API will be changed, but I don't know when. The developers asked me to come up with a wish list, so they have plans to extend the API.
But there is an awful lot to do and I don't know how high we are on the priority list.
I myself decided not to wait. I developed my program with the API as it is now. I could do everything by combining the API, the new releases feed and the project pages, with a database I made myself.
You have to decide yourself what you are going to do, especially because I cannot tell you what the changes in the API will be.

Bart
Alle Nederlandstalige projecten op de Librivox Boekenplank
Lees mee met Alles Zingt

Post Reply