BitTorrenting!

Non-reading activities need your help too!
Lucy_k_p
Posts: 2901
Joined: February 16th, 2009, 7:19 am
Location: Bath, UK
Contact:

Post by Lucy_k_p »

Actually, at one point we had quite a few Chinese listeners say they couldn't access our files, presumably because Archive.org is blocked in China. If this is still the case a Torrent set-up separate from Archive might be worthwhile. (Or torrents, even if they are originally from Archive, may get around the Chinese Firewall, I have no idea.)
So little space, so much to say.
Glitch
Posts: 37
Joined: October 12th, 2010, 10:32 am
Location: San Diego, CA, USA

Post by Glitch »

I do recall that.

As I understand the technology, if Archive.org's tracker is blocked, then the torrents can still work via DHT and PEX. I don't know if their system is setup that way. Even if it is, there would have to be people seeding for DHT and PEX to work without access to the tracker. They may not be able to get to the book directly, but with a friend they can work around the block.

I'm currently researching hosting options to setup a tracker for this and other projects I'm working on. If we still want to do something for Librivox, the tools will be there.

-Matt
mtiernan
Posts: 4
Joined: March 18th, 2014, 7:24 am

Post by mtiernan »

I'm a new user poking around and I wanted to offer a suggestion (worth what's paid for it).

If someone could generate a file of torrents to be circulated, some of us (I have a dedicated bittorrent system) could provide a seed for them.

(It'll give me a chance to read more things too.)

Maybe do something like update the torrents on a periodic basis?
annise
LibriVox Admin Team
Posts: 38542
Joined: April 3rd, 2008, 3:55 am
Location: Melbourne,Australia

Post by annise »

I know little about torrents except in theory, but Archive has been generating them with our files for a year or so so they are availanle on our Archive pages - which are linked from our catalogue pages

Anne
mtiernan
Posts: 4
Joined: March 18th, 2014, 7:24 am

Post by mtiernan »

annise wrote:but Archive has been generating them with our files
Anne
What I'd like to get now is a list of those torrents so that I can seed a bunch. I have CPU cycles to spare and am willing to help make the stuff available.
annise
LibriVox Admin Team
Posts: 38542
Joined: April 3rd, 2008, 3:55 am
Location: Melbourne,Australia

Post by annise »

You could experiment with search options at archive - we do not have the information about them - or as they are named with the same base as other LibriVox files , maybe you could use that somehow - or you could ask in an Archive forum perhaps but do chek their FAQ's first

Anne
philchenevert
LibriVox Admin Team
Posts: 24587
Joined: October 17th, 2010, 9:23 pm
Location: Basking by the Bayou
Contact:

Post by philchenevert »

I have been seeding a few books for about 9 months just to be sure they are available. Torrenting our recordings does not seem to have nearly as much traffic as catalog downloads but people still obtain them via torrents every day.

I am in favor of making our catalog available in as many forms as possible because people have different needs and resources.
So thanks for the offer to put some or all of them on your server. :clap:
Peruvian owls always hunt in pairs because they are inca hoots.
89 Decibels? Easy Peasy ! https://youtu.be/aSKR55RDVpk
dlolso21
LibriVox Admin Team
Posts: 4827
Joined: January 11th, 2011, 12:13 pm

Post by dlolso21 »

mtiernan wrote: What I'd like to get now is a list of those torrents so that I can seed a bunch. I have CPU cycles to spare and am willing to help make the stuff available.
This Advanced Search on Internet Archive will give you a list of the first 8000 "Librivox Identifiers" (we haven't hit 8,000 yet) in JSON format and sorted in date order:

http://archive.org/advancedsearch.php?q=%28collection%3Alibrivoxaudio+OR+mediatype%3Alibrivoxaudio%29+AND+-mediatype%3Acollection&fl[]=identifier&sort[]=addeddate+asc&sort[]=&sort[]=&rows=8000&page=1&output=json&callback=callback

A short sample of the ouptut => {"identifier":"warble_lilac_time_0804_librivox"},{"identifier":"oldfashioned_girl_0903_librivox"},

You could create a list Torrent list from this because of the consistent naming format (see the bold text below for the two torrents that match the sample librivox identifiers above)

https://archive.org/download/warble_lilac_time_0804_librivox/warble_lilac_time_0804_librivox_archive.torrent
https://archive.org/download/oldfashioned_girl_0903_librivox/oldfashioned_girl_0903_librivox_archive.torrent

David O
Glitch
Posts: 37
Joined: October 12th, 2010, 10:32 am
Location: San Diego, CA, USA

Post by Glitch »

I can't express how proud I am that the work that we put into this years ago has resulting in a lasting system that makes Librivox books available all over the world, possibly even in places where they are considered banned or censored. If I haven't done so yet (and I know I haven't done so recently) I want to thank you all for the effort you put into this.

Worth mentioning is an apology for resurrecting a dead thread, but considering I was the OP (long story)...I feel I have the right to use at least one Phoenix Down, eh? In any case, seeing how the torrent technology has settled in I think it's time to start looking at more features and community services. Over the years I have seen a small handful of very dedicated individuals that have shown interest in expanding our BitTorrenting capabilities, and that's where my interests lie as well, so here is my vision of what comes next, and please...let's hear some conversation!

Now that it is both possible and easy to torrent individual books, it is time to start distributing collections.

On the grand scale, downloading/seeding the entire librivox collection is an achievement that warrants celebration, but also greatly enhances the liquidity of the library and offers some insights into the 'big data' of usage.

On smaller scales, being able to easily download all books from a specific author, or of a series, or of a genre/language/time period has it's obvious advantages.

All of this is quite easy to manage, once established, through bittorrent technology. Another great advantage of this project is how easy it is to coordinate, collaborate, and effectively use each other's time to accomplish great things.

The way I see this working is to create a 'Torrent of torrents'. Imagine, if you will, a single torrent link that, once activated, downloads every torrent file associated with the Librivox collection (or an author/series/genre as above). Once downloaded (a relatively small amount of data), it is simple to import whichever of those torrent files into whatever client you prefer and begin downloading the actual books. As it is a torrent of official torrents, anything you already have in your collection is skipped by the majority of mainstream clients. When a specific 'Torrent of Torrents' is outdated (because another book is added, or something had to be corrected), everyone easily transitions to the new swarm and the old one slowly dies out. Again, no wasted data transfer exists because any individual files that are already transmitted are simply accounted for.

I would like to hear your thoughts and opinions!

-M

p.s. I am in the process of manually downloading each individual torrent. If anyone has access to a larger set or even the entire set of torrent files, I would LOVE to save a few dozen hours of clicking.
Interested in downloading the entire collection? PM me for details! (Bittorrent project)
Glitch
Posts: 37
Joined: October 12th, 2010, 10:32 am
Location: San Diego, CA, USA

Post by Glitch »

I've been able to get my hands on all of the torrent files, but I'm running into the issue where the client that I am using doesn't like to work with more than a few hundred active torrents. This has been the case on my low level and higher level PCs, I'm beginning to suspect that it's a limitation of the client software and not a hardware bottle-neck.

Any suggestions out there for a high-volume torrent client?
Interested in downloading the entire collection? PM me for details! (Bittorrent project)
taosubmarines
Posts: 1
Joined: April 2nd, 2018, 4:38 am

Post by taosubmarines »

Hi Glitch
I am new here, cant pm you :(
I want a complete collection of all the Librivox torrents on one page, for easy scraping, wgetting, downloading and seeding.

"
I much appreciate Librivox volunteers work, and initiatives to rebuild the public domain and commons

I would like to see a single web-page that enables me to download ALL the librivox torrents and hash (security) signatures.

Then I would like to put them in a new, auditable, BLOCKCHAIN, that you can listen to, that way in every village in africa/asia/americas is a potential seed of the blockchain...

as per the 'public domain', and legal definitions thereof, as defined by nation states...
"
annise
LibriVox Admin Team
Posts: 38542
Joined: April 3rd, 2008, 3:55 am
Location: Melbourne,Australia

Post by annise »

Well to start off with - I'm no torrent expert, but our files are all PD in the USA - we don't make any claims for their status anywhere else. So we would not be approving of anything that claimed world wide copyright free status for our files - and as rules differ everywhere , it would not be easy for anyone to do so . Even the Giants like Ggoogle block downloads of books that it is remotely possible are still in copyright anywhere - and very frustrating it is sometimes too

However I will PM Glich and point him to your post :D

Anne
Post Reply