Uploading revisited

tis · Post by **tis** » November 2nd, 2005, 8:23 pm

Could someone who has been through it walk us novices through the process of uploading files to archive.org?

What do we need to check (file format, ID3 tags, filenames?)
How and where do we upload?

If someone can describe the process in sufficient detail, we ought to be able to automate it... I know archive.org have a python tool for doing uploads, and there are python ID3 editors out there as well... and I have been known to write Python code for a living...

Post by **hugh** » November 2nd, 2005, 11:59 pm

hey chris, a python tool would be wonderbar. I have beenv busy this week but plan to put librivox techies in touch with archive techies to try to work this out. ...

in fact it's just a matter of a big FTP transfer, but we need to think (longer term) about security...

PullMeUnder · Post by **PullMeUnder** » December 19th, 2005, 3:05 pm

Continuing from my post in the technical questions forum, I'd love to get into the details of the file hosting and uploading issue. What's the progress on the automated upload tool? I'd like to help in some way, even if it's only to see my pet project (i.e. usage of speex encodes

) realized, lol. I have some experience in Python, though I wouldn't consider myself an expert. GUIs especially are somewhat alien to me, though I did a rather messy TKInter thing once (yay me). Though it'd probably be more intelligent to wait for some feedback especially from Chris, here are the question I have right now, in more detail:

What language is/will the tool be written in?
Server side or client side? From what I've read so far, you all seem pretty interested in Python/Ruby and it's installed on the server, so I guess it'd be server-side...
The original question of this post: What are the details of the upload process at archive.org? Is it at all possible to upload other file formats and link to them?
How much integration is there to be between this and the database system that's in the making/being planned/might be happening?

I guess there will be more, and I'm sure I forgot a few right now, but this should suffice for the moment.

tis · Post by **tis** » December 19th, 2005, 3:32 pm

PullMeUnder wrote:What language is/will the tool be written in?

Ruby. You're in the same boat as me, here - I'd never done any Ruby before, just some Python. But Ruby is easy to pick up (and very nice to work with.

PullMeUnder wrote:Server side or client side? From what I've read so far, you all seem pretty interested in Python/Ruby and it's installed on the server, so I guess it'd be server-side...

Server

PullMeUnder wrote:The original question of this post: What are the details of the upload process at archive.org? Is it at all possible to upload other file formats and link to them?

The default is that we upload 128kbps mp3 and they autogenerate 64kpbs mp3 and ogg vorbis. Other file format can also be uploaded - so if we can generate speex, I think it would be easy to include them. Archive.org will not stream them, or generate anything else from them, they will just hold them.

PullMeUnder wrote:How much integration is there to be between this and the database system that's in the making/being planned/might be happening?

It will be integrated to some extent, not sure of details yet.

If you want to do your pet project, then this is what we need - a mechanism to transcode an mp3 (128kpbs) into a speex. I know transcoding is bad, but I suspect a single transcode from quite a high quality mp3 will be ok. The transcode would need to run under linux (Debian).

If you can work out how to make that happen (don't worry about language - if you can do it in bash, python, C or whatever) we can integrate it.

Thanks for the offer! I'd love to see speex files. I'm off to build the darwinports version of speex on my mac...

Post by **hugh** » December 19th, 2005, 4:28 pm

that's the spirit pullme! good luck!