Question Marks in the Place of Foreign Characters

Comments about LibriVox? Suggestions to improve things? News?
HerrSchildkroete
Posts: 130
Joined: January 11th, 2006, 1:38 pm
Location: Aachen, Germany
Contact:

Post by HerrSchildkroete » September 8th, 2006, 12:24 am

harvey wrote: The Web server now declares all its Web pages to be in iso-8859-1
(character encoding), which was changed from utf-8 in the past few days.
What is the reason for that change?

My guess is that the characters are stored as UTF-8 in the database. However, since the page is now declared to be ISO-8859-1, they are misinterpreted.

By the way: the webserver itself does not provide any encoding information in the HTTP header. The encoding is only declared in the html code.
Jabber ID: smurflord@jabber.org

a.r.dobbs
Posts: 3215
Joined: February 23rd, 2006, 1:04 am
Location: Boston

Post by a.r.dobbs » September 8th, 2006, 11:22 am

Actually... this morning, view source:

Code: Select all

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html dir="ltr">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta http-equiv="Content-Style-Type" content="text/css">
Yeah, about the http header thing, that was totally a grope in the murk (the murk in my mind that passes for "sort of following what they're talking about" in the W3C document I quoted from).
Anita

a.r.dobbs
Posts: 3215
Joined: February 23rd, 2006, 1:04 am
Location: Boston

Post by a.r.dobbs » September 10th, 2006, 12:35 pm

Hi Folks,

DigiSage gave this bit of background (we've been fretting in admin, you see) ... left over bad code is being cleaned up by hand, bit by bit. If you see something that still needs help, please note it here. Thanks!
DigiSage wrote:Why did this happen?

It's a long story. Here it is.

There are a lot of other sites on this same server. The server has 8gbit/s of bandwidth and plenty of CPU/RAM to go around, so this shared hosting environment usually works just fine. I've had several sites "slashdotted", as well as linked to by cnn.com, news.com, other big sites, never had any problems.

Recently, one site sent out a mass email ... This caused an extremely high increase in traffic to their web site. Much, much more than anything I've ever gotten from slashdot or anything else.

Anyway, this traffic totally overloaded the server. It prevented basically every site from working... I had to do something to fix the issue. As a temporary fix, I shut down that site, which helped a bit, but didn't totally "fix" the issue because there was still a ton of traffic, people were just being served a "this site is disabled" message instead of their actual site.

While the server was a little less broken, thanks to taking their site down, I looked into what my options were. We were running apache v1.3 at the time. I knew that Apache2.0 was much, much more efficient. We hadn't upgraded to Apache2.0 before because we didn't need the better performance, and the potential for causing problems with php sites (among other things) was high.

Well, at this point in time, I didn't really have a choice. So I upgraded to Apache2, which required upgrading php, all of php's modules, and I also upgraded MySQL while this was going on, also for performance increase reasons.

Once I brought the new Apache2 up, the whole server went back to normal, as far as performance goes. In fact I'd say that it's faster than it was before. But anyway, somewhere along all these upgrades, the character encoding issues appeared. It's an extremely common problem, especially when moving from apache1 to apache2.

So, this server software upgrade is what broke the Librivox forums, and for that you have my most sincere apologies. Unfortunately this sort of thing happens on shared hosting servers. As much as web hosts try to avoid it, things like this happen. The only sure-fire way of avoiding it is to get your own dedicated server, which costs a lot of money.

I've made changes to the forum software, Apache2.0, php and MySQL to ensure that ALL new data entered into MySQL from this forum or anywhere else will guaranteed be stored as UTF-8. And I've forced the forum software to always display in UTF-8. So this problem shouldn't happen again.
Anita

Xander
Posts: 200
Joined: May 27th, 2006, 5:29 pm
Location: Franklin, Kentucky
Contact:

Post by Xander » September 10th, 2006, 1:18 pm

WOW!! I didn't know that a simple little problem I had could expand into something so massive and technical. I thought it was just some configuration error on my part.
-Xander

"Quid quid latine dictum sit, altum videtur."

www.zamws.com

a.r.dobbs
Posts: 3215
Joined: February 23rd, 2006, 1:04 am
Location: Boston

Post by a.r.dobbs » September 11th, 2006, 1:21 pm

Xander wrote:WOW!! I didn't know that a simple little problem I had could expand into something so massive and technical. I thought it was just some configuration error on my part.
Eh?
That's a little turned around, I b'lieve. It's more that...
what you perceived as a little problem with your configuration
was actually a single symptom of a software upgrade ... that was precipitated through a domino effect by an event in a remote corner of the galaxy.

But . . . maybe it's more fun to say it's all a result of Xander doing some goof-up on his computer configuration. :twisted:

Edit: Oh wait! Now I get it!! You were saying:
"Here I thought I was asking a simple question. I thought I'd get a simple answer, like, 'Click this box . . .' But noooooo! I get a saga and a half. Yow."
Gotcha. :wink:
Anita

kristin
Posts: 4595
Joined: June 1st, 2006, 10:47 am
Location: Des Moines

Post by kristin » September 11th, 2006, 11:06 pm

a.r.dobbs wrote: If you see something that still needs help, please note it here. Thanks!
I was just looking at the first post of Pirates and noticed some of the ' and " and - are now ?.

kayray
LibriVox Admin Team
Posts: 11867
Joined: September 26th, 2005, 9:10 am
Location: Union City, California
Contact:

Post by kayray » September 12th, 2006, 8:07 am

kristin wrote:
a.r.dobbs wrote: If you see something that still needs help, please note it here. Thanks!
I was just looking at the first post of Pirates and noticed some of the ' and " and - are now ?.
Fixed! :)
Kara
http://kayray.org/
--------
"Mary wished to say something very sensible into her Zoom H2 Handy Recorder, but knew not how." -- Jane Austen (& Kara)

kristin
Posts: 4595
Joined: June 1st, 2006, 10:47 am
Location: Des Moines

Post by kristin » September 12th, 2006, 12:11 pm

Thanks Kara. :D

Post Reply