Adventures in Bug Hunting: Or, PHP 5.3, why can’t you handle whitespace?

Caution: NERD TALK AHEAD

Soldier in Starship Troopers shoots a bug
Bug Hunting isn’t this exciting

A couple years ago, I created a request system for my department which would allow users to register their teaching preferences.  This wasn’t really anything new, many other people have done it (I did it for my department at University of Florida years ago).  In the years since, I’ve tweaked it and added features, but it’s been essentially unchanged.

On Tuesday, our department’s primary user of the system pointed out a bug in the way it handled new course requests.  I went in and fixed the bug that afternoon. No problem.

But the next day, she told me it was not working right.  I thought maybe it was a similar minor error.  Nope — it was downright broken.

Here’s the problem I was getting:

When the user tried to download a spreadsheet of requests, the system spit out an error indicating that the header (which tells your computer that a file is on its way) had already been sent.

This was very puzzling, as I had not changed anything having to do with headers.  Nonetheless, I started tracking the error back by forcing it, moving the code that replicated the error up the program chunk by chunk until I figured out that the initial headers were being set very early in the program.

In fact, they were being set when the program used “include” to grab a file of subroutines stored externally.  Now I was really mystified — this shouldn’t yield headers at all.  So finally I found where the error was being generated, but I had NO idea why.

Then I remembered: yesterday I got a message from my web host that the PHP on my servers needed to be upgraded to 5.3.  “Sure thing,” I said as I clicked the upgrade button.  Dammit.

To Google!

Finally I discover that the error code I was getting showed where the mystery headers were being generated.  It was at the end of the file.  After the ?> which closes the program.

That’s right, upgrading PHP meant it was now unable to ignore two blank lines at the end of the file.  Bang head on desk for five minutes.  Delete two blanks lines.  Revel in properly working program.

“C’mon you apes, you want to live forever?!”

 

How to view two Excel Spreadsheets in two different windows

But here’s how you do it in Windows 8:

1. Open Excel.

2. Switch to your app screen or whatever they call it

3. Right click on the Excel app

4. Select “open new window” in the option bar at the bottom.

This will open Excel as a separate window which you can then treat like any multi-window program.

This is the lamest tech help post I’ve had to make thus far, but Excel 2010 has a really dumb “feature,” namely that all the windows are conserved within one window.  In other words, if you have two excel spreadsheets open and you switch from one to the other, the spreadsheet showing in the window changes.  This means you can’t see two spreadsheets at once.  How annoying.

Excel has a built-in feature in the “view” window that allows you to look at spreadsheets side by side, but this is clunky to use and frankly, even with help, I couldn’t get it to look the way I wanted.  I just wanted separate windows, dammit.

It turns out the only way to get these separate windows is to open two instances of Excel.  Lame.  Hey MS Office programmers: please make a keyboard command to do this.  May I suggest CTRL-ALT-N?

Server maintenance

It looks like Dreamhost will be moving my site to a new server in their massive server facility on the moon this Thursday, 18 July.  We’re warned that there may be some slow-time or down-time as the move happens.

You’ve been warned.

Well that was unexpected

Check out the bottom left option in this “related to I Sell the Dead” screen.  Weird, eh?

Once More with Feeling
Once More with Feeling

I suspect all Chaz Palmenteri or Drea de Matteo films end up in the zombie list, just because.

Hack Update

[youtube:http://www.youtube.com/watch?v=Ie-Wvqw5zYM]

In the continuing battle against some breech in my security, my SF book club site became malware-laden again.  I’ve zipped and removed it completely at present so as to minimize damage to my other sites.  My family photo site is in a similar state of disrepair.

So far no more notices about this blog, but if it becomes infected again then the presumption will have to be that the problem is in the database itself, a scary prospect as I have no idea how to clean the database.

Will keep you posted.  In the meantime, if you encounter the scary “don’t got there” warning, please follow its advice.  After several infections that redirect Explorer users, the malware that showed up on my book club website infected my computer with a virus-scan scam that took some serious effort to get rid of.  AARRRHGGHGGHHH!

That said, look for a nice Downton Abbey post this afternoon, if I’m still here.

Hacked and Fixed, AGAIN

"I Miss You, Computer" by Neofob
"I Miss You, Computer" by Neofob

A couple days ago, I got notice from Google (and many fine friends who pointed out my blacklisted status), that my blogs had been hacked.  As has happened before, the software infects all of my blogs at once, so I had a lot of cleaning to do.

Thanks to John for his helpful post.  This is not the first time my blog has been hacked, but this is by far the most thorough I’ve been in trying to stamp it out.  Here are the steps I took yesterday, in no particular order:

  • All related passwords changed (ftp, user, database)
  • WordPress software freshly installed
  • Plugins scoured for offending code
  • All dormant or unused websites shut down, removed, and databases backed up then deleted.  I lamented taking down some of my student work, but since I don’t really look at it afterward, it’s causing me negative time.  GONE.
  • Using the shell, I did plenty of grepping and finding to locate ‘open to the world’ directories and a few errant infected files.  While the latter were bad, I suspect it’s the former that allowed my sites to get infected in the first place.  I also discovered several seemingly viable files (with names like ‘https.php’ that were evil, and a couple directories that had been chmodded so I was not allowed to read or write in them.  Inside? evil files.

The only opening I see is if my database itself has offending code in it.  I could not find any pages that discussed either how to diagnose or to fix corrupt databases themselves, so I’ve backed up the newly clean sites entirely, and will do a complete reinstall if I get infected again — and will have to figure out how to figure out about corrupted mysql files.

Ugh.  Oh well, I’ve requested a review from Google, so hopefully you’ll find my site un-blacklisted sometime soon.  Enjoy.

Copy and Paste Weirdness

I had a strange development Monday.  I had changed my FTP password a few days ago (part of my ongoing fight against the malware that keeps infecting the site) and was updating it in my Filezilla client and it wasn’t working.

So Tuesday morning I changed it again, figuring I must have been using the wrong password or made an error somewhere.  When I tried to update it (by copying and pasting from my email) it failed AGAIN.  So then I tried logging in via puTTy and that didn’t work either.

So I tried typing the password by hand instead of copying it, and it worked.  I tried the same in Filezilla and it worked.  What the heck?  Does anyone know about any weirdness in copy-and-paste in Thunderbird that would cause this error?

Adventures in reverse-image search

Watching A Miracle on 34th Street last week, I noticed this weird photo framed on Judge Harper’s office wall.  I wondered what it is and whether it’s just some random picture the prop department hung up, or if it’s famous in some way.  So I thought I’d try to find it on the web.  First, here’s the screen-cap of the image in the film:

And to the right here is the crop of the photo that I used in my image search.

First, I tried Google’s TinEye, which searches the web for exact images. It’s looking not at graphical similarity, but at similarity in code.  You can use it to find places where an image has been directly copied and reposted without being re-formatted or re-edited.  Not surprisingly, as I created the image myself with a screen cap, it’s not already on the web.  Oh well.

Next, I tried using words to describe the image to see if I could find it using keywords.  Here are unsuccessful searches I tried:

  • photo rocket launch gun “miracle on 34th street”
  • photo gun rocket world war 2
  • photo gun emplacement world war 2
  • famous photo rocket gun photo world war 2
  • missile rocket launcher world war 2

Next, after a search online for more upload services, I found BYO Image Search Lab.  This one found images with similar color palettes and shapes, but alas, not the image I’m looking at.

So, my tech-savvy friends: where would you go next?

 

 

Hacked!

Alas, my website was hacked.  This isn’t the first time I’ve found my websites monkeyed around with, but it is the most annoying.  The main problem is that I like to save stuff — I do big projects with students using CMS systems, and then I don’t delete them because I like the work they did.  But in doing so, I leave myself open to trouble because out-dated CMSes are like broken windows in a neighborhood, they attract all the wrong people.

So I’m closing down many of my venerable subdomains which have mouldered in the–eep–years since I’ve used them.   I’m backing up everything first, of course.  And then — off we go!  Just for your amusement, here’s a step-by-step of what happened and how I’m fixing it.

The Hack

In one of my oldy moldy CMSes, some script breached security and spurted ugly code all over my subdomains.  It would have gone un-discovered if it had been smart enough to see that my only domain with much traffic was curragh-labs.org and stayed away from that.  Thankfully, it was dumb enough to do it there.

The script created an .htaccess file that redirected users coming from a search engine to some random page.  If you went to my URL directly, it ignored you.  Thus, many of those it would capture are the folks not coming directly here on purpose.  A clever hack.  Fortunately, I have a couple folks who get to my site not using bookmarks or memorized URLs, but by searching Google.  They emailed me and I found the .htaccess file, as well as a couple sneaky php files stinking up my home directory.

Triage

  • First, I deleted the .htaccess file and the other PHP files sitting in my home directory.
  • Next, I glanced through a few subdirectories to see if any other sneaky files were there — no, so that’s helpful.
  • Then I changed my SFTP password and my Dreamhost Web Panel password and contacted my web hosts.  They suggested that I update all my outdated CMS software and stop being such a security hole.  Their form letter is nice about it, but I feel chastened nonetheless.

Cleaning House

  • My new rule is that if I’m not committed enough to a subdomain or site to visit at least once a month to update the CMS, I can’t have a CMS there.
  • So first, I need to close down the subdomains I don’t care enough about to leave up on the web.  This is simple enough: back up, unregister on Dreamhost, delete files.
    • 1924.curragh-labs.org – year-based class writing project, gone
    • 1945.curragh-labs.org – another iteration of same, gone
    • 1969.curragh-labs.org – first iteration of same, gone
    • book.curragh-labs.org – an old subdomain with nothing on it, built for a project that ended up elsewhere
    • tracker.curragh-labs.org – an old subdomain with nothing on it, built for a project that didn’t launch
  • Then, I cleaned up my home directory on my web host account, backing up and deleting files I don’t use anymore.

Repairing the Damage

  • Next, I needed to round up the domains I don’t use anymore and stabilize them.
    • The ARG domains I bought for last year’s ARG class are all decomissioned, except for CNB labs, because that’s just cool.  I plan to move that info onto arg.curragh-labs.org when I’ve got some, ahem, free time.  So in cleaning up my hosting panel, I zip archived those and put them into backup.  Then I deleted them from my hosting list.  Then, I updated ARG.curragh-labs.org to current MediaWiki software.
    • I also went into my pile of MySQL databases and deleted the ones I’m not using anymore.
  • Then I consulted the “hacked files” list Dreamhost sent me.  Most of the files were taken care of in the house-clearing, but a few required updating on sites I want to keep live.
    • I discovered a few corrupted files in one of the unused themes on my blog.  Since I only use one theme and these others are potential security breaches, I closed that door by deleting all the themes I don’t use.
    • Dreamhost also alerted me to a few hacked files on other blogs I have, so I’ve scoured those out too.
    • One file belongs to a user who doesn’t hang around anymore, so I’ve had to request that the admins delete it.
    • I found a couple directories that might have been old projects I don’t remember, or might have been hacker work.  Deleted.
    • Next, I pulled the old Drupal install off BadMovieClub and installed WordPress instead.
    • Last, I updated the two remaining trouble sites, bringing projectloop up to safe Mediawiki code, and CNB labs up to current WP standards.

Ugh.  There went a morning.  But it was overdue.  On to more productive places!

xemacs cannot open load file: ezimage

Oh yeah, it’s time for another kick-ass post about a nitty-gritty technical problem that will only ever interest some obscure user who never reads this blog.  Remember last time?  Yeah you do.  Enjoy this new bit of nerdistry, my regular peeps.

In using xemacs 21.4 for a php programming project recently, I tried to invoke php-mode using the command

M-x php-mode

I got this error:

cannot open load file: ezimage

Oh noes!  After more searching than was reasonable, I found a forum with this advice:

it needs to have cedet-common instal if you get the error launching xemacs

So here’s how I solved the problem.

  1. Go to Tools -> Packages -> List and Install
  2. Select ‘cedet-common’ by pressing ‘enter’
  3. Press ‘x’ to add the package

Then php-mode worked!

 

Summary of our Wi-Fi woes, and what I did to “fix” them, sort of

"Wires" by expense
"Wires" by expense

Our wi-fi in the house has been getting steadily worse over the last few months, but I didn’t really notice how bad it had gotten until mid-December, when our cable was downgraded (long story) so we decided to upgrade it.  Here’s how it played out:

  1. The Comcast service dude came to upgrade the cable by putting in a cable box on one TV and updating the cable card on our main T.V.  when he left, it appeared that the cable box was working, but in fact, neither were showing us many of the channels we were supposed to get.
  2. After one long call to customer service (and another two shorter calls in the following days), the cable guy RETURNED four days later and did another round of troubleshooting before they determined that it was a problem with the upstream signal somewhere.
  3. Along the line, our wireless internet stopped working pretty much entirely.  I’m still thinking this is coincidence of timing rather than a network issue, but who knows? After doing some simple diagnostics, I decide our old router is shot and I buy a new Linksys router.
  4. When I install the router, I find that the wireless sucks.  The SIGNAL is strong and steady, maintaining log-in to the router at 60 or 70 mbps.  But the bandwidth, either browsing or using a speed test, is coming in somewhere around 1 to 1.5 mbps.  Ugly slow.  When I plug into the router with an ethernet cable, though, I get 20mbps, very consistently.  Also, if I plug into the modem directly, 20mbps.  So I decide it must be something in the wireless signal.
  5. After two long tech support calls with Linksys, they decide it is a bad router and arrange for an exchange by mail.  Only after reading the fine print in the confirmation email do I realize that they expect ME to PAY to mail them the broken router (which is about a week old at this point).  I call to complain about that and, in the end, they tell me to return it to Best Buy.  I call Comcast to confirm that this couldn’t be a problem with their network somehow.  They say no.
  6. Best Buy takes it back cheerfully and exchanges it for another.
  7. When I get the new Linksys router home (an E2000 if you care), it has the SAME PROBLEM.  Argh.  So I call Comcast again and explain the problem.  I get put on hold and then hung up on.  Wondering if it’s somehow a problem with the Linksys hardware interacting with my other hardware, I return it again to get a different brand.
  8. Best Buy cheerfully takes it back and exchanges it for a different brand (a NetGear dual band router this time).
  9. I get the new Netgear router home and now the problem has become more complex.  The Netgear router has two bands, the old standard 2.4Ghz and the new 5.0Ghz.  For reasons I don’t understand yet, my laptop can see the 5.0 Ghz network and works at blazin’ fast speeds (18mbps).  The other two wireless computers in the house, one of which is only a month old and not at all a cheapo) can only see the 2.4Ghz network and they have the same problem again — 20mbps ethernet, 1.5 wireless.
  10. So now I’m wondering if it’s some sort of interference.  I adjusted the 2.4ghz router to reduce its speed (from 300mbps connection to “neighbor-friendly 145mbps”) and changed it to one channel rather than “auto” and presto! I’m up to 5mbps most of the time.

So My neighbors and I must be having a channel battle, and by reducing my broadcast range/speed, I’m soldifiying my signal, or something.  5 is still way slower than 15, so I’m going to fiddle some more later.  But for now, I’ve got it working satisfactorily.  Yikes.

Big Computer Reboot

My nerd readers out there are going to be annoyed with me.  I went back to XP.

The story: A couple years ago (3 to be precise), I rebooted my computer and went halfsies, installing Ubuntu on half and WinXP on the other half.  I didn’t go all Ubuntu because I wanted to play games (w/o messing around with WINE) and my scanner is Windows only.  Since then, Ubuntu has been very nice, and I’ve had almost no problems with it.  Windows has been, well, Windows.

BUT: There are quite a few workarounds.  I have to keep rebooting to use my scanner.  To use some bit of software I need.  To do this or that.  And I don’t really gain anything from Ubuntu (except quick install time and geek cred.)

So with my tax return from this year I bought a new 1.5TB hard drive and a new(er) graphics card (Radeon 9500GT 1GB, up from a Radeon X800GT 250mb), so it’s time to do a full reboot on my home PC.  And I’ve decided not to futz around with Ubuntu.  I feel bad about this, sorta.  But it will also be nice to scan right away, to game right away.  The install is going to be a bummer, though.

  1. I started on Tuesday morning.  Putting in the new hardware took about 20 minutes.  The Dell I have is made for hardware upgrading, so I only had to undo 1 screw — all the rest was latches that pop open.  Nice.
  2. Then I went looking for my software key for Windows.  I couldn’t find it.  After two hours, I was about to give up, when I found it in the first folder I looked in–why it wasn’t there before, I blame UBUNTU.
  3. Then I started the computer and found that it worked.  Put in the XP disc and away we go.  Oh crap, I forgot that I was going to have to format my giant new hard drive.  Cue 3.5 hours of formatting.  Sigh.

Break for Tuesday afternoon date with Jenny.  **ENDORSEMENT: If you have a career that lets you be at home on Tuesday afternoons, and so does your wife, hire a babysitter and go out every week.  You won’t regret it.**

  1. That evening, I updated XP and downloaded the newest versions of FIREFOX, THUNDERBIRD, OPENOFFICE, PICASA, AUDACITY.  I then installed the Adobe Design suite, Quicken, and a few drivers and stuff.
  2. Steam is gonna take forever.  Left it downloading Half-Life2 overnight.

Someone Comes to Town, Someone Leaves Town

Someone Comes to Town, Someone Leaves Town
Someone Comes to Town, Someone Leaves Town

by Cory Doctorow; narrated by the author

GoodReads says I’ve been reading Someone Comes to Town since 3 April. That’s when I discovered and downloaded the back-episodes of Cory Doctorow’s ongoing reading of his novel. It finished last week and I’m pleased.

The book tells the story of a man in a magical family of oddities: his father is a mountain, his mother a washing machine. He’s got a bunch of magical brothers, including an evil one. The main character also undertakes crazy maker projects, like sanding his entire house and filling it with bookshelves. And helping a dumpster-diver build a citywide free wifi network.

It’s an enchanting book with major drama and a good arc, but it’s in the little details that it really succeeds. The sections about network philosophy could be excised from the magical horrorshow and be their own thing. A few other thoughts:

  • More than some of Doctorow’s other books, I feel like there are a lot of tangents or threads that weave in and out of the story but don’t get resolved. Under that umbrella of real-life’s unexplained and unrelated events, the magical elements of the story stay in bounds, and the story doesn’t feel like a cheat.
  • There’s an amusing twist in the idea that Alan (the main character) and his other brothers are each named ordinally, using the alphabet for their first initials. The names themselves are less important, so the narrator and characters refer to Alan by any random A name. It’s a little disconcerting early on but it works later.
  • That the audio-book is being performed by the author gives the interpretive elements a distinction and value that are just great. The down side, however, is that the timely updates on Doctorow’s status will seem odd and annoying in archived versions of the book. I also tired of the cuckoo clock. I like it, but Doctorow commented about how he liked it every darn time it went off.
  • I mentioned the book shelf project above, but I wanted to quote the relevant passage below. It’s the bibliophile’s fantasy.
  • I’ve pondered the title, which doesn’t have the utilitarian title like Eastern Standard Tribe or the jaunty zazz of Down and Out in the Magic Kingdom. Zazz aside, I don’t understand it. It could refer to Alan’s brothers, who interrupt his wacky life in his polished-wood bookshelf house. He could be both, the person who comes and the person who leaves.

Anyhow, worth a look or listen if you’re into his books or this sounds interesting to you.

The Book fantasy, from Cory Doctorow’s Someone Comes to Town, Someone Leaves Town, follows after the break.

Continue reading Someone Comes to Town, Someone Leaves Town

Now why y’all pickin’ on us?

I DENY YOUR SERVICE! (image by basykes)
I DENY YOUR SERVICE! (image by basykes)

In my email today — helpfully, I’m working at home today:

To the College Community,

Columbia College Chicago is currently experiencing a Denial of Service (DoS) Attack on our network. This is affecting external access to Columbia College Chicago’s web and mail servers.

NTT, our Internet Service Provider, is working on resolving the issue. This problem can only be addressed and resolved by NTT. We will provide updates as they develop and apologize for any inconvenience this delay causes.

Thank you for your patience,
Information Technology

I wonder what the motive of a DOS attack on a college would be? Also, how did they send the email?

Digikam “failed to update old database to new database format”

So my database in Digikam (my Linux photo management program) got borked and the program wouldn’t start. I just got the error message:

Failed to update old database to new database format”

and the program died. A lot of the help I found was focused on Digikam setups that had been working and had now failed through upgrades or this or that or the other. I just wanted to start the program again. It will categorize all the images and basically start over. Warning: this is a bad idea if you have used the database for very much (like tagging, sorting, or whatever). I don’t use those features, so I didn’t mind starting over with a clean database.

Here’s the solution I found:

  • Check in the top archive folder for Digikam
  • Look for a database file called something like “digikam3.db” — this is the corrupted file
  • Delete that file.
  • Start Digikam.

This option is essentially like starting over with a fresh install of Digikam. It won’t hurt your pictures, but the other stuff might be gone.