Schopenhauer is pretty much the philosopher of pessimism. He was a hypochondriac, paranoid enough to sleep with loaded pistols next to him, and overall a complete jerk. (For what it's worth, many famous philosophers were also complete jerks, but it's not always reflected in their philosophy.)
Many of his contemporaries saw humans as primarily intellectual, or a kind of "meat robot," or a primarily intellectual meat robot, and they saw human progress as inevitably arriving at some state of perfection. Schopenhauer knew that was all bullshit. Humans are animals, and though they are rather smart, they are animals first, driven by personal motives and emotions. We will always need some kind of legal structure in place to keep us from killing each other.
As for the contention that we are living in the worst of all possible worlds, I don't think the argument holds up to scrutiny. For one, the fact that there is suffering at all does not mean there is a net amount of suffering when weighed against the joy in the world. Additionally, there are many other ways the world could have turned out that would make life worse. If the earth were different enough that there were fewer domesticable plants and animals, it would be far more difficult to develop agriculture and, consequently, civilization itself.*
I am thinking here of a world where no civilization advanced much further than pre-Columbian native Americans. If you think that would be a "good thing" because our species would be "more in touch with nature" and "more spiritual" than you are wrong and/or full of shit. It is a fickle existence where mere misfortune can kill your family and ruin your life. It's like a game of "Oregon Trail," where Buttface has died of dysentery, except there is no "end" to the game. So, the next year, Poop and Fart die of scarlet fever, and before you know it, you've had eleven children and run out of dirty names for them, so you kick yourself that it was only Kimberly and Thomas who lived to adulthood. That is the kind of misfortune I am talking about: a world where children with hilarious names never survive. And that's horrible.
Take that, Schopenhauer. In our world, we have people with names like Chip Munk, Anita Bath, and Dina Soares. There is no way this is the worst of all possible worlds.
* - For a more in-depth study of this, you should read Jared Diamond's "Guns, Germs, and Steel." It's a very good book. It also made me want to play the game "Civilization" bad enough that I went out and got Civlization IV, and got addicted to it for a month or two.