Jay Cost AKA Horse Race Blog has a great take on the party switch.
In the 2004 GOP primary every county in metro Pittsburgh voted for Toomey over Specter - and Specter failed to crack 40% in several of them. In the general election that year, Specter ran behind Bush in six of the seven counties in metro Pittsburgh, even though he won the state by almost ten points and Bush lost it by two and a half. In 1992 - the last time Specter faced a tough general election challenge - his opponent, Lynn Yeakel, won six of the seven counties that border Ohio. Additionally, Toomey defeated Specter in York and Lancaster counties in the 2004 primary. Specter's narrow victory in the primary depended entirely on him sweeping Toomey in metropolitan Philadelphia, whose declining importance in the statewide Republican electorate has now made Specter exceedingly vulnerable.
When he entered the Senate, metropolitan Philadelphia, his home base, was also the GOP's base in the state. In 1980 four of the five counties in Philadelphia voted for Reagan while five of the seven counties in metro Pittsburgh voted for Carter. This has basically been inverted in the last quarter century - and while neither party's presidential candidate has been better off statewide for this shift, Arlen Specter has personally been on the losing end.
The interpretation from the wise political sages in Washington, D.C. is inevitably going to be about how the hardened, conservative rump Republican Party is so intolerant of a moderate like Arlen Specter that he had no choice but to bolt. However, this is quite an oversimplification. There is a big geographical component to this story: the west has become more important in party politics, and Specter has long been weak in the west.