So what would happen if, let's just say, the state could no longer pay its debts or borrow any more money? States cannot declare bankruptcy (they have sovereign immunity, which means creditors would have an almost impossible time filing suit). But as explained by Slate's Christopher Beam, the federal government can place California into receivership.
This would involve the assignment of an accountant to manage the state's debt, overseen by a judge. It would be a lot like bankruptcy, except instead of following a structured set of steps--informing creditors, appointing creditors' committees, a 120-day window to file a plan, etc.--a receiver has the authority to force creditors to renegotiate loans in a speedy fashion. However, the accountant in charge would not have the power to make decisions about the state's budget, such as which programs needed to be cut and which taxes had to be raised.
No state has ever gone into receivership, and despite what some folks have suggested it's very, very unlikely that it will happen now.