Who hasn't lost a point in which you and your partner just watch ball go by between the two of you?
Most of the action in doubles happens in the middle of the court, simply because it is easier to hit crosscourt than down the line. Points won by a shot hit in the alleys are far less more common that a point won by a ball hit down the middle, between the two players. Therefore both players should cover the middle and a clash of racquets is better than an awkward stare.
There is, however, a general rule. If you play on the deuce side, you are supposed to cover the middle when the ball is coming from the deuce side of your opponents' court. And if you play on the ad side, you are supposed to cover the middle when the ball comes from the ad side of your opponents' court. So if the ball is coming cross court toward you, the middle of the court is yours to cover. Your partner will protect his/her alley on both situations.
Nevertheless, more important than following the rule is always be ready, as a team, to protect the middle, rather than the alleys. At the end, percentage tennis wins.