It’s supported by the show itself. Brain’s plans always fail. The one time Pinky was given a chance, he would have succeeded had the Brain not intervened.