History does not suggest you are correct about defeating "affirmative action". What I call the "new era" in "affirmative action" began with the Nixon administration's Philadelphia Plan in 1969. (The initial instance was Roosevelt's wartime executive order to the defense industry to hire blacks.)
So it's been almost 40 years, and "affirmative action" is alive and well. If you go back and look at the political and legal commentary leading up to the USSC Gratz and Grutter cases (which were decided in favor of upholding the University of Michigan's racist admission policies) you'll see that all the predictions were that this would be the death knell for "affirmative action". Instead, "thanks" to the soon to depart from the court Sandra Day O'Connor, "affirmative action" got a huge revival under the new euphemism "diversity". There is no indication, therefore, that it will depart anytime soon. White males therefore have to organize and fight it (and similar anti-WM trends) in the only arena left in the US that counts - politics.