We tried it the other way for a while and found that if people are completely free to choose who they associate with (specifically in the business space), it compromises other national ideals, such as meritocracy.
The curtailments on freedom of association are very narrow and focused on specific constraints.
But this country was founded on an understanding that there are some fundamental freedoms of choice that aren't there if you want to have a functioning society work together. And some of the ones that the founders believed would be acceptable we fought a bloody war to remove. The freedom to choose is inalienable but (much as we still have a right to liberty and jails at the same time) inalienable rights can be curtailed in the name of having a functioning society.
The curtailments on freedom of association are very narrow and focused on specific constraints.
But this country was founded on an understanding that there are some fundamental freedoms of choice that aren't there if you want to have a functioning society work together. And some of the ones that the founders believed would be acceptable we fought a bloody war to remove. The freedom to choose is inalienable but (much as we still have a right to liberty and jails at the same time) inalienable rights can be curtailed in the name of having a functioning society.