So on a whim your buddy talked you into marching in a protest run by racists, but you didn't know this and you're not a racist so that makes it all grand.
Fairplay. That's pretty impressive.
I'm not labelling this as something else.
The march was organised by far right racists who have ran for election on anti immigration manifestos and been rejected at the ballot box.
You can disagree with this fact all day long and it won't change.
They don't organise the match so this is a false equivalence as well as being simplistic nonsense.
If you are happy to throw your lot in with racist scumbags as a means to an end then by all means do so. Don't get upset if the same label is applied to you.