Are we supposed to be able to use the Xbox controller outside of simulation mode?

I cannot get my controller to work regularly with the mixed reality portal app not in simulation mode. However, I have read reviews of people using it to teleport around and rotate themselves. Has anyone run into a similar issue?