Manipulating Objects with the HoloLens

Hi,
So I'm relatively new to both the HoloLens and unity.

What I'm trying to do is write a script that allows me to manipulate a hologram. Scale, rotation, that kind of stuff.

The way I'm doing this is loading in some model, and then drawing little spheres around it in basically a bounding box.

Each of the little bounding spheres are instantiated with an input script attached to them. I'm using the IManipulationHandler (system?) to manipulate stuff.

So, the way it should work is that I click one of the bounding spheres and drag to, for example, rotate the parent model around the Y axis.

Now this actually works, BUT, when the cursor falls off the sphere, dragging stops working. So I can, for example, rotate it as long as the cursor is over the given bounding sphere, but as soon as it stops hitting it I can't rotate anymore.

What I want is for this to not happen. I want to be rotating as long as I'm clicked down.

I've been trying various ways to fix this and I'm not really getting anywhere. I could use some help, thank you.

Here's some example code from the manipulation script attached to the bounding spheres

public void OnManipulationUpdated(ManipulationEventData eventData)
{

            //Click and drag any of these spheres to rotate about the z axis 
    if (XY.Contains(this.gameObject)) 
    {
        this.GetComponent<Renderer>().material.color = Color.blue;
        model.transform.eulerAngles += new Vector3(0,0,  -eventData.CumulativeDelta.z * rotateRate); //Rotate about model Z 
    }

    }