The video above is taken from Rubin’s Flash Panorama Laboratory. It is the result of experiments using non-professional commercial cameras to create “the world’s most affordable video solution capable of shooting and virtually recreating all possible viewing directions of the environment”. As you can see, the custom created video player allows you to move around the field of view, and zoom in and out. You can play with it on their site.
It would be interesting to explore the sense of immersion that could be produced by tying this into a head mounted (or even just more directly mouse controlled) interface, with the possibility of recreating recorded first-person experiences a la Johnny Mnemonic.