in MR
Productivity in MR

Peer-Reviewed Conference Publications


R. Li (UW), T. Seyed, N. Marquardt, E. Ofek. S. Hodges, M. Sinclair, H. Romat, M. Pahud, J. Sharma, W.A.S. Buxton, K. Hinckley and N. Riche

AdHocProx: Sensing Mobile, Ad-Hoc Collaborative Device Formations using Dual Ultra-Wideband Radios

To appear at CHI 2023, Hamburg, Germany

Y. Tao (Stanford), C.Y. Wang (Cornell), A.D. Wilson, E. Ofek and M. Gonzalez-Franco

Embodying Physics-Aware Avatars in Virtual Reality

To appear at CHI 2023, Hamburg, Germany

V. Ranganeni, M. Sinclair, E. Ofek, A. Miller, J. Campbell, A. Kolobov & E. Cutrell

Exploring Levels of Control for a Navigation Assistant for Blind Travelers

To appear at Human-Robot Interaction 2023, Stockholm, Sweden

Video, Paper, arXiv version


C.Y. Wang, E. Ofek, D. McDuff, Oron Nir, S. Vemprala, A. Kapoor, M. Gonzalez-Franco

CityLifeSim: A High-Fidelity Pedestrian and Vehicle Simulation with Complex Behaviors


Video, Paper, Github code

J. Lee, R. Natarrajan, P. Panda, S. Rodriguez & E. Ofek

RemoteLab: Virtual Reality Remote Study ToolKit

UIST 2022, Article No.: 51, pp 1–9

Paper, Open-Source project

P. Panda, M. Nicholas, M. Gonzalez-Franco, K. Inkpen, E. Ofek, R. Cutler, K. Hinckley and J. Lanier

All Together: Effect of Avatars in Mixed-Modality Conferencing Environments



Applying facial animations consistently over different Microsoft Rocketbox avtars.

M. Volonte, E. Ofek, K. Jakubzak, S. Bruner, and M. Gonzalez-Franco

HeadBox: A Facial Blendshape Animation Toolkit for the Microsoft Rocketbox Library.

2022 IEEE VR Workshop (VRW)



D. Jain (UW), S. Junuzovic, E. Ofek, M. Sinclair, J.R. Porter, C. Yoon, S. Machanavajhala and M. Ringel Morris

Towards Sound Accessibility in Virtual Reality

ACM ICMI 2021 

Paper video

D. Jain, S. Junuzovic, E. Ofek, M. Sinclair, J. R. Porter, C. Yoon, S. Machanavajhala, M. Ringel Morris.

A taxonomy of Sounds in Virtual Reality.

Best Paper DIS 2021

Paper Talk

DOI: 10.1145/3461778.3462106

Best Paper

Suzuki R., Ofek, E., M. Sinclair, M., Leithinger, D. and Gonzalez-Franco, M.

HapticBots: Distributed Encountered-type Haptics for VR with Multiple Shape-changing Mobile Robots

UIST 2021

Paper, Video

DOI: 10.1145/3472749.3474821

Gonzalez E.J., Ofek, E. Gonzalez-Franco M. and M. Sinclair

X-Rings: A Hand-mounted 360 Degree Shape Display for Grasping in Virtual Reality

Best Paper: Honorable mention, UIST 2021

Paper, Video

DOI: 10.1145/3472749.3474782

Best Paper – Honorable Mention

Marquant, N., Henry Riche, N., Holz, C. Romat, H., Pahud, M., Brudy, F., Ledo, D., Park, C., Nicholas, M.J., Seyed, T., Ofek, E., Lee, B., Buxton, W.A.S. and Hinkley, K.

AirConstellations: In-Air Device Formations for Cross-Device Interaction via Multiple Spatially-Aware Armatures

UIST 2021

Paper, Short_Video, Video

DOI: 10.1145/3472749.3474820


M. Gonzalez-Franco, Z. Egan, M. Peachey, A. Antley(MS), T. Randhavane (MS), P. Panda (NC State U), Y. Zhang, D. F. Reilly (Dalhousie U), T. C Peck (Davidson College), A. Stevenson Won (Cornell), C. Y. Wang (Cornell), A. Steed (UCL) and E. Ofek

MoveBox: Democratizing MoCap for the Microsoft Rocketbox Avatar Library

AIVR 2020


B. Cohn, A. Maselli, E. Ofek and M. Gonzalez-Franco

SnapMove: Movement Projection Mapping in Virtual Reality

AIVR 2020

Paper,  Talk

T. Gesslein, V. Biener, P. Gagel, D. Schneider, E. Ofek, M. Pahud, P.O. Kristensson and J. Grubert 

Pen-based Interaction with Spreadsheets in Mobile Virtual Reality

ISMAR 2020

arXivVideoVentureBeat “Researchers bring Google Sheets and Microsoft Excel into VR

V. Biener, D. Schneider, T. Gesslein, A. Otte, B. Kuth,,  P.O. Kristensson, E. Ofek, M. Pahud, and J. Grubert 

Breaking the Screen: Interaction Across Touchscreen Boundaries in Virtual Reality for Mobile Knowledge Workers

ISMAR Nov 2020

arXivVideoIEEE ISMAR 2020 presentationDOI

M. Gonzalez Franco, M. Sinclair, and E. Ofek

Asymmetry of Grasp in Haptic Perception

To Appear: ACM Symposium on Applied Perception 2020 (SAP ’20) | September 2020


DOI: 10.1145/3385955.3407934

E. Ofek, J. Grubert (Coburg), M. Pahud, M, Phillips (Coburg), and Per-Ola Kristensson (Cambridge)

Towards a Practical Virtual Office for Mobile Knowledge Workers

Symposium on The New Future of Work 2020 (NFW’20)

Paper, BibTex, Video

M. Gonzalez Franco (MSR), B. Cohn (USC), D. Burin, E. Ofek (MSR), and A. Maselli

The Self-Avatar Follower Effect in Virtual Reality

IEEE VR 2020 Best Paper Candidate.


R. Gruen, E. Ofek, A. Steed (UCL), R. Gal, M. Sinclair & M. Gonzalez Franco

Measuring System Visual Latency through Cognitive Latency on Video See-Through AR devices

IEEE VR 2020 Honorable Mention paper (Top 5 papers)


Best Paper – Honorable Mention

M. Gonzalez-Franco (MSR) , A. Steed (UCL), S. Hoogendyk (Microsoft) and E. Ofek (MSR).

Using Facial Animation to Increase the Enfacement Illusion and Avatar Self-Identification. 

IEEE VR 2020


D. Schneider  (Coburg), A. Otte (Coburg), A. S. Kublin (Coburg), P. O. Kristensson  (Cambridge),  E. Ofek (MSR), M. Pahud (MSR), A. Martschenko (Coburg) and J. Grubert (Coburg)

Accuracy of Commodity Finger Tracking Systems for Virtual Reality Head-Mounted Displays.

IEEE VR Poster 2020

A. F. Siu (Stanford), M. Sinclair, R. Kovacs, C. Holz, E. Ofek & E. Cutrell.

Virtual Reality Without Vision: A Haptic and Auditory White Cane to Navigate Complex Virtual Worlds. 

CHI 2020 Honorable Mention paper

PaperVideoProjectMSR BlogCHI 2020 Talk – Alexa F. Sui 

Scientific American: “Virtual Reality Has an Accessibility Problem”

Best Paper – Honorable Mention


S. Marwecki (MSR & Hasso Plattner Institute, U. of Potsdam), A. Wilson, E. Ofek, M. Gonzalez Franco and C. Holz

Mise-Unseen: Using Eye Tracking to Hide Virtual Reality Scene Changes in Plain Sight. 

UIST 2019

PaperVideoMicrosoft Research Blog


Upload: “Microsoft Researchers Explore Eye Tracking Uses For VR”

Engadget: “Microsoft’s latest VR experiment is a literal walk in the park”

J. Yang (Stanford & MSR), C. Holz, E. Ofek and A. Wilson

DreamWalker: Substituting Real-World Walking Experiences with a Virtual Reality. 

UIST 2019

PaperVideoMicrosoft Research Blog

ArsTechnica:  “Microsoft’s DreamWalker VR turns your daily commute into a totally different one”

RoadToVR: “Microsoft ‘DreamWalker’ Experiment Takes First Steps into Always-on World-scale VR”

Engadget: “Microsoft’s latest VR experiment is a literal walk in the park”

Gizmodo: “Microsoft’s Over-the-Top VR Rig Lets You Explore a Virtual World While Walking IRL”

MSN:  “Microsoft DreamWalker project takes VR technology outdoors “

ZDnet: “Dream or nightmare? Microsoft’s VR swaps your real walking route for a virtual one”

DailyMail: “Microsoft reveals its DreamWalker VR rig that lets users explore a virtual world while walking around in real life”

M. Sinclair, E. Ofek, M. Gonzalez-Franco and C. Holz

CapstanCrunch: A Haptic VR Controller with User-supplied Force Feedback.

UIST 2019 Honorable Mention demo

PaperVideoMicrosoft Research BlogDOI:10.1145/3332165.3347891

Engadget: “Microsoft’s latest VR experiment is a literal walk in the park”

Upload: “Microsoft Shows Off Force Resistance VR Grasping Feature Prototype”

WinBuzzer: “Microsoft Research Reveals CapstanCrunch VR Haptic Control Method”

Best Demo – Honorable Mention

D. Schneider1, A. Otte1, T. Gesslein1, P. Gagel1, B. Kuth1, M. S. Damlakhi1, O. Dietz1, E. Ofek2, M. Pahud2, P. O. Kristensson3, J. Muller4 and J. Grubert1  

1 Cuborg University, 2 Microsoft Research 3 Cambridge 4 Bayreuth

ReconViguRation: Reconfiguring Physical Keyboards in Virtual Reality.  

ISMAR 2019, Best Paper Award


Best Paper

Y. Zhao (Cornell Tech & MSR), E. CutrellC. HolzM. Ringel MorrisE.Ofek and A. Wilson

SeeingVR: A Set of Tools to Make Virtual Reality More Accessible to People with Low Vision. 

CHI 2019.

PaperVideoMicrosoft Research BlogA Demo at Microsoft Research’s Faculty Summit 2019

DOI: 10.1145/3290605.3300341

Engadget: “Microsoft is making VR better for those with vision problems” 

RoadToVR: “Microsoft Aims to Improve VR for Users with Vision Problems”

Scientific American: “Virtual Reality Has an Accessibility Problem”

J. Lee (Kaist & MSR), M. Sinclair, M. Gonzalez-Franko, E. Ofek and C. Holz

TORC: A Virtual Reality Controller for In-Hand High-Dexterity Finger Interaction. 

CHI 2019.

PaperVideoMicrosoft Research Blog,


VentureBeat: “Microsoft’s TORC will let you feel squeezable objects in AR and VR”

UploadVR: “Microsoft Shows Off Haptic VR Controller, Simulates Grabbing With Thumb & Two Fingers”

P. Abtahi (Stanford & MSR), M. Gonzalez-Franco, E. Ofek, A. Steed (UCL)

I’m a Giant: Walking in Large Virtual Environments at High Speed Gains. 

CHI 2019.


PaperVideoMicrosoft Research Blog

J. Hartmann (Waterloo & MSR), C. Holz, E. Ofek, & A. D. Wilson

Reality Check: Blending Virtual Environments with Situated Physical Reality

CHI 2019.




C. Holz & E. Ofek

Doubling the Signal Quality of Smartphone Camera Pulse Oximetry Using the Display Screen As a Controllable Selective Light Source.. IEEE Engineering in Medicine and Biology Society (EMBC’18)  2018


I. Choi (Stanford), E. Ofek, H. Benko, M. Sinclair and C.Holz

CLAW: A Multifunctional Handheld Haptic Controller for Grasping, Touching, and Triggering in Virtual Reality

CHI 2018

Paper, VideoMicrosoft Research Blog,


Cited by more than 150 publications

Engadget (“Microsoft’s mad scientists are making AR more tactile”), Co. Design (“Crazy Microsoft is the best Microsoft”), GizmodoRoad To VR

VR Times: “The Best Haptic VR Devices and Innovations for the Real Feel in VR Environments”

E. Strasnick (Stanford), C. Holz, E. Ofek, M. Sinclair and Hrvoje Benko

Haptic Links: Bimanual Haptics for Virtual Reality Using Variable Stiffness Actuation 

CHI 2018

PaperVideo,  Microsoft Research Blog,


Engadget (“Microsoft’s mad scientists are making AR more tactile”), Co. Design (“Crazy Microsoft is the best Microsoft”), GizmodoRoad To VR,  SlashgearVRScout

VR Times: “The Best Haptic VR Devices and Innovations for the Real Feel in VR Environments”

E. Whitmire (UW), H. Benko, C. Holz, E. Ofek and Mike Sinclair

Haptic Revolver: Touch, Shear, Texture, and Shape Rendering on a Reconfigurable Virtual Reality Controller

CHI 2018 Best Paper: Honorable Mentioned

PaperVideo. Microsoft Research Blog,

Cited by more than 150 publications

Engadget (“Microsoft’s mad scientists are making AR more tactile”), Co. Design (“Crazy Microsoft is the best Microsoft”), GizmodoRoad To VR

Best Paper – Honorable Mention

J. Grubert (Cuborg), L. Witzani, E. Ofek, M. Pahud, M. Kraz and P.O. Kristensson (Cambridge)

Effects of Hand Representations for Typing in Virtual Reality

IEEE VR Mar. 2018

PaperVideo, Microsoft Research Blog

J. Grubert (Cuborg), L. Witzani, E. Ofek, M. Pahud, M. Kraz and P.O. Kristensson (Cambridge)

Text Entry in Immersive Head-Mounted Display-based Virtual Reality using Standard Keyboards

IEEE VR Mar. 2018

PaperVideo,  Microsoft Research Blog


L. P. Cheng (HPI) E. Ofek, C. Holz, H. Benko & A. D. Wilson

Sparse Haptic Proxy: Touch Feedback in Virtual Environment Using a General Passive Prop.

CHI 2017

Cited by more than 130 publications


J. Grubert (Cuborg), E. Ofel, M. Pahud,

GlassHands: Interaction Around Unmodified Mobile Devices Using Sunglasses

ACM ISS 2016

PaperVideo, msPowerUser (“Microsoft’s latest Human Interaction Research will blow your mind”) , CMWinBuzzerIT News

H. BenkoC. Holz, M. Sinclair, E. Ofek
NormalTouch and TextureTouch: High-fidelity 3D Haptic Shape Rendering on Handheld Virtual Reality Controllers
UIST 2016 Honorable Mentioned paper,

Wired UKDigital ThrendsRoad to VRMakezine

Cited by more than 200 publications

Best Paper – Honorable Mention

B.  Nuernberger, E. Ofek, H. Benko &  A. D. Wilson
SnapToReality: Aligning Augmented Reality to the Real World. 

CHI 2016

M. Azmandian, M. Hancock, H. Benko, E. Ofek, A. D Wilson
Haptic Retargeting: Dynamic Repurposing of Passive Haptics for Enhanced Virtual Reality Experiences
CHI 2016 (Best video)
Paper, Video, Gamasutra, TechSpotWearable

Cited by more than 300 publications.

B. Ens (Manitoba), E. Ofek, N. Bruce (Manitoba) & P. Irani (Manitoba)
Spatial Constancy of Surface-Embedded Layouts across Multiple Environments
The ACM Symposium on Spatial User Interaction (SUI)  2015


J.Wilk, D. Molnar, E. Ofek, C. Rossbach, B. Livshits, A. Moshchuk, H.J.Wang & R. Gal

SurroundWeb: Mitigating Privacy Concerns in a 3D Web Browser. IEEE Security & Privacy 2015

Video, Tech Report

Selected media coverage:

BBC (TV coverage), Gizmodo, PCWorld, The Telegraph, SlashGear, Daily Mail


S. Jana, D. Molnar, A. Moshchuk, A. Dunn, B. LivshitsH. J. Wang, & E. Ofek

Enabling Fine-Grained Permissions for Augmented Reality Applications With Recognizers

22nd Usenix Security Symposium 2013

Y. Eshet, S. Korman, E. Ofek & S. Avidan

DCSH – Matching Patches in RGBD Images



L. D’Antoni, A. Dunn, S. Jana, T. Kohno, B. LivshitsD. Molnar,
A. Moshchuk, E. Ofek, F. Roesner, S. Saponas, M. Veanes & H. J. Wang

Operating System Support For Augmented Reality Applications

The 14th Workshop on Hot Topics in Operating Systems (HotOS XIV), 2013, New Mexcio, USA


B. Jones, H. Benko, E. Ofek & A. Wilson


CHI 2013 Best Paper

CHI 2013 Golden Mouse award (Best Video show)

Cited by more than 300 publications

Selected Media Coverage: Engadget | The Verge | NBC | CNET | IGN | Kotaku

Best Paper


B. Zhang, Q. Li, H. Chao, B. Chen, E. Ofek & Y.Q. Xu

Annotating and Navigating Tourist Videos


B. Epshtein, Y. Wexler & E. Ofek 

Detecting Text in Natural Scenes with Stroke Width Transform

IEEE CVPR 2010 (Oral – Top 10%)

Video of the CVPR talkCited by more than 1800 publications).

R. Gal (Tel Aviv), Y. Wexler, H. Hoppe, E. Ofek & D. Cohen-Or

Seamless Montage for Texturing Models

EuroGraphics 2010

Paper     Supplimental Materials    Optimization Animation

 Cited by more than 150 publications


N. VillarS. Izadi, H. Benko, J. Helmes, D. Rosenfeld,E. Ofek, J. Westhues, A. Butler, X. Cao, B. Chen &  S. Hodges

Mouse 2.0: Multi-Touch Meets the Mouse

UIST 2009 Best Paper

Paper      Video  

Best Paper

T. Yamaguchi, B. Wilburn , Z. Cao & E. Ofek
Video-based modeling of dynamic hair.   

Pacific-Rim Symp. on Image & Video Technology 2009, Tokyo, Japan



P. Mishra, E. Ofek & G. Kimchi
Validation of Vector Data using Oblique Images
ACM SIGGIS 2008, CA USA (Full Paper)


B. Epshtein, E. Ofek, Y. Wexler and P. Zhang

Hierarchical photo organization using geo-relevanceACM GIS 2007 (Full paper)



N.Li, N. Moraveji, H. Kimura and E. Ofek

Improving the Experience of Controlling Avatars in Camera-Based Games Using Physical Input

The 14th ACM Int. Conf. on Multimedia, Santa Barbara, CA 2006

Paper, Videos – Whack-a-mole, Boxing (RT, using 2D camera).

H. Jiang, E. Ofek, N. Moraveji and S. Yuanchun

Direct Pointer: Direct Manipulation for Large Display Interaction using Handheld Cameras

ACM CHI  2006

Paper    Video


Y. Matsushita, E. Ofek, X. Tang and H. Shum

Video Completion with Motion Inpainting for Video Stabilization. IEEE CVPR  2005 (Oral)

Paper (Cited by more than 180 publications).

B. Chen (Stanford), E. Ofek, H. Shum and M. Levoy (Stanford)

Interactive Deformation of Light Fields


Paper   Video

X. Cao, E. Ofek, and D. Vronay

Evaluation of Alterantive Presentation Control Techniques

ACM CHI 2005


R. Gvili, A. Kaplan, E. Ofek, G. Yahav

 Depth KeyStereoscopic Displays and Applications: The Engineering Reality of Virtual Reality 2003  (Proceedings of SPIE/IS&T Volume 5006)

Video      Video2

Cited by more than 160 publications


C. Fehn, P. Kauff, M. Op de Beeck, F. Ernst, W. IJsselsteijn, M. Pollefeys, L. Van Gool, E. Ofek and I. Sexton

An Evolutionary And Optimized Approach On 3D-TV. Proc. of IBC, Sep.  2002. Vol 2. pp. 357-365

Cited by more than 220 publications.

Peer-Reviewed Workshop Papers


J. Grubert (Coburg), E. Ofek, M. Pahud, Per-Ola Kristensson (Cambridge)

Back to the Future: Revisiting Mouse and Keyboard Interaction for HMD-based Immersive Analytics

CHI2020 workshop Immersive Analytics: Envisioning Future Productivity for Immersive Analytics (IA2020), Mar 2020.


M. Mott, E. Cutrell, M. Gonzalez Franco, C. Holz, E. Ofek and M. Ringel Morris

Accessible by Design: An Opportunity for Virtual Reality

ISMAR 2019 Workshop on Mixed Reality and Accessibility | Oct 2019


M. Pahud, E. Ofek, N. H. Riche, M. C. Hurter (ENAC) and J. Grubert (Coburg)

Mobiles as Portals for Interaction with Virtual Data Visualizations

Data Visualization on Mobile Devices Workshop 2018



C. Hurter, N. H. Riche, M. Pahud, E. Ofek, S. Drucker, B. Lee, D. Brown, C. Wong

Into the mixed reality data sphere: mapping user’s movements to data exploration tools

#Immersive2017 Workshop on Immersive Analytics: Exploring Future Interaction and Visualization Technologies for Data Analytics

at IEEE VIS, 1st October, Phoenix.

arXiv papers


Yamagami, M., Junuzovic, S., Gonzalez-Franco, M., Ofek, E., Cutrell, E., Porter, J., Wilson, A. and Mott, M.

Two-In-One: A Design Space for Mapping Unimanual Input into Bimanual Interactions in VR for Users with Limited Movement


Book Chapters


B. Ens, E. Ofek, N. Bruce, and P. Irani. 2016.

Shared Façades: Surface-embedded layout management for ad-hoc collaboration using head-worn displays.

Collaboration Meets Interactive Surfaces and Spaces (CMISS) – Theory and Practice.

C. Anslow, P. Campos, J. Jorge, (Eds). Springer, 2016