ExPixel: PixelShader for multiplex-image hiding in consumer 3D flat panels

Motivation: Beyond 3D display

In recent years, 3D technology has become so widespread that the technology alone no longer fascinates viewers. New breakthroughs are necessary to keep the audience entertained. Thus, a generic multiplexing technique using existing 3D stereoscopic technology was developed. This breakthrough 3D technology enables the viewing of two contents in one single screen at the same time. It is considered to meet users’ expectations. The application of multiplex image hiding techniques to the widespread consumer 3D flat panels without any hardware modification can be valuable in terms of the new usage of 3D devices.

Challenge: multiplex-image hiding on 3D flat panels

Polarization glasses, a passive 3D stereoscopic technology that shows two images with the parallax to the right and left eyes, are currently the most widely used 3D technique in virtual reality and movie theaters. In previous research, ScritterHDR and 2x3D were proposed to realize multiplex-image hiding using a multi-projection-based technique [Nagano et al. 2010;Fujimura et al.2012]. These techniques are valuable in maintaining the resolution in large displays, such as in theaters. However, the stacked projector setup is complicated and requires a large space for consumer use at home. The application of the multiplex-image hiding imagery technique was used on a 3D flat panel based on an existing line-by-line polarized device, such as Xpol and/or CINEMA3D. In this line-by-line method, each line is designed for either the left or the right eye, and the two lines right above and below it are to be seen by the other eye. A 3D stereoscopic view is seen when this setup is viewed with polarized glasses. As a first result, multiple imagery works well in most consumer products. Two images A and B are turned into images made of odd and even pixel lines, respectively, and combined to display image AB on a flat panel screen. This approach is similar to the SimulView technique by SONY’s passive 3D display. For hidden imagery as a new function, each image is mapped into a limited range of actual devices, such as 0-255, and image cancelation considers gamma correction.

ExPixel: PixelShader for line-by-line multiplex hiding

Using the below algorithm, image C may be placed on the even line and image B on the odd line. This multiplex hidden image is called “ExPixel.” A real-time PixelShader technique operates this algorithm on GPU. To make this multiplex-image hiding method accessible, Unity 3D plugin was developed for the easy use of programmers and artists in creating new games and contents. “ExPixel” increases the interactivity of current display. Therefore, game developers and artists have a new way to express themselves and create new types of works.

Reference

FUJIMURA, W., KOIDE, Y., SONGER, R., HAYAKAWA, T., SHIRAI, A., AND YANAKA, K. 2012. 2x3d: Real time shader for simultaneous 2d/3d hybrid theater. In SIGGRAPH Asia 2012 Emerging Technologies, ACM, New York, NY, USA, SA ’12, 1:1–1:2.
NAGANO, K., UTSUGI, T., HIRANO, M., HAMADA, T., SHIRAI,A., AND NAKAJIMA, M. 2010. A new ”multiplex content” displaying system compatible with current 3d projection tech-nology. In ACM SIGGRAPH 2010 Posters, ACM, New York,NY, USA, SIGGRAPH ’10, 79:1–79:1.
Pranav Mistry: ThirdEye-Multiple viewers can see different things on a single screen，SIGGRAPH Asia 2009.
Ian E. McDowall, et al. : Snared Illumination, SIGGRAPH 2004, Emerging Technology.
Sony Computer Entertainment America: Display viewing system and methods for optimizing display view based on active tracking, Patent No. US8269822B2
Takeo HAMADA，Koki NAGANO，Takeru UTSUGI，Mika HIRANO，Akihiko SHIRAI：Scritter: A multiplexed image system for a public screen，Proceedings of Virtual Reality International Conference (VRIC 2010), 2010.
櫻井智史，浅井和広，北村喜文：Mysterious POND, 日本バーチャルリアリティ学会論文誌,vol.15, pp.75-81, 2010-03-31.
宇津木健，長野光希，谷中一寿，白井暁彦，山口雅浩: 多重化映像表示における隠蔽映像生成アルゴリズム, 第15回日本VR学会大会論文集,2010.
藤村航, 小出雄空明, 早川貴奉, 谷中一寿, 白井暁彦: 2X3D：2D+3D同時上映可能なハイブリッドシアター, 第17回日本VR学会大会論文集, 2012.
小出雄空明, 奈良優斗, 藤村航, 白井暁彦:UbiCode:パブリックディスプレイへのバーチャルインタラクティビティの追加, 第17回日本VR学会大会論文集, 2012.
八谷和彦：FairyFinder，www.petworks.co.jp/~hachiya/works/FairyFinder.html
Satoshi Sakurai, Yoshifumi Kitamura, Sriram Subramanian, and Fumio Kishino: A Visibility Control System for Collaborative Digital Table, Personal and Ubiquitous Computing, Special Issue on Interaction with Coupled and Public Displays, Vol. 13, Issue 8, pp. 619-632, Springer (2009). [PDF]
Satoshi Sakurai, Yoshifumi Kitamura, Sriram Subramanian, Fumio Kishino: Visibility Control using Revolving Polarizer; Proceedings of IEEE Workshop on Tabletops and Interactive Surfaces, pp. 173-180 (2008). [PDF]
櫻井智史, 浅井和広, 北村喜文: Mysterious POND, 日本バーチャルリアリティ学会論文誌, Vol. 15, No. 1, pp. 75-82, 2010年3月. [PDF]
櫻井智史, 北村喜文, Sriram Subramanian, 岸野文郎, 回転する直線偏光板を用いた情報可視性の制御方法, ヒューマンインタフェース学会論文誌, Vol. 11, No. 1, pp. 145-153, 2009年2月. [PDF]
櫻井智史, 北村喜文, Sriram Subramanian, 岸野文郎: 回転偏光フィルタにより情報の可視性を制御するテーブルトップ型ディスプレイ,
情報処理学会論文誌, Vol. 50, No. 1, pp. 332-343, 2009年1月. [PDF]
Gary M. Zalewski. Sep. 18, 2012. Display Viewing System and Methods for Optimizing Display View Based on Active Tracking, US Patent No.: 8269822 B2
Nagano, K., Utsugi, T., Yanaka, K., Shirai, A. and Nakajima, M. 2011. ScritterHDR: Multiplex-Hidden Imaging on High Dynamic Range Projection, SIGGRAPH ASIA 2011 Technical Sketches & Posters.
Fujimura, W., Koide, Y., Songer, R., Hayakawa, T., Shirai, A. and Yanaka, K. 2012. 2x3D: Real-Time Shader for Simultaneous 2D/3D Hybrid Theater, ACM SIGGRAPH ASIA 2012 Emerging Technologies, Singapore.
Fujimura, W., Koide, Y., Sakai, T., Songer, R., Kosaka, T. and Shirai, A. 2012. PARAOKE alpha: a new application development of multiplex-hidden display technique for music entertainment system, NICOGRAPH International 2012, Bali.
Xiaolin Wu and Guangtao Zhai. 2012. Backward compatible stereoscopic displays via temporal psychovisual modulation. In SIGGRAPH Asia 2012 Emerging Technologies (SA '12).