Yirui Jiang
1
,
Trung Hieu Tran
2
,
Leon Williams
3
,
Jaime Palmer
4
,
Edgar Simson
5
,
Daniel Benson
6
,
Michael Christopher
7
,
Daila Christopher
8
1
Cranfield Centre for Competitive Creative Design, School of Water, Energy and EnvironmentCranfield University, Bedfordshire, UK
2
Cranfield Centre for Competitive Creative Design, School of Water, Energy and EnvironmentCranfield University, Bedfordshire, UK
3
Cranfield Centre for Competitive Creative Design, School of Water, Energy and EnvironmentCranfield University, Bedfordshire, UK
4
OrangeLV, Inc., Seattle WA, USA
5
OrangeLV, Inc., Seattle WA, USA
6
OrangeLV, Inc., Seattle WA, USA
7
OrangeLV, Inc., Seattle WA, USA
8
OrangeLV, Inc., Seattle WA, USA
Abstract
Nowadays, customization by mixed reality to enhance the cus-tomer experience plays an important role in the retail industry. Customers can choose and customize products with their images and labels in a virtual reality environment. However, the existing asset creation pipelines are labor-intensive and time-consuming to display the images and labels (aka logos) on 3D product models, and cannot be easily customized by customers in real-time. In this paper, we thus propose a real-time 3D logo mapping framework for converting 3D logo mesh from a specified image and fitting it to the 3D product models. In the framework, Convolutional Neural Net-work (CNN) is adopted to reconstruct 3D logo/product models from their images. The detailed 3D information and the logo location provided by a customer are used to select the effective sampling points to mesh deformation. This method can preserve both the vi-sual quality and details of 3D product models. Experimental results,carried out on various sizes of logos and types of products, show that our method can produce accurately and quickly customized logos on 3D product models.
Keywords
Mixed reality,Convolutional neural network,Customization, Logo,Retail industry
How to Cite
Jiang, Y., Tran, T. H., Williams, L., Palmer, J., Simson, E., Benson, D., … Christopher, D. (2026). Enhancing the Customer Experience by Mixed Reality in the Retail Industry. Asia Journal of Social Innovation and Development, 2(1), 5. Retrieved from https://www.ajsid.org/index.php/pub/article/view/29
📄[1] Furukawa, Y., & Ponce, J. 2009. Accurate, dense, and robust multiview stereopsis.IEEE transactions on pattern analysis and machine intelligence, 32(8), 1362-1376.
📄[2] Wen, C., Zhang, Y., Li, Z., & Fu, Y. 2019. Pixel2mesh++: Multi-view 3d mesh gener-ation via deformation. In Proceedings of the IEEE/CVF International Conference on Computer Vision (pp. 1042-1051).
📄[3] Bentley, J. L. 1975. Multidimensional binary search trees used for associative searching. Communications of the ACM, 18(9), 509-517.
📄[4] Yan, D. M., Liu, Y., & Wang, W. 2006. Quadric surface extraction by variational shape approximation. In International conference on geometric modeling and processing (pp. 73-86). Springer, Berlin, Heidelberg.
📄[5] Sorkine, O., & Alexa, M. 2007. As-rigid-as-possible surface modeling. In Sympo-sium on Geometry processing (Vol. 4, pp. 109-116).
📄[6] Choy, C. B., Xu, D., Gwak, J., Chen, K., & Savarese, S. 2016. 3d-r2n2: A uni-fied approach for single and multi-view 3d object reconstruction. In European conference on computer vision (pp. 628-644). Springer, Cham.
📄[7] Smink, A. R., Frowijn, S., van Reijmersdal. 2019. Try online before you buy: How does shopping with augmented reality affect brand responses and personal data disclosure. Electronic Commerce Research and Applications, 35, 100854.
📄[8] Tulsiani, S., Kar, A., Carreira, J., & Malik, J. 2016. Learning category-specific deformable 3d models for object reconstruction. IEEE transactions on pattern analysis and machine intelligence, 39(4), 719-731.
📄[9] Tulsiani, S., Efros, A. A., & Malik, J. 2018. Multi-view consistency as supervi-sory signal for learning shape and pose prediction. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 2897-2905).
📄[10] Gwak, J., Choy, C. B., Garg, A., Chandraker, M., & Savarese, S. 2017. Weakly supervised generative adversarial networks for 3d reconstruction. arXiv preprint arXiv:1705.10904, 2.
📄[11] Pumarola, A., Agudo, A., Porzi, L., Sanfeliu, A. 2018. Geometry-aware network for non-rigid shape prediction from a single view. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 4681-4690).
📄[12] Wang, N., Zhang, Y., Li, Z., Fu. 2018. Pixel2mesh: Generating 3d mesh models from single rgb images. In Proceedings of the European Conference on Computer Vision (ECCV) (pp. 52-67).
📄[13] Fan, H., Su, H., & Guibas, L. J. 2017. A point set generation network for 3d object reconstruction from a single image. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 605-613).
📄[14] Kar, A., Häne, C., & Malik, J. 2017. Learning a multi-view stereo machine. arXiv preprint arXiv:1708.05375.
📄[15] Kobbelt, L., Campagna, S., Vorsatz, J., & Seidel, H. P. 1998. Interactive multi-resolution modeling on arbitrary meshes. In Proceedings of the 25th annual conference on Computer graphics and interactive techniques (pp. 105-114).
📄[16] Botsch, M., & Kobbelt, L. 2004. An intuitive framework for real-time freeform modeling. ACM Transactions on Graphics (TOG), 23(3), 630-634