Project Name

A Navigation System for Media Generation by Consumers

About this project

CORE10

This project is financially supported by Microsoft Research CORE10.
For the accepted project list of CORE10, please access here.

Project Goal

The goal of the project was to navigate ordinary users to generate higher quality media data by using big multimedia data on the Internet. Although taking photos and videos and sharing them on the social network has become very popular and common, it does not necessarily mean that the users are getting better at taking photos and videos, annotating them, and generating new contents from them than before. In this project, three types of navigation systems have been investigated:
  • Navigation of user activity design: navigating users where to go and what to do for taking good photos and for having good experiences based on their personal preferences (Fig. 1).
  • Navigation of multimedia item generation: navigating users to take good photos and videos and annotate with good text tags (Fig. 2).
  • Navigation of multimedia content generation: navigating users to merge multimedia items such as images, video clips, texts, etc., and to generate high quality multimedia content such as presentations.
          
howto_take           
Fig. 1. Personalized travel route navigation.
howto_take
Fig. 2. Navigation of how to take photos.

Members

  • Toshihiko Yamasaki, Principal Investigator, Associate Professor of The University of Tokyo
  • Shumpei Sano, Social network analysis, Master course student of The University of Tokyo
  • Ibuki Nakamura, RGB-D camera, Master course student of The University of Tokyo
  • Ryosuke Furuta, Presentation analysis, Master course student of The University of Tokyo
  • Yusuke Fukushima, Presentation analysis, Bachelor course student of The University of Tokyo
  • Jiani Hu, Social network analysis, Research student of The University of Tokyo
  • Litian Sun, Presentation analysis, Ph.D student of The University of Tokyo
  • Yuki Arase, Collaborator, Associate Professor at Osaka Univ. and previously at MSRA
  • Tao Mei, Microsoft Research
  • Danushka Bollegala, Collaborator, Associate Professor at Liverpool Univ., UK


Technical breakthrough

Not shown because some papers are under review now.

Innovative Applications

Not shown because some papers are under review now.

Publications

Publications starting with “**” are the ones with explicit acknowledgement to the grant of Microsoft Research Asia.

Journal

1) ** T. Yamasaki, Andrew Gallagher, Tsuhan Chen, and K. Aizawa, " Geotag-Based Travel Route Recommendation Featuring Seasonal and Temporal Popularity," Journal of the Institute of Electronics, Information and Communication Engineers , Vol. J970D, No. 9, pp. 1437-1444, 2014. (in Japanese)
2) ** T. Yamasaki, I. Nakamura, and K. Aizawa, " Real-Time Face Model Reconstruction and synthesis Using an RGB-D Camera for face Reorientation," The Journal of The Institute of Image Information and Television Engineers, (submitted, in Japanese)

International Conferences with Review

1) T. Yamasaki and K. Aizawa, “Travel and Photo-Shooting Navigation Based on Large-Scale Travel Photos on the Internet,” Proc. International Technical Conference on Circuits/Systems, Computers and Communications (ITC-CSCC), pp. 948-951, Thai, July 2014. (invited)
2) L. Sun, T. Yamasaki, and K. Aizawa “Relationship between visual complexity and aesthetics: application to beauty prediction of photos” 2nd Workshop on Computer VISion for ART Analysis In conjunction with ECCV 2014, CD-ROM proceedings, Zurich, Sep. 6th, 2014.
3) T. Yamasaki, T. Maeda, and K. Aizawa, “SVM is not always confident: telling whether the output from SVM is true or false by analyzing its confidence values,” IEEE International Workshop on Multimedia Signal Processing (MMSP), USB Proceedings #48, Jakarta, Indonesia, Sep 22-24, 2014.
4) S. Sano, T. Yamasaki, and K. Aizawa, “Degree of loop assessment in microvideo,” Proc. International Conference of Image Processing (ICIP), pp. 5182-5185, Paris, France, Oct 27-30, 2014.
5) ** T. Yamasaki, S. Sano, and K. Aizawa, “Social Popularity Score : Predicting Numbers of Views, Comments, and Favorites of Social Photos Using Annotations,” First International Workshop on Internet-Scale Multimedia Management (ISMM) in conjunction with ACM Multimedia, pp. 3-8, Orlando, United States, Nov. 7th, 2014.
6) ** T. Yamasaki, S. Sano, and T. Mei, “Revealing Relationships between Folksonomy and Social Popularity Score in Image/Video Sharing Services, ” IEEE International Conference on Consumer Electronics-Taiwan, Taipei, Taiwan, June 6-8 2015. (invited)
7) ** T. Yamasaki, I. Nakamura, and K. Aizawa, “Real-time face model reconstruction and synthesis using an RGB-D camera and its subjective evaluation,” Proc. International Conference of Image Processing (ICIP), 2015. (submitted)

Domestic Conferences

1) ** T. Yamasaki, H. Mukai, and K. Aizawa, ”Travel suggestion based meda data analysis and image retrieval using internet-scale images,” IEICE MVE, pp. 35-40, Tokyo, July 1st-2nd, 2014. (in Japanese)
2) A. Sano, T. Yamasaki, and K. Aizawa, “Degree of Loop Assessment in Microvideo,” Meeting on Image Recognition and Understanding (MIRU2014), SS3-23, Okayama, July 28-31, 2014.
3) S. Sano, T. Yamasaki, and K. Aizawa, ”Social Popularity Score: Predicting Numbers of Views, Comments, and Favorites of Social Photos Using Only Annotations,” IEICE MVE, MVE-2014-40, pp. 69-74, Oct. 9-10, 2014. (in Japanese)
4) T. Yamasaki, “Confidence Analysis for One-vs-All SVM-Based Multiclass Object Recognition,” IMPS, I-3-16, Nov. 12-14, 2014. (in Japanese)
5) S. Sano, T. Yamasaki, and K. Aizawa, “Analysis on the Relationships between Folksonomy and Social Popularity Score in Image/Video Sharing Services,” IMPS, I-4-06, Nov. 12-14, 2014. (in Japanese)
6) Y. Fukushima, T. Yamasaki, and K. Aizawa, “Classification and Assessment of Oral Presentations Using Internet-Scale Presentation Videos,” ITE HI, Vol. 38, No. 46, HI2014-75, pp. 57-60, Nov. 28-29, 2014. (in Japanese)
7) I. Nakamura, T. Yamasaki, and K. Aizawa, “Real-Time Face Model Reconstruction and Synthesis using an RGB-D Camera -Face Reorientation for Video Chatting and Mirror App-,” IEICE PRMU, PRMU2014-157, pp. 227-232, Feb. 19-20, 2015. (in Japanese)
8) J. Hu, S. Sano, and T. Yamasaki, “Chronological Analysis of Influential Tags on Social-Popularity in Image Sharing Service” IEICE CW, CW2014-16, pp. 39-42, Feb. 25, 2015. (in Japanese)
9) ** T. Yamasaki, Y. Fukushima, J. Xu, S. and Sakazawa, “Impression Estimation of Oral Presentations Based on Document and Voice Analysis,” IEICE MVE, MVE-2014-99, pp. 119-122, Mar. 3-4, 2015.
10) ** I. Nakamura, T. Yamasaki, and K. Aizawa, “Robust Gaze Correction for Personal Video Chatting using a Single Depth Camera,” ITE annual conf., 12-2, Aug 31st - Sep. 2, 2014. (in Japanese)
11) ** Y. Fukushima, T. Yamasaki, and K. Aizawa, “Presentation Video Assessment Based on Document Analysis,” ITE winter conf., 11-1, Dec. 17-18, 2014. (in Japanese)