Crowd behaviour Mining at the Royal Society Summer Science Exhibition 2015

Crowd Behavior Mining with Virtual Environments

Crowd Behavior Mining with Virtual Environments

Invited article – Presence: Teleoperators & Virtual Environments, MIT Press. [link]

This article explores ways in which virtual environments can be used for crowdsourcing and behavior mining for filling gaps within the information space of topical research. Behavior mining in this article refers to the act of harvesting the latent or instinctive behavior of participants, usually a crowd, and injecting the population behavior into a pre-set context, such as within a virtual environment so that the subjective behaviors and the contexts are merged. The experimental approach combines various modalities centered upon virtual environments so as to induce presence in order to bring participants into the context. This approach is new and not well studied, however, it has real potentials in research dealing with behaviors and culture in reconstructed virtual environments. Two virtual environments case studies at the 2012 and 2015 Royal Society Summer Science Exhibition were presented, which demonstrated that the unique crowdsourcing activity is able to fill up gaps within the information space so that answers to research questions can be more complete. Thus, by reconstructing and replicating a lost landscape, and by injecting harvested human behavior into the context of the landscape, we may be able to gather much more information than conventional methods will allow.

Crowd behaviour Mining at the Royal Society Summer Science Exhibition 2015

Crowd behaviour Mining at the Royal Society Summer Science Exhibition 2015

A quetzal from one of the temple walls at Teotihuacan, Mexico, photographed in 1982. This serves to represent culture in danger of being forgotten. Contributed anonymously.

CFP Special Issue on VR for Culture and Heritage (Presence MIT Press)

The MIT Press Presence: Teleoperators & Virtual Environments is pleased to announce a highly exciting special issue on VR for Culture and Heritage (published yesterday our website). I’ve included a brief guide on the topics below for your information, an extended summary of scope are in the PDF file attached. The special issue is highly interdisciplinary in nature and therefore, submissions which promote collaboration between the science and engineering and arts and humanities are looked upon favourably. Please help us to disseminate the CFP to your networks. We hope to hear from you and your networks of experts.


A quetzal from one of the temple walls at Teotihuacan, Mexico, photographed in 1982. This serves to represent culture in danger of being forgotten. Contributed anonymously.

A quetzal from one of the temple walls at Teotihuacan, Mexico, photographed in 1982. This serves to represent culture in danger of being forgotten. Contributed anonymously.

Guest Editors
Eugene Ch’ng, NVIDIA Joint-Lab on Mixed Reality, University of Nottingham Ningbo China

Yiyu Cai, School of Mechanical & Aerospace Engineering, Nanyang Technological University, Singapore

Zhigang Pan, Digital Media and HCI Research Center,  Hangzhou Normal University, China

Harold Thwaites, Centre for Research Creation in Digital Media, Sunway University, Malaysia


Scope of the Special Issue
Human culture is profound and is constantly evolving and changing. Heritage, on the other hand, remains perpetually frozen in time, molded or engraved in tangible materials as a testament to past cultures. Yet some intangible heritage lives on, sewn into the intermingled fabric of present cultures. Others are at risk of diminishing as newer generations, enticed by disparate modern cultures, shun the traditions that were handed down to them. This special issue aims to gather researchers working on all aspects of culture and heritage that uses virtual environments or aspects of the mixed reality continuum to conserve, analyze and communicate contents from the past and the present. It therefore encompasses contemporary public facing work in GLAMs (Galleries, Libraries, Archives and Museums), and the backdrop of in-depth investigations to bring to light the contents and contexts of culture and heritage via virtual environments.

This call for papers is a continuation of the previous special issue on Virtual Heritage: Cultural Agents, Environments, and Objects, PRESENCE 24(3), which highlighted the need for closer inspection of our method of work in merging often disparate focuses from multiple disciplines:

“Opportunities in better heritage technologies are due to the digital revolution. Yet, any researchers in the more technical disciplines aiming to make use of digital technologies for heritage must be careful so as not to be overly focused on technology itself, lest the cultural heritage content that technology is meant to convey becomes secondary. Digital technology must support the research, conservation, and communication of cultural heritage, and reciprocally, heritage data used for developing better technologies for supporting heritage research is encouraged.”

The previous special issue also brought to light the state of work and present thoughts on virtual heritage research, particularly on cultural agents, cultural environments, and cultural objects and 3D printing.


Submission Topics
Virtual heritage is a testament to the impact of digital transformation in the arts and humanities, and a driving force for technological innovation generated through the arts and humanities’ increasing appetite for digital technology. In this special issue, we aim to examine present trends in culture and heritage within the context of virtual reality and augmented reality. The scope of the special issue includes the following topics:

  • New approaches in culture and heritage applications and interpretations
  • Responsive, adaptive and evolvable behaviors in immersive virtual environments that capture culture and tangible and intangible heritage
  • Multiuser virtual environments 
  • Mixed reality and the experience of real and virtual environments
  • Presence and phenomenology in culture and heritage applications
  • High definition imaging, stereoscopic displays, interactive cinema
  • Intelligent and High Performance Computing for Virtual Cultural Heritage
  • Ubiquitous computing and new forms of culture and heritage representations via VR and AR
  • VR and AR in Galleries, Libraries, Archives and Museums
  • Interactive Exhibits in Public Spaces
  • Digital Transformations of Museums with Immersive & Interactive Time Machines
  • VR and AR as a narrative
  • Education in culture and heritage via VR and AR
  • Tools, techniques, frameworks and methodologies
  • Virtual environments case studies 

Schedule
• Manuscript submission deadline: March 1, 2017
• Final revisions: September 1, 2017
• Planned publication: PRESENCE 27-1 (Early 2018)

Submissions
Manuscripts should conform to the journal’s submission guidelines:
http://www.mitpressjournals.org/page/sub/pres

Authors, please note that audio and video files can be hosted as supplementary onlinematerial accompanying published articles. For more information about multimedia file formats and submission guidelines, please contact presence@mit.edu.

Contact
Dr. Eugene Ch’ng, Director, NVIDIA Joint-Lab on Mixed Reality, University of Nottingham (China Campus). Email: eugene.chng@nottingham.edu.cn

Further information: http://www.mitpressjournals.org/loi/pres

Tesla K80 Open Workstation

Building our own Deep Learning and VR workstations

I had a bunch of computer science students becoming my summer interns each year where I cultivate their ability to not only do well academically, but also to have real hands-on practical experience in hardware and software integration. This year I adopted 7 student interns as part of the Nvidia joint-lab on Mixed Reality. The students worked on Deep Learning algorithms and Virtual Reality projects. One of the task was to build workstations from scratch, assembling PC components and installing both Windows and Ubuntu and the associated software and libraries that uses GPU acceleration. The picture below is one of the machines that we had, with the Nvidia Tesla K80.

Note: The fans in the photo are temporary, we’ve since built a funnel with a noisy industrial grade fan.

Tesla K80 Open Workstation

Tesla K80 Open Workstation

MoI Representatives from Nvidia and UNNC

UNNC & NVIDIA establish joint-lab on mixed reality (Visualisation and AI)

A Memorandum of Intent between UNNC and US technology giant NVIDIA to establish a Joint-Lab was signed at a ceremony on campus on 27th June.

The MoI and the Joint-Lab, led by Dr Eugene Ch’ng, Associate Professor in Computer Science, is the culmination of collaborative research at UNNC School of Computer Science’s Big Data and Visual Analytics lab. Together with Simon See, Chief Technology Officer of NVIDIA APAC, Dr. Ch’ng’s work has produced a number of important publications and research ideas.
Professor Chris Rudd, Provost of UNNC and Ashok Pandey, Vice-President of NVIDIA and GM of Professional Solution Group & Cloud Computing signed the agreement.
Those representing UNNC at the signing were Uwe Aickelin (Vice Provost forTeaching and Learning), Michael George (Vice Provost for Research and Knowledge Exchange), Edward Wu (Director of Campus Services) and Harry Hua (Director of IT).

Eugene addressing the MoI

Eugene addressing the MoI

NVIDIA was represented by Simon See (Chief Technology Officer of NVIDIA APAC ),Sengmeng Koo (Programme Manager for NTC Singapore), David Yuan (Higher
Education & Research Industry Business Development Director NVIDIA China) and Rita Zhang (Solution Architect & Deep Learning Community Manager, NVIDIA China).

James Lin, Vice President of Shanghai Jiao Tong University’s HPC Centre – who delivered a speech and a lecture at the event – will be the Joint-Lab’s supporting partner in the MoI.

The agreement will provide a number of direct boosts to various research projects at the University. One such project is the €2.5m ERC “Lost Frontiers” Advanced Research Grant, which will use NVIDIA GPU-acceleration for DNA sequencing, complex systems simulation and visualization for reconstructing the land bridge that connected Britain and Europe some 12,000 years ago.

MoI Representatives from Nvidia and UNNC

MoI Representatives from Nvidia and UNNC

Other projects to benefit include crowdsourcing reconstructions of heritage sites and monuments, using big data to understand social networks, and deep learning for real-world applications.

The UNNC-NVIDIA agreement also has numerous benefits for research, knowledge exchange and industrialisation at the University, including talent cultivation and training in Big Data, Machine Learning, Visualisation and VR. One such beneficiary is International Doctoral Innovation Centre PhD student Mengdi Li, who recently started an internship at NVIDIA. Another group of five computer science students are presently taking mentorships in Deep Learning techniques.

UNNC Provost, Prof. Chris Rudd said of the agreement: “The NVIDIA partnership exemplifies how UNNC can exploit industry links not only to solve real-world problems but also to provide a first class training environment for our students. Partnerships like this depend upon people and I am proud that we have such talented and committed researchers who recognise the value of these win-win partnerships and are prepared to put in the hard work and team building that will deliver long term success.”

Simon See, Chief Technology Officer of NVIDIA APAC  said: “Mixed reality and artificial intelligence are starting an upsurge around the world. NVIDIA’s advanced GPU technology and integral software platforms have promoted a great number of innovative technologies and applications in multiple industrial and commercial fields. NVIDIA and UNNC enjoy a shared vision in these areas.”

Dr Eugene Ch’ng, director of the NVIDIA joint-lab on Mixed Reality, sees bright prospects for UNNC research and student cultivation: “NVIDIA’s global lead in GPU-accelerated computing and visualisation will ensure that all our projects are powered by advanced technology for the best research output. With Simon See to co-direct the lab, the bright future for Big Data, AI and mixed reality research is assured”.

Since 1993, NVIDIA has pioneered the art and science of visual computing. With its singular focus, the company offers specialised platforms for the gaming, automotive, data centre and professional visualisation markets. Its products, services and software power new experiences in virtual reality, artificial intelligence and autonomous cars.

Centre Website: NVIDIA Joint-Lab on Mixed Reality (Visualisation and AI)

Original Post: http://www.nottingham.edu.cn/en/news/2016/unnc-nvidia-establish-joint-lab-on-mixed-reality-visualisation-and-ai.aspx

VR with the Oculus Rift DK2

VR in UNESCO 2016 Int’ Congress on ICT in Education

I represented the University of Nottingham Ningbo China at the  very prestigious 2016 International Congress on ICT in Education and Innovative Achievement Exhibition of ICT in Education. The event is held from 22-24 June in Qingdao and is organised by UNESCO, Chinese National Commission for UNESCO, Education Management Information Centre from the MoE PRC, Shandong Provincial Education Department. The event is hosted by the Education Bureau of Qingdao and the Chinese Journal of ICT in Education. Last year, President Xi Jinping sent a congratulatory message to the conference, which was attended by high profile figures such as director-general of UNESCO Irina Bokova, Vice premier of PRC Liu Yan Dong, and other delegates.

IMG_5474 copy

VR demo with the Oculus Rift DK2

 

 

 

 

The purpose of the event was to expand international outlook, further explore the innovative model integrating ICT and education, and build a platform for cooperation and exchange in terms of ICT in education through showcasing global leading information technology and latest achievements and exemplary cases of ICT in education.

IMG_1269 copy

University of Nottingham Ningbo were 1 out of 9 selected universities to represent the event. The university considers this a highly important platform from which to showcase our world-class teaching, learning, and research-led teaching not only to high-profile visitors from within China, but also to international delegates, and via the media (CCTV, etc).

Together with my trusted colleagues, we showcased VR demos and pilot applications using our Oculus Rift DK2 and Samsung Gear VR with S6 Edge. The event was well attended and our booth seems to attract a lot of attention!

Chinas Ministry of Education Delegates Trying out our VR Demos!

Chinas Ministry of Education Delegates Trying out our VR Demos!

RS-Sim09

Special Issue on Virtual Heritage: Cultural Agents, Environments, and Objects

Special Issue on Virtual Heritage: Cultural Agents, Environments, and Objects is published!

Presence: Teleoperators and Virtual Environments, MIT PRESS

Guest Editor’s Introduction: http://www.mitpressjournals.org/doi/abs/10.1162/PRES_e_00228#.V56OfpN940o

RS-Sim09

Summary

Virtual heritage – the use of digital technology and virtual environments for researching, conserving and conveying our cultural heritage, offers exciting new ways to learn and experience the cultural treasures of the world, both past and present.

Virtual heritage has its origin at the formation of the Virtual Systems and Multimedia (VSMM) Society conference at Gifu, Japan in 1995, and can be attributed to many of the champions (Addison, 2000; Refsland, Ojika, Addison, & Stone, 2000; Stone & Ojika, 2000) of this particular strand of work. Since then, the heritage community has witnessed considerable investment from various institutions for heritage works that involve or are related to the use of digital technology, particularly with one of the larger funding bodies in Europe. Two notable projects totaling 10 million euros that has garnered wide media attention for their discoveries, including being selected as exhibits for the 2012 and 2015 Royal Society Summer Science Exhibition, bear witness to the importance of digital- and technology-oriented heritage projects. These two complex archaeological heritage projects with large spatial-temporal scales – Europe’s Lost World and the Stonehenge Hidden Landscapes project are pushing technological boundaries beyond what contemporary techniques can afford (see Ch’ng et al., 2011; Gaffney, Fitch, & Smith, 2009; Gaffney, Thomson, & Fitch, 2007). These are specimen works that has demonstrated that heritage could contribute to pioneering geophysics instrumentations, digital and computational approaches. Many other funded projects worldwide involving cutting-edge technologies overseen by a consortium of academic and heritage institutions are pioneers to the many digital heritage innovations today, the list of which is too numerous to be listed here. The support for heritage research has important meaning, as stated in the European Commission’s website: “Cultural heritage enriches the individual lives of citizens, is a driving force for the cultural and creative sectors, and plays a role in creating and enhancing Europe’s social capital. It is also an important resource for economic growth, employment and social cohesion, offering the potential to revitalise urban and rural areas and promote sustainable tourism.” (EC, 2015)… [continue reading]

 

Social Information Landscapes of the Boston Bombing Fans

Fans of Boston Bombing Terrorist Tracked Online Via Big Data Research

The interest of this research lies in the controversial online teen #FreeJahar movement calling for the freedom of the Boston bombing suspect Dzhokhar Tsarnaev because teenage girls believe he is “too beautiful to be a terrorist” (Nelson, 2013). Concerns were raised that in on-line forums the younger Boston Bombing suspect appeared to attract a cult teen following expressing affection and concern for him (DailyMail, 2013). Facebook, Tumblr tribute accounts were set up in support of the teen with the #FreeJahar Twitter tag, which were trending when these activities first appeared. Teen activities in Twitter do need to be monitored (Wiederhold, 2012) as observed below:

  • “I can’t be the only one who finds the suspected bomber to be sexy, can I?” 19 April 2013.
  • “i don’t even care if jahar is a terrorist he’s cute i don’t want him to die.” @******, 20 April 2013.
  • “I’m not gonna lie, the second bombing suspect, Dzhokhar Tsarnaev, is hot. #sorrynotsorry” @******, 20 April 2013.
  • “Yes I like Justin Bieber and I like Jahar but that has nothing to do with why i support him. I know hes innocent, he is far too beautiful” @******, 25 April 2013.
  • “How many RT’s for our boy jahar look at that beautiful face #freejahar pic.twitter.com/K9xKFvv5HT” @******, 4 May 2013.
  • “Getting one of Jahar’s tweets tattooed on me tomorrow. Guess you could say I’m a #FreeJahar supporter,” @******, 7 May 2013.

I started building scalable algorithms on scalable hardware-software platforms which I developed in 2013 due to the interest above. The research is now published.

Social Information Landscapes of the Boston Bombing Fans

Social Information Landscapes of the Boston Bombing Fans

Abstract of Paper “The Bottom-Up Formation and Maintenance of a Twitter Community: Analysis of the #FreeJahar Twitter Community” [Link to Article]

Purpose – The article explores the formation, maintenance and disintegration of a fringe Twitter community in order to understand if offline community structure applies to online communities

Design/methodology/approach – The research adopted Big Data methodological approaches in tracking user-generated contents over a series of months and mapped online Twitter interactions as a multimodal, longitudinal ‘social information landscape’. Centrality measures were employed to gauge the importance of particular user nodes within the complete network and time-series analysis were used to track ego centralities in order to see if this particular online communities were maintained by specific egos.

Findings – The case study shows that communities with distinct boundaries and memberships can form and exist within Twitter’s limited user content and sequential policies, which unlike other social media services, do not support formal groups, demonstrating the resilience of desperate online users when their ideology overcome social media limitations. Analysis in this article using social networks approaches also reveals that communities are formed and maintained from the bottom-up.

Research limitations/implications – The research data is based on a particular dataset which occurred within a specific time and space. However, due to the rapid, polarising group behaviour, growth, disintegration and decline of the online community, the dataset presents a ‘laboratory’ case from which many other online community can be compared with. It is highly possible that the case can be generalised to a broader range of communities and from which online community theories can be proved/disproved.

Practical implications – The article showed that particular group of egos with high activities, if removed, could entirely break the cohesiveness of the community. Conversely, strengthening such egos will reinforce the community strength. The questions mooted within the paper and the methodology outlined can potentially be applied in a variety of social science research areas. The contribution to the understanding of a complex social and political arena, as outlined in the paper, is a key example of such an application within an increasingly strategic research area ­ and this will surely be applied and developed further by the computer science and security community.

Originality/value – The majority of researches that cover these domains have not focused on communities that are multimodal and longitudinal. This is mainly due to the challenges associated with the collection and analysis of continuous datasets that have high volume and velocity. Such datasets are therefore unexploited with regards to cyber-community research.

Keywords – Social network analysis, big data, twitter, online communities, social media, multimodal network, longitudinal network, freejahar, centrality measure

Four other research papers describes the process:

  1. Ch’ng E. (2015) The Bottom-Up Formation and Maintenance of a Twitter Community: Analysis of the #FreeJahar Twitter Community, Industrial Management & Data Systems 115(4), p.612-624. [Link to Article]
  2. Ch’ng E. (2015) Social Information Landscapes: Automated Mapping of Large Multimodal, Longitudinal Social Networks, Industrial Management & Data Systems, Vol. 115 Iss: 9, pp.1724 – 1751 [Link to Article]
  3. Ch’ng E. (2015) Local Interactions and the Emergence and Maintenance of a Twitter Small-World Network, Social Networking 4(2), p.33-40. [Link to Article]
  4. E. Ch’ng (2014) The Value of Using Big Data Technology in Computational Social Science, The 3rd ASE Big Data Science Conference, 4-7 August 2014, Tsinghua University Beijing China. [Link to Article]

 

Royal Society Summer Science Exhibition 2015

Royal Society Summer Science Exhibition 2015: Stonehenge Hidden Landscapes


 

I was invited for the second time to participate at the Royal Society Summer Science Exhibition 2015 held in London 30 June – 5 July. Our exhibit featured the Stonehenge Hidden Landscapes project, with the discovery of 17 monuments using remote sensing methods and VR visualisation. I developed a multitouch table system for acquiring human behaviour, described in an invited article “Crowd Behaviour Mining with Virtual Environments” published in PRESENCE: Teleoperators & Virtual Environments [pdf].

Royal Society Summer Science Exhibition 2015

Royal Society Summer Science Exhibition 2015

Reference Article: Ch’ng, E. (2015) Crowd Behaviour Mining with Virtual Environments, Invited Article, PRESENCE 24(4), p347–358.

Link: Royal Society Summer Science Exhibition 2015

Visual Heritage in the Digital Age, Springer Series on Cultural Computing

Visual Heritage in the Digital Age Springer

Visual Heritage in the Digital Age

My edited book titled “Visual Heritage in the Digital Age” recently got published in the Springer Series on Cultural Computing. The articles presented in the volume relates to research within the domains of digital heritage and computational archaeology. [Link to Publisher Page]

Keywords » Computational Archaeology – Cultural Technologies – Digital Heritage – Heritage Computing – Modelling Landscape Change – Visualisation of Space and Movement

Heritage is everywhere, and an understanding of our past is increasingly critical to the understanding of our contemporary cultural context and place in global society. Visual Heritage in the Digital Age presents the state-of-the-art in the application of digital technologies to heritage studies, with the chapters collectively demonstrating the ways in which current developments are liberating the study, conservation and management of the past. Digital approaches to heritage have developed significantly over recent decades in terms of both the quantity and range of applications. However, rather than merely improving and enriching the ways in which we understand and engage with the past, this technology is enabling us to do this in entirely new ways.

The chapters contained within this volume present a broad range of technologies for capturing data (such as high-definition laser scanning survey and geophysical survey), modelling (including GIS, data fusion, agent-based modelling), and engaging with heritage through novel digital interfaces (mobile technologies and the use of multi-touch interfaces in public spaces). The case studies presented include sites, landscapes and buildings from across Europe, North and Central America, and collections relating to the ancient civilisations of the Middle East and North Africa. The chronological span is immense, extending from the end of the last ice age through to the twentieth century. These case studies reveal new ways of approaching heritage using digital tools, whether from the perspective of interrogating historical textual data, or through the applications of complexity theory and the modelling of agents and behaviours. Beyond the data itself, Visual Heritage in the Digital Age also presents fresh ways of thinking about digital heritage. It explores more theoretical perspectives concerning the role of digital data and the challenges that are presented in terms of its management and preservation.