I am looking forward to starting two new research projects in the spring.
The first is my VR funded 4 year Starter Grant Designing New Speech Interfaces with Ambient Audio.
While microphone equipped devices such as mobile phones are surrounded by spontaneous person-to-person talk and ambient environment audio, this is seldom drawn upon to guide systems. While this audio is much less constrained than dialogic speech directed at a device, and as a consequence, extremely challenging to recognise and model, it offers a rich potential resource for system input and human computer interaction that has been almost entirely neglected. Indeed, while speech research has been an active area of computer research, human-computer interaction research on speech has been given much less focus – potentially limiting the opportunities for applications of speech, but also for research understandings of how speech systems can fit with user activity and system use. Accordingly, this project is focused around understanding non-system directed audio (such as ordinary conversation and environmental, ambient audio) and designing for the impact it could have on system behaviour and user interfaces. We hope to open up new opportunities for human computer interaction using audio detection and processing.
With the growth in ubiquitous- and IoT-based systems there is now the opportunity to make significant improvements in how technology benefits everyday life. Yet existing systems are beset with manifest human interaction problems. Each individual system has been designed with a particular, limited, interaction model: the smart lighting system in your apartment has not been designed for the sharing economy, the lawn mower robot might run off and leave your garden. Different parts of your entertainment system turn the volume up and down and fail to work together. Each smart object comes with its own form of interaction, its own mobile app, its own upgrade requirements and its own manner of calling for users’ attention. Interaction models have been inherited from the desktop-metaphor, and sometimes mobile interaction have their own apps that use non-standardised, icons, sounds or notification frameworks. When put together, the current forms of smart technology do not blend, they cannot interface one-another, and most importantly, as end-users we have to learn how to interact with them each time, one by one.
This project is built around developing a new interface paradigm that we call smart implicit interaction. Implicit interactions stay in the background, thriving on data analysis of speech, movements, and other contextual data, avoiding unnecessarily disturbing us or grabbing our attention. When we turn to them, depending on context and functionality, they either shift into an explicit interaction – engaging us in a classical interaction dialogue (but starting from analysis of the context at hand) – or they continue to engage us implicitly using entirely different modalities that do not require an explicit dialogue – that is through the ways we move or engage in other tasks, the smart objects responds to us.
My current project, running until March 2017 at the Mobile Life Centre is Reality Mining
This project looks at the amount of data that can be collected, and is being collected, about individuals and groups. We focus on the implication that as this data collection increase so does the threat of abuse, and the need for control. Looking at the upcoming range of Internet of Things and Personal Informatics sensors we will endeavour to understand how the data can be used, what impact that will have on individuals and society, and what control systems – in terms of HCI and regulation – are necessary to ensure integrity, autonomy, and innovation.
Previous Research Projects
Homes & Cities.
As the location for much of our life, cities and homes are exciting settings to conduct research in. While in the home we have control and the needs of family, in the city we must regulate our interactions with and through strangers, commercial enterprises, and government authorities. How can we build technology to bridge both of these spaces? What role does mobile technology play in changing the technical landscape for design in the home and in the city?
Both settings meet in so called ‘liminal spaces’ – spaces that are between the city and the home. We are thinking of places such as playgrounds, gardens, or “tvättstuga” (shared laundry rooms), but also parts of the home that become liminal, the political poster in a window or the Christmas wreath on a door. Since the home has been a clear focus of much existing research, it is wise to focus this project on the city and the connections between the city and the home. Building on last years workshop and commentary on visions of the ‘smart home’ and the ‘smart city’, in this year of the project we will conduct two focused interventions, one study and one technology intervention.
One arm will focus on liminal home/city spaces, and the other will take a broader vision approach to understanding the role of technology in the city more broadly. The first arm is based around focused studies which will inform a broader vision of technology in the city. Our goal is to inform the vision of a technological city with empirical work that will examine the current role of technology in our city streets. Through mobile life extensive connections we hope to produce a study of the existing state of technology in the city – how traffic is monitored, how environmental data is shared, and the ways in which citizens themselves are engaging – or not – with their own city. We want to move away though from a data centric view of city life to one that understands that technology is mostly experienced in the city in its primary physical form. This will also necessitate an engagement with processes of change in the city, in particular town planning, public space architecture and the role of codes and laws in shaping public space.
A second arm is to experiment with technology in shared liminal spaces. We will conduct a set of design experiments in these spaces to explore how we might push the limits of what technology can do in shared spaces, and what demands it can make on the people who pass through and in part control these collective environments.
I am also working on the Virtual Communities part of the The Network of Excellence in Internet Science EU project.
The Internet and other areas of ICT have enabled distance relationships to form – from social network sites to large-scale virtual worlds in which people socialize – and have enabled the formation of virtual communities for workplace collaboration, distance education, medical services, and politics, among others. Ultimately, the integration of online social networks with tools for distributed creation of knowledge, interfaced to the real world through network of sensors and rooted in real communities, is expected to create an extended collective awareness which will be the basis for innovative and sustainable actions, at individual and collective levels, in multiple fields (environmental, social, political, etc.), as captured by the recent EU initiative on Platforms for Collective Awareness and Action. This working group aims to distil knowledge and best practices from such efforts and to develop social design methodologies for development and experimentation within virtual communities – including user needs analysis and the impact on technological design choices affecting the future Internet (such as open standards, Quality of Service) – taking into account socio-economic, security and privacy concerns. Outreach to the wider social science community examining virtual communities will include exchange with leading scholars in such conferences as State of Play. Ultimately, Internet Science itself will become both an offline and online virtual community of ICT and social science scholars exchanging best practice (for example through the dissemination and exchange activities that will complement this working group). The group’s partners are inter-disciplinary institutions based on the social sciences, from law (UESSEX), socio-economic analysis (IBBT), social science (UNIBO, LSE), anthropology (OXF – Dunbar), and communications studies (TUD, LSE). They are complemented by CNR, TCLR, ULANC, UoA and MLS.
The Clouds & Surfaces project has ended, although work related to the finding and the data collected is ongoing.
The concept of surfaces as the form of computing encourages us to think about computer use as something that is not simply mobile in the sense of portable, but in the sense of being inter-threaded with movement of different sorts, such as gesture, exchange or being shared. So, for example, when we go and visit a colleagues office, we might be physically stationary during the time spent meeting, but the need to move to meet has a number of implications with respect to our access to information and computing. In the interactive situation itself we might want to occlude certain parts of information and share others – or we might want to pass over information itself in a form that can be immediately shared and interacted around (such as in giving someone a paper document). These are all aspects where computing in the form of portable surfaces fits much better than information on fixed displays. In terms of research around surfaces the opportunity is in understanding the range of situations where computing can be brought into interaction or mobility in ways in which it would have previously been too disruptive.
How can surface computing support new interactional situations of use? How do enjoyable activities change as they are instantiated in different and new forms? What is the emotional and practical relationship with managing objects that are now digital? How does the move away from location-based storage and computation influence mobile computing? One study we are currently undertaking involves video recording of surface computing use to understand how surface computing is interweaved with interaction. A second study explores how cloud computing has changed attitudes and the use of ‘files’ and online document services.