Nature is complex – often too complex for humans to see. But squint-controlled glasses that let people see 3D thermal images and a camera that can capture the inner workings of high-speed chemical reactions are helping to push the limits of human perception.
Human senses have already been highly tuned by millions of years of evolution. Our eyes, for example, sit on the front of our faces, allowing us to see in three dimensions, while the cells of our retina are sensitive to different wavelengths of light, giving us colour vision.
But our senses have limits. A speeding bullet, for example, travels too fast for the human eye to see.
‘There are a lot of things we have a hard time perceiving with the senses we have,’ said Professor Albrecht Schmidt, a computer scientist at the Ludwig Maximilian University of Munich, Germany and leader of a project called AMPLIFY.
‘Our project is focused on finding ways of extending the traditional senses to enhance human perception with digital technologies.’
Cameras, for example, can capture light that is beyond the visible spectrum and reveal movements that are too fast for the human eye.
‘There are a lot of things we have a hard time perceiving with the senses we have.’
Professor Albrecht Schmidt, Ludwig Maximilian University of Munich, Germany
‘The question is how do we make them intuitive to use,’ said Prof. Schmidt. A lot of people focus on creating implants, he says, but the issue is that they cannot be taken off.
‘If you have something integrated into a pair of glasses, which I wear without consciously realising most of the time, we could amplify the senses only when needed.’
Prototypes
Prof. Schmidt, together with colleagues at the University of Stuttgart, has built a number of prototypes to do just this.
One is a pair of glasses that triggers a thermal image when the wearer squints their eyes. A camera incorporated into the frame produces images using visible light, near infrared and far infrared. These give the wearer the ability to see normal visible scenes, but also three-dimensional thermal images.
Sensors capable of detecting the faint electrical signals produced by muscles are also built into the frames to pick up the movements associated with the eyes narrowing or squinting.
The team have created a version that can be built into the helmets worn by fire-fighters, giving them the ability to spot hidden fires or find people who are trapped inside buildings.
‘It’s very different from using a thermal camera as you have to actively look away to see what is going on,’ said Prof. Schmidt. ‘This way you can change the way you see very easily, even subconsciously.’
He envisages a similar device that might be able to zoom into a far-off scene or slow down the action.
‘If I look out of the window I can see some trees, but I could squint my eyes to zoom in and I would see a bird on a branch,’ explained Prof. Schmidt. ‘If the bird takes off, my cognitive load might go up, which could be measured from my brain activity, and that would tell the camera to slow down the motion.
‘Then when I look back to my computer in from of me, it could return to normal. All of this happens without me having to think much about it and it is this control we have been focusing on.’
It is the team’s overall aim to try to create devices that can be either immediately intuitive to use or can very quickly become second nature.
Another prototype is a pair of swimming goggles that help the wearer orientate themselves while their head is submerged under water. Open water swimmers in particular can have difficulty swimming in the right direction as the lack of visual information can disorientate them.
But by fitting a pair of goggles with an accelerometer and magnetometer, the team were able to provide navigational cues using LED lights in the peripheral vision.
They have also been testing another pair of glasses that gives the wearer eyes in the back of their head by using cameras to give a 360-degree view. By displaying images in the peripheral vision, it allows the wearer to spot potential hazards, such as when crossing the road, and so turn to look at them.
But even when armed with the latest in camera technology, there are also some things that will escape our eyes. Chemical reactions, for example, happen in time scales that are too fast to capture with modern high-speed cameras – they can be over in just a couple of trillionths (a millionth of a millionth) of a second.
Dr Maria Ana Cataluna, a physicist at the Institute of Photonics and Quantum Sciences at Heriot-Watt University, Edinburgh, UK, is leading a project aimed at overcoming this. The UPTIME project is attempting to build the fastest camera on the planet.
By exploiting new ultrafast lasers that send a flash of light lasting just between 10-100 femtoseconds (million billionths of a second) and sampling systems that can rapidly capture the photons reflected, they hope to be able to capture events that were previously impossible for us to see.
Capture
‘If we can’t visualise it, we can’t further understand it,’ said Dr Cataluna. ‘This means that the inner machinery of irreversible process widely present in physics, biology and engineering remain, in essence, unobservable.’
Among the events she hopes to be able to capture with the new camera are high speed biochemical reactions, to help reveal their inner workings. It could also allow scientists to capture phase changes, such as the ethereal moment when a liquid turns into a gas. And it could also help better understand what happens to a material when it is transformed using laser-based manufacturing processes. This new knowledge could be used to help optimise the processes.
‘A short burst of light lasting for only one microsecond enables a fast camera to capture a bullet in mid-flight,’ explained Dr Cataluna.
‘In a timescale more than six orders of magnitude below this, optical pulses with sub-picosecond or femtosecond durations enable the capture of microscopic ultrafast phenomena. We want to be able to take snapshots of these.’
While she is reluctant to say too much about how the technology would work until she has filed a patent for it, she is currently developing the core components that will be necessary to build the camera. She hopes, however, that the new ultrafast camera will become a reality within three years.
As new technology gives us the ability to perceive in ways that have never been possible before, Prof. Schmidt urges caution. He believes that once we have these devices, the digital and social divides will widen.
‘We will get into a space where digital technology will create many more superpowers, and those who have it will have a great advantage.
‘We will need to be careful to ensure those who cannot afford the technology are not left behind.’
The research in this article was funded by the European Research Council. If you liked this article, please consider sharing it on social media.
Originally published on Horizon