[ad_1]
They say that beauty is in the eye of the beholder, but the eye also contains a reflection of that which has the beholder so enamored. Researchers from the University of Maryland have devised a way to extract those reflections from a photo and render it in 3D. The result is a sometimes accurate model of whatever happens to be in front of the subject.
The human eye is not flat, which naturally presents a problem when training algorithms to understand these reflections. We are lucky, however, that corneas are pretty consistent across the board. The physical geometry and texture of corneas (the eye’s transparent top layer) are almost the same for all healthy adults. With cornea pose optimization and iris texture decomposition, the researchers created a model that can compensate for blurry, low-resolution reflections. They essentially align the 3D shapes so they continue to resemble the original article from more angles than the one in the photograph.
With the algorithm in hand, the team created 3D models from various eye reflections. You can see some examples below, and they can be pretty impressive. Scenes like a collection of toys are rendered with surprising clarity—Kirby still looks like Kirby from multiple angles, even though the source data is a single still image. Some past attempts to do the same with eye reflections required high-resolution video with multiple reflection states.
The examples in the video above are the best the team has to offer, but they’re also designed to be that way. These are images captured in a laboratory setting where the team controls the lighting, reflections, and camera angle. The team also used a few synthetic eyeballs, which provided even clearer images. The project’s website, which will eventually host the rendering source code, includes some “in the wild” attempts using high-resolution portrait photos of Lady Gaga and Miley Cyrus. The researchers managed to extract some reflections, but they don’t look like much of anything. The team speculates that there might be a person’s torso reflected in one photo, but most professional portraits are shot in such a way as to minimize reflections.
The researchers don’t get into potential applications, but there’s ample speculation that such a technique could reveal evidence in criminal investigations. Most crimes don’t involve high-resolution photos of a person’s eye, but you never know. The researchers speak more generally about exploring the unseen information in the media we create. Of course, that same urge to explore could lead to some Black Mirror-style privacy invasions. The study is published on the arXiv pre-print server while it undergoes peer review.
[ad_2]
Source link
