The Null Device

Diminished and/or augmented reality

A team in Germany has developed software which can edit objects out of live video in real time. Termed, catchily, "Diminished Reality", the software works a bit like Photoshop's content-aware fill, but is able to track, and eliminate, objects in moving video. The team from the Technische Universität Ilmenau are planning to release an Android port, so you too can be Stalin.

Perhaps even more interesting is software from the Max Planck Institute which can alter the body shapes of actors in video. The software contains data obtained from 3D scans of 120 naked people of different body types, apparently using a machine-learning algorithm, to form a 3D body model with a number of controllable attributes, such as height, muscularity and waist girth. The system can pick out human figures in video (in some conditions, anyway), map them to the model, adjust it, and then rerender the video with the adjusted model. The team have demonstrated this with a clip from the old TV series Baywatch, in which the male lead is given Conan The Barbarian-style musculature.

The article gives a number of potential applications for such technologies:

The technology has obvious applications in films like Raging Bull, for which Robert de Niro put on 27 kilograms in two months to portray his character. "The actor wouldn't need to go to all that trouble," says Theobalt. It could also be a cost-saver for advertising companies. Because standards of beauty vary across cultures, it is the norm to shoot several adverts for a single product. With the new software, firms could make one film and tweak the model's dimensions to suit different countries.
The possibilities don't, of course, stop there. In the market-driven entertainment ecosystem, film and TV companies are competing for the attention (and money and/or eyeballs to sell to advertisers) of a public, a large segment of which is captivated by spectacle. With improved special-effects technology comes "awesomeness inflation", where yesterday's blockbusters look boring compared to the latest; so anything that can capture the eyeballs of the sensation-hungry, compulsively channel-surfing consumer (whom William Gibson memorably described as "something the size of a baby hippo, the color of a week-old boiled potato, that lives by itself, in the dark, in a double-wide on the outskirts of Topeka. It's covered with eyes and it sweats constantly. The sweat runs into those eyes and makes them sting. It has no mouth... no genitals, and can only express its mute extremes of murderous rage and infantile desire by changing the channels on a universal remote") could give a film studio or TV network the edge; that extra average five seconds before the viewer changes the channel which, aggregated over an audience of hundreds of millions, means a lot of ad revenue.

It's perhaps obvious that film studios will use the software as another computer effect, making their actors more cartoonishly exaggerated, more punchily extreme, with taller, more ruggedly muscular action heroes, more exaggerated comic short/fat/skinny guys, leading ladies/love interests whose waists could not physically support their breasts, and so on. Eventually the public will get used to this, and the old films with realistically physiqued (by Hollywood standards) actors will look as shabbily unattractive as those films from the 70s they're always remaking because the pace's too slow, the scenes look crappy (didn't the ancients even know about orange and teal colour grading?) and there aren't enough awesome explosions and sex scenes. If the software's cheap enough (as it will eventually be), though, they won't even need to remake things: imagine, for example, a channel that shows reruns of popular old series, "digitally remastered for extra awesomeness". And so, every year, the stars in yesteryear's classic serials become that bit more like animated action figures and/or anime schoolgirls, culminating in a 8-foot, musclebound Jack Bauer who can shoot laser beams from his eyes. (The remastering process would also quicken the pace, by speeding up scenes and cutting out pauses, which would both hold the audience's attention for longer and leave more time for ad breaks.) Meanwhile, Criterion sell box sets of the original, unretouched versions in tasteful packaging; these become a highbrow affectation, a signifier of refined taste, and end up featured on Stuff White People Like.

Of course, in this universe, there'd be an epidemic of body-image disorders, with large numbers of deaths from anorexia, steroid overdoses and black-market plastic surgery. At least until physique augmentation ends up as a universal feature of compact cameras and/or Facebook uploading software, and gradually the survivors come to accept that it's OK to look imperfect, as long as you don't do so on film or video.

There are no comments yet on "Diminished and/or augmented reality"