Face Morphing: Why It Can Threaten National Security & How to Protect Against It

Facial morphing is an image synthesis task. Say there are portraits of two individuals. Morphing can fully transform one face into another, but if you stop the process midway, you’ll get a morph—an image of a non-existent person who resembles both people at the same time.

We can most often see morphing technology in action as a special effect in big-screen movies. As of late, there are also plenty of face morpher apps available on the web that allow you to play with photographs and morph two faces. For instance, you can easily check out what a mashup of your favorite celebrities would look like.

In a nutshell, the face morphing sequence consists of three main steps:

• landmark identification; 

• geometry warping;

• texture and color blending. 

Landmarks are key points on the face that will serve as a coordinate system. They usually correspond to prominent facial features, such as mouth, nose, eyes, and chin, and serve to identify their location and outline their shape.

During warping, we influence the shape of the face and its geometry. By positioning the key points to the average of the corresponding key points in both photos, we can get a midway face. 

Blending fills in the color and texture of the midway face. Quite often, the details, such as wrinkles, pigmentation, and pores are smoothed in the process, so the resulting image gives the impression that it’s under some Instagram filter. But it’s not always the case. There are also post-processing methods that address quality degeneration caused by blending and remove possible morphing artifacts. Thus, morphed faces can have characteristics similar to genuine images.

Yes, facial morphing can be a threat, the prevention of which requires deliberate and well-thought-out actions. At Regula, we first encountered this type of identity fraud back in 2011 and have witnessed the evolution of the method.

One of the main application areas of facial morphing for criminal purposes is forging identity documents. The attack targets face-based identity verification systems and procedures. Most often it involves passports; however, any ID document with a photo can be compromised.

One well-known case happened in 2018 when a group of activists merged together a photo of Federica Mogherini, the High Representative of the European Union for Foreign Affairs and Security Policy, and a member of their group. Using this morphed photo, they managed to obtain an authentic German passport.

If there’s a morphed photo in the ID document, then two (or even more) individuals can share it. In this case, there are all sorts of consequences that arise related to fake identity documents: from fraudulent financial operations to illegal migration and border crossings by criminals and terrorists who threaten national security. An attempt to exploit a morphed image to fool border control staff is the most common use case.

Prof. Dr. Christoph Busch, in this presentation, states that there have been over 1,000 reported cases of face morphing attacks. In fact, the scale of the problem may be more serious, as visual editors and printing devices have been developing, so methods to use face morphing for cheating identity verification systems have never been as accessible as today.

Worse still, documents with morphs are already in circulation. Even if all the possible security measures are instantly implemented, we must beware that passports potentially containing a morphed image will still be in circulation for the next 10 years.

The main challenge with facial morphing is effectively detecting it. Unfortunately, there are no solutions yet that can reliably detect morphing during the application process or spot morphed photos in existing documents. There are workarounds, of course. For example, you can use forensic devices which can help you detect morphing by indirect signs, such as traces of alterations in a document. But there are no ways to make 100% sure if it’s a morph or not in the photo.

Also, note that morphs may differ in quality and production methods used. While low-quality auto-generated images may include visible artifacts, such as shadows or ghosting silhouettes, a real attack by a criminal would likely involve their best efforts at selecting lookalike individuals and creating a high-quality morph with careful post-processing. Such morphs look like real people and are able to deceive examiners.

The experiments conducted in 2019 to investigate human performance with high-quality morphs comparable to those that can be used by criminals, proved that people were highly error-prone. Not only was their performance at morph detection poor, but specialized training didn’t make an improvement: the accuracy during training was at chance levels (51%).

Advances in visual editing along with criminals’ determination to avoid being caught, have led to sophisticated morphs undetectable to the human eye. Criminals also rely on the human factor at border controls: a large flow of people, insufficient lighting, and mere negligence. In such environments, the chances are that an officer might dismiss slight dissimilarities as due to age changes or a new hairstyle.

Computer algorithms, however, still aren’t a silver bullet for detecting morphs. The National Institute of Standards and Technology (NIST) runs ongoing algorithm assessments to evaluate algorithmic capabilities to detect face morphing and face recognition algorithms’ resistance against morphing. For testing purposes, they have three datasets of morphed images:

• Tier 1: Low-quality morphs generated with available free tools

• Tier 2: Automated morphs created by academic algorithms with best practices implemented

• Tier 3: High-quality morphs that involve manual processing and post-processing

There are two metrics used for evaluating results:

• Attack Presentation Classification Error Rate (APCER) — Shows the proportion of morphed images incorrectly classified as non-morphs (bona fide)

• Bona Fide Presentation Classification Error Rate (BPCER) — The proportion of genuine images incorrectly classified as morphs. 

In the ideal situation, both APCER and BPCER results should be low.

Currently, identifying low-quality morphs is relatively successful: the best algorithm performed with 0.156 APCER results. That means only 15 synthesized photos out of 100 will be overlooked by the algorithm. However, the extrapolation of this encouraging result to real-life situations is disturbing. For example, if we have eGates that operate using this winning algorithm, then 15 out of 100 individuals with morphed photos in their IDs will pass the automatic airport screening.

As for high-quality morphs, the current error rate of the best-performing algorithm may reach as high as 0.880, meaning that 88% of synthesized images will pass the check as genuine.

There are two possible scenarios of how morphed photographs get into ID documents.

Difficulty to carry out: This machination is only possible in countries where submitting a photo for an ID is the applicant’s responsibility. It can be carried out during remote application submission, or when individuals bring in their printed photos.

Difficulty to detect the forgery: From hard to impossible.

How it works: This is the worst-case scenario, where a morphed photo gets into an ID at the application step.

The scheme requires a criminal and a “clean” citizen with no criminal record as an accomplice, since the submission is usually made through e-government. The criminal exploits the passport and personal data of the accomplice, but submits a morphed photograph that resembles both individuals.

If the morphed photo is accepted, all individuals that contributed to the morphed face can be verified against it. Worse still, not only is there a morphed image in the visual inspection zone, but the same image is also encoded in the chip. This way an imposter who is, for example, forbidden to leave the country or wanted by the authorities gets a de facto genuine passport and can freely cross the border.

Difficulty to carry out: This method is simple but the result depends on the skills of the counterfeiter.

Difficulty to detect the forgery: Easy to detect when special technical means are used.

How it works: The forgery requires a genuine document that can be stolen, lost, or obtained through an illegal deal. Once they possess the photograph in the document, the counterfeiters then morph it with another individual’s photo.

Using a special printer, the morphed image is applied over the existing photo on the personal data page. They may overlay the face entirely, or alter just some facial features, for example, the ears or eyes. Since the alterations are printed in one thin layer, the changes are hardly noticeable to the naked eye or to the touch.

Unlike traditional counterfeiting, there’s no need to damage the security features of the document, for example, by peeling off the laminate, erasing data, adding new data, then either gluing it back or completely replacing the page. Since all these manipulations destroy a number of security features, it’s quite easy to detect the signs of change. But when a morphed image is printed over the top, most security features are intact, so it’s harder to detect the fake.

The way a morphed image gets into an identity document determines the methods needed for effective morph detection and how difficult it is to do so in general. We won’t dive deep into the morphing attack detection (MAD) methodology (although we encourage you to have a look at this extensive survey); rather, we will provide a high-level understanding of what can be done to handle the threat.

As for FOGs, where a morphed photo is accepted at the application step, they have no signs of counterfeiting, so they can’t be identified by criminalistic means alone. The main method of dealing with this threat would be preventing fraudulent applications for the issuance of documents.

One of the most secure and widespread methods is to allow in-person document submission only, where all the biometrics are captured on the spot. A person who’d like to get a new passport must come, pass a preliminary identity check, and sit in front of the camera for a shot that will be used in their ID. 

At the same time, we cannot ignore the ongoing trend toward remote services, which has been largely accelerated during the pandemic. Despite the fact that live enrollment is a more proven and reliable solution today, online enrollment is still possible. 

In Finland, for example, photo studios send photographs for documents directly to the police. The UK chose another path: its Home Office has been running trials of biometric technologies for remote self-enrollment apps and kiosks.

In particular, it requires the implementation of a powerful solution to handle the digital identity verification process, such as Regula Document Reader SDK and Regula Face SDK. These solutions must be able to reliably read an existing document, including the information encoded in its chip, evaluate the quality of submitted images, perform liveness checks, and match a portrait to another reference photo (a selfie, a portrait from a chip, an external database, etc.). This way, the remotely submitted photo is authenticated through a number of checks.

As for detecting manipulations to already issued documents, there’s nothing better than forensic methods, because it’s easier to identify the signs of morphing rather than morphing itself. 

As a rule, ID documents have a polycarbonate or laminated data page. If it’s laminated, then all the data on the page is inside. If it’s plastic, the laser beam goes through the top layers and the image also appears inside. In both cases, there’s nothing printed on the surface. The page is homogeneously smooth: glossy or matte.

If there are morphed elements printed on top, there’s a matte (or, on the contrary, glossy) spot in that area where the overprint was made. Examination in different light sources, e.g., infrared, may also show different pass-through rates of inks, doubling of elements, areas where luminescence is blocked, or blind embossing. Some micro-relief will also appear.

If you also check the RFID chip, there will be a different portrait, so criminals usually disable it. If it can’t be read, that is a reason for a closer inspection. In fact, there are plenty of signs, but you need to know about this type of counterfeiting and how to look for it.

Without technical means, however, all of those signs are extremely difficult to detect. The large flow of people and limited time to work with each individual also play into the hands of criminals. Using forensic devices solves both problems: they make it possible to instantly detect signs of counterfeiting, while also making it easier to deal with a heavy traffic flow. 

The main requirement for forensic devices to detect morphing attempts is having a high-quality oblique and sliding light source (with a very sharp angle). You need to be explicit about it when negotiating with vendors, as not all of them offer this feature. Such devices can be divided into two categories:

• The first line of defense: Devices used by frontline law enforcement and border control officers. Typically, they need to be quite compact so as to fit a limited workspace. The Regula 4325 is a good example of such a device. It employs a special microscope that features a sliding light with a very sharp angle. With this device, you can instantly see any changes in glossiness and microrelief from dried ink drops.

• The second line of defense: Advanced tools used by forensic experts in labs. Among their other powerful capabilities, they allow you to identify and visually demonstrate signs of morphing, e.g., by building a 3D model of a questionable fragment. A device that fulfills this purpose is the Regula 4308. It includes two high-resolution cameras, over 30 types of light sources, over 20 light filters, and all the necessary tools for precise forensic analysis.

These days, photo morphing is a problem for a number of reasons:

• Quality morphs are indistinguishable from real humans.

• All sorts of specialized printers are available for sale.

• There’s minimal interference with the document’s security features.

• High passenger/customer traffic increases the chances of non-detection.

This threat cannot be ignored, and yet, the lion might be not as fierce as he is painted. But you must apply the correct approach: effective prevention requires the use of technical means to automate or semi-automate the document verification process. Not only does this accelerate the workflow, but it also adds an extra protection layer at every step.

Modern technologies have taken identity fraud to a new level but will also help you cope with it. Over three decades, Regula has been accumulating expertise to face any upcoming challenges. Do you have any other questions related to facial morphing? Feel free to get in touch; we’re here to help.