Overview
Luxand FaceSDK is a cross-platform face detection and recognition library that can be easily
integrated into the customer’s application. FaceSDK offers the API (Application
Programming Interface) to detect and track faces and facial features, to recognize gender and
facial expressions (if a smile is present and if the eyes are open or closed), and to recognize
faces on still images and videos.
FaceSDK is provided with Tracker API which allows tracking and recognizing faces in live
video. Tracker API simplifies working with video streams, offering the functions to tag
subjects with names and recognize them further.
The SDK provides the coordinates of 70 facial feature points (including eyes, eyebrows,
mouth, nose and face contours). Luxand FaceSDK uses multiple processor cores to speed up
recognition. The library supports DirectShow-compatible web cameras and IP cameras with
an MJPEG interface
Luxand FaceSDK is a dynamic link library available for 32-bit and 64-bit versions of
Windows and Linux, 64-bit MacOS X, iOS, Android. The SDK contains interface header files
and
sample
applications
for
C++,
Microsoft
Visual
C++
6.0/2005/2008/2010/2012/2013/2015, Visual Basic .NET 2005/2008/2010+, Microsoft C#
.NET 2005/2008 / 2010+, Borland Delphi 6.0+, Netbeans (Java), Xcode 4.2+ (iOS), Eclipse
ADT (Android), Android Studio (Android), Visual Basic 6.0 and C++Builder 6.0.
Requirements
The FaceSDK library supports the following platforms:
· Windows 2000/XP/2003/Vista/2008/2012, Windows 7, Windows 8, Windows 10
· Linux (RHEL 5+, CentOS 5+ and other)
· Mac OS X 10.5+ x86_64
· iOS 5.0+, armv7/x86 (iPhone 3GS+, iPad 1+, simulator)
· iOS 7.0+, arm64/x86_64(iPhone 5S+, iPad Air+, iPad mini retina+, simulator)
· Android 4.0+ (platform version 14+), armv7 (armeabi-v7a)/x86
An Intel processor is recommended for better performance.
Minimum system requirements:
· 1 GHz processor
· 256 MB RAM
Recommended system requirements:
· Intel Core i7 or Xeon processor
· 2 GB RAM
· DirectShow-compatible webcam
· IP camera with MJPEG interface (like AXIS IP cameras)
Note that the web camera functions are available only for the Windows platform. IP cameras
are accessible on all platforms.
Technical Specifications
The FaceSDK library has the following technical specifications:
6
Face Detection
· Robust frontal face detection
· Detection of multiple faces in a photo
· Head rotation support: –30..30 degrees of in-plane rotation and –30..30 degrees out-
of-plane rotation
· Determines in-plane face rotation angle
· Detection speed:
o Realtime detection (webcam resolution, –15..15 degrees of in-plane head
*
rotation): 0.003456 sec (289 FPS) (Intel ), 0.006420 sec (156 FPS) (iOS*),
0.020 sec (50 FPS) (Android*)
o Reliable detection (digital camera resolution, –30..30 degrees of in-plane head
rotation): 0.075710 sec (Intel), 0.22235 sec (iOS), 0.642 sec (Android)
· Returned information for each detected face: (x,y) coordinates of face center, face
width and rotation angle
· Easy configuration of face detection parameters
Face Matching
· Matching of two faces at given FAR (False Acceptance Rate) and FRR (False
Rejection Rate)
· Enrollment time:
o Webcam resolution, using
FSDK_DetectEyes/FSDK_GetFaceTemplateUsingEyes: 0.01455 seconds (69
FPS) (Intel), 0.01311 seconds (76 FPS) (Intel Xeon*), 0.03398 seconds (29
FPS) (iOS), 0.091 seconds (11 FPS) (Android)
o Webcam resolution, using FSDK_GetFaceTemplateInRegion (higher
accuracy): 0.014603 seconds (68 FPS) (Intel), 0.013035 seconds (76 FPS)
(Intel Xeon), 0.033867 seconds (29 FPS) (iOS), 0.097 seconds (10 FPS)
(Android)
· Template Size: 13 kb
· Matching speed:
o Single thread, templates per second: 77073 (Intel), 96899 (Intel Xeon) 80671
(iOS), 12080 (Android)
o Multiple parallel threads, templates per second: 311526 (Intel), 442635 (Intel
Xeon), 154440 (iOS), 45353 (Android)
· Returned information: facial similarity level
Live Video Recognition with Tracker API
· Assigns a unique ID to each subject detected in video
· Allows tagging any subject in video with a name, and recognizing it further
· No requirement for a subject to pose to be enrolled
· Constant learning of subjects’ appearance
· Provides with estimates of false acceptance rate and recognition rate
· Tracks multiple faces and their facial features
· Recognizes male and female genders
· Recognizes facial expressions
7
Facial Feature Detection
· Detection of 70 facial feature points (eyes, eyebrows, mouth, nose, face contour)
· Detection time (using FSDK_DetectFacialFeaturesInRegion, not including face
detection stage): 0.00244 seconds (408 FPS) (Intel), 0.0043 seconds (233 FPS) (iOS),
0.0115 seconds (86 FPS) (Android)
· Allowed head rotation: –30..30 degrees of in-plane rotation, –20..20 degrees out-of-
plane rotation
· Returned information: array of 70 (x,y) coordinates of each facial feature point
Eye Centers Detection
· Detection of eye centers only, detection time (not including face detection stage):
0.002434 seconds (410 FPS) (Intel), 0.00442 seconds (226 FPS) (iOS), 0.0112 (89
FPS) seconds (Android)
· Returned information: two (x,y) coordinates of left eye center and right eye center
Gender Recognition
· Recognition of different genders
· Gender recognition time (not including face and facial feature detection stages):
0.00629 seconds (Intel), 0.0087 seconds (iOS), 0.028 seconds (Android)
· Returned information: confidence level in each gender
Facial Expression Recognition
· Recognizes if the subject smiles and if the eyes are open or closed
· Expression recognition time (not including face and facial feature detection stages):
0.00629 seconds (Intel), 0.0087 seconds (iOS), 0.028 seconds (Android)
· Returned information: confidence level in each facial expression
Multi-Core Support
· The library uses all available processor cores when executing face detection or
recognition functions to maximize the performance.
Library Size
· The size of the redistributables does not exceed 40MB for each platform.
*
Measured on Intel Core i7 4850HQ processor with 8 threads, Intel Xeon E3-1270 V2
processor with 8 threads, iPad Pro with 2 threads, Asus - MeMO Pad 7 K013 Tablet with 4
threads.
Installation
Windows
To install Luxand FaceSDK, run the installation file:
8
Luxand_FaceSDK_Setup.exe
and follow the instructions.
FaceSDK is installed to the C:\Program Files\Luxand\FaceSDK directory by
default. FaceSDK is a copy-protected library, and your application must activate the library
on startup (see the Library Activation chapter).
Linux/Mac OS X
Unpack theLuxand_FaceSDK.tar.bz2 archive into the desired directory.
Directory Structure
The FaceSDK directory contains the following directories and files:
bin\
FaceSDK binary files
bin\android
bin\iOS
FaceSDK Android binaries
FaceSDK iOS binaries
bin\linux_x86
bin\linux_x86_64
bin\osx_x86_64
bin\win32
bin\win64
demo\
FaceSDK Linux 32-bit binaries
FaceSDK Linux 64-bit binaries
FaceSDK Mac OS X 64-bit binaries
FaceSDK Widows 32-bit binaries and stub library files
FaceSDK Windows 64-bit binaries and stub library files
Demo applications (win32)
include\
Header files
samples\
Sample applications
Sample Applications
FaceSDK is distributed with the following sample applications (they can be found in the
FaceSDK samples\ directory):
1. LiveRecognition
This application receives video from a camera, allows tagging any subject with a
name, and then display the name (recognizing the subject). The application utilizes
Tracker API. Source code is available on Microsoft C# 2005/2008 / 2010 and higher,
iOS, Android (Eclipse and Android Studio), Borland Delphi 6.0 and higher, Microsoft
Visual C++ 2005 / 2008 / 2010 / 2012 / 2013 / 2015, Microsoft Visual Basic .NET
2005/2008 / 2010 and higher, Java and Visual Basic 6.0. The iOS/Android versions
are published in the Apple AppStore and in Google Play (“Luxand Faсe Recognition”
application).
9
2. FaceTracking
This application receives video from a webcam and highlights all detected faces with
rectangles. The application utilizes Tracker API. Source code is available on
Microsoft C# 2005/2008 / 2010 and higher, Borland Delphi 6.0 and higher, Microsoft
Visual C++ 2005 / 2008 / 2010 / 2012 / 2013 / 2015, Microsoft Visual Basic .NET
2005/2008 / 2010 and higher, Java and Visual Basic 6.0.
3. Lookalikes
This application allows the user to create a database of faces and run a search for the
best matches (the most similar face from the database is shown). Source code is
available on Microsoft Visual C++ 2005 / 2008 / 2010 / 2012 / 2013 / 2015, Microsoft
C# 2005 / 2008 / 2010 and higher, Java and Borland Delphi 6.0 and higher. There is
an example of working with Microsoft SQL database on Microsoft C# 2005 / 2008 /
2010 and higher, and with and SQLite on Microsoft Visual C++ 2005 / 2008 / 2010 /
2012 / 2013 / 2015. To run the Microsoft SQL example, you need to attach the
database (located in the DB folder of the sample) to the Microsoft SQL Server.
4. LiveFacialFeatures
This application tracks users’ facial features in real time using a web camera. The
coordinates of facial features are smoothed by Tracker API to prevent jitter. Source
code is available on Microsoft C# 2005/2008 / 2010 and higher, Borland Delphi 6.0
and higher, Java, Microsoft Visual C++ 2005 / 2008 / 2010 / 2012 / 2013 / 2015, iOS,
Android (Eclipse and Android Studio) and Microsoft Visual Basic .NET 2005 / 2008 /
2010 and higher.
5. GenderRecognition
Using Tracker API, this application recognizes the gender of a subject looking into a
webcam. Source code is available on Microsoft C# 2005 / 2008 / 2010 and higher,
Borland Delphi 6.0 and higher, Java, Microsoft Visual C++ 2005 / 2008 / 2010 / 2012
/ 2013 / 2015, iOS, Android (Eclipse and Android Studio) and Microsoft Visual Basic
.NET 2005 / 2008 / 2010 and higher.
6. ExpressionRecognition
Using Tracker API, this application recognizes if a subject looking into a webcam is
smiling, and if the subject's eyes are open or closed. Source code is available on
Microsoft C# 2005 / 2008 / 2010 and higher, Borland Delphi 6.0 and higher, Java,
Microsoft Visual C++ 2005 / 2008 / 2010 / 2012 / 2013 / 2015, iOS, Android (Eclipse
and Android Studio) and Microsoft Visual Basic .NET 2005 / 2008 / 2010 and higher.
7. FacialFeatures
This application opens a photo, detects a face in a photo (only one face, the one that
can be detected best), detects facial features and draws a frame around the detected
face and detected features. Source code is available on Microsoft C# 2005 / 2008 /
2010 and higher, Borland C++ Builder 6.0, Borland Delphi 6.0 and higher, Java,
Microsoft Visual C++ 2005 / 2008 / 2010 / 2012 / 2013 / 2015, iOS, Android (Eclipse
and Android Studio), Microsoft Visual Basic .NET 2005 / 2008 / 2010 and higher,
Visual Basic 6.0.
8. IPCamera
This application opens an IP camera (allowing the user to specify its address, user
name and password), displays the image from the camera and tracks faces. The
10
application utilizes Tracker API. Source code is available on Microsoft C# 2005 /
2008 / 2010 and higher, Borland Delphi 6.0 and higher, Java, Microsoft Visual C++
2005 / 2008 / 2010 and higher, Microsoft Visual Basic .NET 2005 / 2008 / 2010 and
higher.
9. Portrait
This application is for the command line. The application receives a picture, detects a
face and, if the face is found, crops it and saves it to a file. Source code is available on
C++.
10. Advanced
This sample provides source code for .NET wrapper that links facesdk.dll
dynamically. Refer to Using with .NET (C# and VB) for details. The sample also
provides source code for Java wrapper.
Using FaceSDK with Programming Languages
To access the FaceSDK library functions, you need to use its binary file in your applications.
The specific file depends on the platform:
· Windows applications usefacesdk.dll
· Windows .NET applications usefacesdk.NET.dll
· Linux and Android applications uselibfsdk.so
· Mac OS X applications uselibfsdk.dylib
· Java applications use facesdk.jar, jna.jar and the appropriate binary file
(facesdk.dll,libfsdk.dylib orlibfsdk.so)
· iOS applications use libfsdk-static.a and libfsdk-static_64.a
· VB6 applications usefacesdk-vb.dll in addition tofacesdk.dll
On Windows, Linux and Mac it is usually recommended to store this file in the directory
where the executable file of your application is located. Alternatively, you may keep the file
in:
· the working directory of your application
· the directory specified in the path environment variable of your system: PATH
(Windows),LD_LIBRARY_PATH (Linux),DYLD_LIBRARY_PATH (Mac OS X).
You need to include interface header files into your application project in order to use
FaceSDK.
Using with .NET (C# and VB)
First, you need install .NET Framework 2.0 on your system. In Windows Vista/2008/7/8/2012
it can be done by enabling Microsoft .NET Framework 3.0 or 3.5 in Control Panel (Programs
and Features – Turn Windows features on or off).
For Microsoft .NET applications, you need to add the .NET component into your project.
Follow these steps to add the component in Visual Studio 2005/2008/2010+:
· Select Project – Add Reference – Browse
· For 32-bit applications, choose the filebin\win32\FaceSDK.NET.dll
· For 64-bit native applications, choose the filebin\win64\FaceSDK.NET.dll
11
· Add the following statement to the beginning of your application:
using Luxand
After that you may use the methods of theLuxand.FSDK namespace for general FaceSDK
functions, andLuxand.FSDKCam namespace for webcam-related functions. You may refer
just toFSDK andFSDKCam namespaces ifusing Luxand is specified.
OnceFaceSDK.NET.dll is added to the references, it will be redistributed automatically
with your application, so no specific deployment actions are required. You do not need to
redistribute facesdk.dll with your application. However, you need to redistribute
Microsoft Visual C++ 2008 SP1 Runtime.
By default, the documentation refers to C/C++ declarations of FaceSDK functions. For
example, the function to detect a face is referred to as FSDK_DetectFace function. To
refer to this function in .NET, replace the FSDK_ prefix withFSDK. namespace. Thus, the
reference to this function becomes FSDK.DetectFace (note that webcam-specific
functions are located in theFSDKCam. namespace; refer to Working with Web Cameras for
details).
Note: This .NET component is available in a binary form, compatible with .NET 2.0, 3.0, 3.5,
4.0 and 4.5. To use the component with .NET 4.0 and 4.5 (in Visual Studio 2010, 2012 and
higher), just add theuseLegacyV2RuntimeActivationPolicy="true" attribute
to the<startup>section of the app.config file of your project. If there is no such file in
your project, just create the file with the following content:
<?xml version="1.0"?>
<configuration>
<startup useLegacyV2RuntimeActivationPolicy="true">
<supportedRuntime version="v4.0"
sku=".NETFramework,Version=v4.0"/>
</startup>
</configuration>
If you need a component for a specific .NET version, you may use the source code available
in the samples\advanced\.NET wrapper directory. Note that this component is
actually a wrapper for facesdk.dll that is linked dynamically, so facesdk.dll must be
redistributed with the application that uses this wrapper.
Using CImage class in .NET
CImage is a class for Microsoft .NET for easy manipulation of images. CImage encapsulates
an HImage handle and provides convenient methods for applying FaceSDK functions to that
image.
To start working with CImage, just create an instance of it. You can pass a file path, HImage
handle, HBITMAP handle or System.Drawing.Image object to the constructor to load the
corresponding object into the image. Alternatively, call the constructor without parameters to
create
an
empty
image.
Refer
to
the
functions
FSDK_LoadImageFromFile,
FSDK_LoadImageFromHBitmap,
FSDK.LoadImageFromCLRImage
and
FSDK_CreateEmptyImage for further details.
A CImage instance has three properties: ImageHandle, Width and Height. ImageHandle is a
handle of the internal representation of the image encapsulated by the class. Width and Height
12
properties are the width and height of an image in pixels (see FSDK_GetImageWidth and
FSDK_GetImageHeight). If you alter the ImageHandle handle directly (for example,
executing an FSDK. method applied to that image handle), you must update the CImage
object by calling the CImage.ReloadFromHandle() method. CImage throws an exception if
any FaceSDK function, called within the CImage method, has returned an error.
Most CImage methods operating with an image (for example, the Resize() method) return the
processed image as the result.
The CImage destructor releases ImageHandle, so there is no need to call FSDK.FreeImage
explicitly after the instance has been destroyed.
Note that when you pass an existing image handle to the constructor, it will be freed after the
destruction of the CImage class instance, and become invalid. If you need the original image
handle to be valid after the CImage class instance is destroyed, consider creating a copy of the
image handle and passing the copy to the CImage constructor.
CImage();
Creates an empty CImage instance.
Syntax:
FSDK.CImage();
CImage(Int);
Creates a CImage instance from image already loaded to FaceSDK.
Syntax:
FSDK.CImage(int ImageHandle);
Parameters:
ImageHandle – the internal handle of an image already loaded to FaceSDK. The destructor
will free the ImageHandle handle.
CImage.ReloadFromHandle();
Updates the internal properties of a CImage instance in accordance with the ImageHandle.
Syntax:
FSDK.CImage.ReloadFromHandle();
Using with C/C++
For Microsoft Visual C++ applications, you need to include the header file
include\C\LuxandFaceSDK.h, and the stub library file facesdk.lib into your
project.
Follow these steps to add the library to your project:
· Copyinclude\C\LuxandFaceSDK.h into the directory of your project
· For 32-bit applications, copy bin\win32\facesdk.dll and
bin\win32\facesdk.lib into the output directory of your project
· For 64-bit applications, copy bin\win64\facesdk.dll and
bin\win64\facesdk.lib into the output directory of your project
13
· Choose Project Properties – Linker – Input – Additional Dependencies, and add
facesdk.lib string
· Choose Project Properties – Linker – General – Additional Library Directories
Dependencies, and add $(OutDir) string (a reference to the output directory)
· Add the following statement to the beginning of your application:
include "LuxandFaceSDK.h"
The output directory$(OutDir) typically refers toDebug\ orRelease\ in the directory
of your solution. You may change it in the Configuration Properties – General of your project.
You may also choose another directory to store the .lib file, but it is recommended to keep
facesdk.dll in the directory where the executable file of your application is located.
You need to redistribute the filefacesdk.dll with your application.
Using with Delphi
For Delphi applications, put facesdk.dll into the working directory of your application
and use theinclude\Delphi\LuxandFaceSDK.pas unit in your project.
You need to redistribute the filefacesdk.dll with your application.
Using with Java
You need JDK 1.6 (or OpenJDK 1.6) to use FaceSDK with Java. The FaceSDK Java wrapper
uses JNA (all information about JNA and the actual version can be found at
https://github.com/twall/jna, but jna.jar is included in the distribution for your convenience).
The FaceSDK java wrapper works with any IDE, but only Netbeans sample projects are
provided with the distribution.
To use FaceSDK in your Netbeans project, follow these steps:
1) Add
FaceSDK.jar
(include\Java\FaceSDK.jar)
and
jna.jar
(include\Java\jna.jar) to the Librariessection of the project.
2) Add the following imports to your source code:
import Luxand.*;
import Luxand.FSDK.*;
import Luxand.FSDKCam.*;
3) Put
the
appropriate facesdk
binaries
(facesdk.dll, libfdsk.so
or
libfdsk.dylib) in the project directory (or to the /usr/libdirectory if using
OpenJDK).
You need to redistribute the FaceSDK binaries(facesdk.dll,libfdsk.so or
libfdsk.dylib) as well as FaceSDK.jar andjna.jar with your application.
Using with Cocoa
If you are using Cocoa to write an application in XCode, make sure that your source code
files (which use FaceSDK) have the.mm extension, so they are compiled as Objective-C++.
If the files have the.m extension, they are compiled as Objective-C and cannot use FaceSDK.
14
Using with VisualBasic 6.0
For Visual Basic 6.0 applications, put the Visual Basic wrapper (bin\win32\FaceSDK-
VB.dll)
into
the
project
directory
and
add LuxandFaceSDK.bas
(include\VB6\LuxandFaceSDK.bas) module to your project (Select Project – Add
module – Existing and choose a module location). Also you need to putfacesdk.dll into
the application working directory.
Note that Tracker API functions employ the Currency data type to store 64-bit integer values.
You need to redistribute both FaceSDK-VB.dll and facesdk.dll with your
application.
Using with iOS
Refer to the Using with Cocoa section. You should use the C++ syntax within your program.
To enable FaceSDK support in your Xcode iOS project, add bin/iOS/libfsdk-
static.a,
bin/iOS/libfsdk-static_64.a
and
include/C/LuxandFaceSDK.h to your project.
Using with Android
You will need to copy the subdirectories contained in bin/android/ to the libs/
directory of your project. Also you need to add theinclude/android/FSDK.java file
tosrc/com/luxand/.
The syntax of some functions on Android is different from the corresponding Java syntax due
to the usage of JNI instead of JNA.
Note: Only armv7 (armeabi-v7a) and x86 architectures are supported by FaceSDK on the
Android platform.
Unicode support
The library supports Unicode filenames on the Windows platform. If you work with file
names in the Unicode character set, use functions FSDK_LoadImageFromFileW and
FSDK_SaveImageToFileW to open and save files.
Redistributables
The following files may be redistributed with your application:
bin\win32\facesdk.dll (for 32-bit systems)
bin\win64\facesdk.dll (for 64-bit systems)
Windows
Linux
bin\win32\facesdk.NET.dll (for 32-bit .NET applications)
bin\win64\facesdk.NET.dll (for 64-bit .NET applications)
bin\win32\FaceSDK-VB.dll (for Visual Basic 6.0 applications)
bin\linux_x86\libfsdk.so (for 32-bit systems)
bin\linux_x86_64\libfsdk.so (for 64-bit systems)
Mac OS X bin\osx_x86_64\libfsdk.dylib (for 64-bit systems)
15