API Sanity Checker

(Redirected from API Sanity Autotest)
Jump to: navigation, search

API Sanity Checker is an automatic generator of basic unit tests for a shared C/C++ library. It is able to generate reasonable (in most, but unfortunately not all, cases) input data for parameters and compose simple ("sanity" or "shallow"-quality) test cases for every function in the API through the analysis of declarations in header files.

The quality of generated tests allows to check absence of critical errors in simple use cases. The tool is able to build and execute generated tests and detect crashes (segfaults), all kinds of emitted signals, non-zero program return code and program hanging.

It may be considered as a tool for out-of-the-box low-cost sanity checking (fuzzing) of the library API or as a test development framework for initial generation of templates for advanced tests. Also it supports universal T2C format of tests, random test generation mode, specialized data types and other useful features.



The tool can be downloaded from github.com.


This program is free software. You may use, redistribute and/or modify it under the terms of either the GNU GPL or LGPL.

Supported Platforms

Linux, FreeBSD, Mac OS X, MS Windows.

System Requirements

The tool requires:

  • ABI Compliance Checker (1.98.7 or newer)
  • G++ (3.0-4.7, 4.8.3 or newer)
  • GNU Binutils (readelf, c++filt, objdump)
  • Perl 5 (5.8 or newer)
  • Ctags (5.8 or newer)

WARNING: if you are using ccache program (i.e. gcc points to /usr/lib/ccache/gcc) then it should be newer than 3.1.2 or disabled.

On Mac OS X the tool also requires Xcode for g++, c++filt, nm and otool.

On MS Windows the tool also requires MinGW, MS Visual C++ (dumpbin, undname, cl), Active Perl 5, adding of tool locations to the PATH and execution of vsvars32.bat script (C:\Microsoft Visual Studio 9.0\Common7\Tools\).


The tool is ready-to-use after extracting the archive. You can also use a Makefile to install the tool into the system:
 cd api-sanity-checker-x.y.z/ 
 sudo make install prefix=PREFIX [/usr, /usr/local, ...] 

This command will install the  api-sanity-checker  program into the  PREFIX/bin  system directory and private modules into the  PREFIX/share .

To verify that the tool is installed correctly and it works on your host run:
 cd tmp/ 
 api-sanity-checker -test 

Detectable Problems

  • Crash (segfault, signal SEGV)
  • All emitted signals: ABRT, FPE, BUS, ILL and others
  • Non-zero exit code
  • Program hanging
  • Requirement failure (if specified)


For generating, building and running tests you should provide the XML descriptor for your library version. It is a simple XML-file that specifies version number, paths to header files and shared libraries and optionally some other information. An example of the descriptor is the following:




Command for generating a test suite:
   api-sanity-checker -lib NAME -d VER.xml -gen 

You can view generated tests using the index file:
or manually in the directory:

Command for building tests:
   api-sanity-checker -l NAME -d VER.xml -build 

Command for running tests:
   api-sanity-checker -l NAME -d VER.xml -run 

The test report will be generated to:


See the article at glibc wiki.

Examples of Generated Tests

Library Version Number of Tests Problems Found
FreeType2 2.3.11 178 13
Glibc 2.13 1996 340
libX11 1.3.4 778 286

Specialized Types

To improve generated tests quality, you can provide the collection of specialized types for the library.

Command-Line Options

See the list of supported options in the output of -help and -info options.

Most useful options are:

  • -debug
  • -header
  • -random
  • -check-retval
  • -optimize-includes
  • -cache
  • -test

How the Tool Works

The basic idea of the test data generation algorithm is to recursively initialize parameters of a function using the values returned (or returned through the out-parameter) by other functions for structured data types (class, struct, union) or by some simple values for intrinsic data types (int, float, enum, ...). The recursion step includes the heuristic selection of the appropriate function, that should be called to initialize complex parameters for other functions. If some parameter of a function cannot be initialized, then the algorithm tries to select other function.

Let's see the example test for FT_Activate_Size FT_Activate_Size ( FT_Size size ) function from the FreeType2 library:

   #include <freetype/freetype.h>
   int main(int argc, char *argv[])
       FT_Library alibrary = 0;
       FT_Init_FreeType(&alibrary);//initialize "alibrary"
       FT_Face face = 0;
           &face);//initialize "face"
       FT_Size size = 0;
       FT_New_Size(face, &size);//initialize "size"
       FT_Activate_Size(size);//target call
       return 0;

In this test case the parameter "size" of target function FT_Activate_Size is initialized through the call of FT_New_Size function using its 2nd out-parameter. The first parameter "face" of FT_New_Size function is recursively initialized by the use of FT_New_Face function's 4th out-parameter. And finally the first parameter "alibrary" of FT_New_Face is initialized by the call of FT_Init_FreeType function on the 3rd recursion step. Other parameters of FT_New_Face function are initialized by intrinsic values.


Please post bug reports, feature requests and questions to the issue tracker.


LSB 4.0 certification test suites for Qt3 (9792 interfaces tested), Qt4 (10803 interfaces tested) and libxml2 (1284 interfaces tested) libraries were developed with the help of this tool, also known as AZOV Code Generator in the past (2007-2009).


The tool is developed by Andrey Ponomarenko.


  1. "Sanity testing in software development", wikipedia.org
  2. "Fuzz testing", wikipedia.org
  3. "Automation of broad sanity test generation", R. S. Zybin , V. V. Kuliamin , A. V. Ponomarenko , V. V. Rubanov and E. S. Chernov
Personal tools