API Sanity Checker

(Redirected from API Sanity Autotest)
Jump to: navigation, search

API Sanity Checker is an automatic generator of basic unit tests for shared C/C++ libraries. It is able to generate reasonable (in most, but unfortunately not all, cases) input data for parameters and compose simple ("sanity" or "shallow"-quality) test cases for every function in the API through the analysis of declarations in header files. The quality of generated tests allows to check absence of critical errors in simple use cases. The tool is able to build and execute generated tests and detect crashes (segfaults), aborts, all kinds of emitted signals, non-zero program return code and program hanging. It may be considered as a tool for out-of-the-box low-cost sanity checking (fuzzing) of the library API or as a test development framework for initial generation of templates for advanced tests. Also it supports universal T2C format of tests, random test generation mode, specialized data types and other useful features.

See also:




All releases can be downloaded from this page or github.com.

Latest release: 1.98.6

Previous release: 1.12.11


Read-only access to the latest development version:
   git clone git://github.com/lvc/api-sanity-checker.git  


This program is free software. You may use, redistribute and/or modify it under the terms of either the GNU GPL or LGPL.

Supported Platforms

GNU/Linux, FreeBSD, Mac OS X, MS Windows.

System Requirements

  • Linux, FreeBSD
    • ABI Compliance Checker (1.98.7 or newer)
    • G++ (3.0-4.7, recommended 4.5 or newer)
    • GNU Binutils (readelf, c++filt, objdump)
    • Perl 5 (5.8 or newer)
    • Ctags (5.8 or newer)
    • WARNING: if you are using ccache program (i.e. gcc points to /usr/lib/ccache/gcc) then it should be newer than 3.1.2 or disabled.
  • MS Windows
    • ABI Compliance Checker (1.98.7 or newer)
    • MinGW (3.0-4.7, recommended 4.5 or newer)
    • MS Visual C++ (dumpbin, undname, cl)
    • Active Perl 5 (5.8 or newer)
    • Ctags (5.8 or newer)
    • Add tool locations to the PATH environment variable
    • Run vsvars32.bat script (C:\Microsoft Visual Studio 9.0\Common7\Tools\)

Examples of Generated Tests

Library Version Number of Tests Problems Found
FreeType2 2.3.11 178 13
Glibc 2.13 1996 340
libX11 1.3.4 778 286

Detectable Problems

  • Crash (segfault, signal SEGV)
  • Abort (signal ABRT)
  • All emitted signals: FPE, BUS, ILL and others
  • Non-zero exit code
  • Program hanging
  • Requirement failure (if specified)


The tool is ready-to-use after extracting the archive. You can also use a Makefile to install the tool into the system:
 cd api-sanity-checker-x.y.z/ 

 sudo perl Makefile.pl -install --prefix=PREFIX [/usr, /usr/local, ...] 

This command will install an  api-sanity-checker  program in the  PREFIX/bin  system directory.

To verify that the tool is installed correctly and it works on your host run:
 cd tmp/ 

 api-sanity-checker -test 


For generating, building and running tests you should provide the XML descriptor for your library version. It is a simple XML-file that specifies version number, paths to header files and shared libraries and optionally some other information. An example of the descriptor is the following (0.3.4.xml):




Generating Tests

Command for generating a test suite:
   api-sanity-checker -lib NAME -d VER.xml -gen 

Viewing Tests

You can view generated tests using the index file:
or manually in the directory:

Building Tests

Command for building tests:
   api-sanity-checker -l NAME -d VER.xml -build 

Running Tests

Command for running tests:
   api-sanity-checker -l NAME -d VER.xml -run 

After a time will be generated test report:

Improving Tests

To improve generated tests quality, you can provide the collection of specialized types for the library.

Command-Line Options

The tool has many useful options for manipulating with the test generation and execution processes. See the list of all options on this page.

Most useful options:


Generate, build and execute the tests for the libssh library:
   api-sanity-checker -l libssh -d 0.3.4.xml -gen -build -run 

Tests will be generated to:

The report will be generated to:

Generate tests in the Template2Code (T2C) format:
   api-sanity-checker -l libssh -d 0.3.4.xml -gen -t2c 

The package with T2C tests will be generated to:

Execute the tests using the Xvfb server:
   api-sanity-checker -l allegro -d 4.9.21.xml -run -xvfb 


See the article at glibc wiki.

Similar Tools

  1. Trinity - The Linux system call fuzzer,
  2. iknowthis - A system call fuzzer for UNIX-like systems,

How the Tool Works

The basic idea of the test data generation algorithm is to recursively initialize parameters of a function using the values returned (or returned through the out-parameter) by other functions for structured data types (class, struct, union) or by some simple values for intrinsic data types (int, float, enum, ...). The recursion step includes the heuristic selection of the appropriate function, that should be called to initialize complex parameters for other functions. If some parameter of a function cannot be initialized, then the algorithm tries to select other function.

Let's see the example test for FT_Activate_Size ( FT_Size size ) function from the FreeType2 library:

   #include <freetype/freetype.h>
   int main(int argc, char *argv[])
       FT_Library alibrary = 0;
       FT_Init_FreeType(&alibrary);//initialize "alibrary"
       FT_Face face = 0;
           &face);//initialize "face"
       FT_Size size = 0;
       FT_New_Size(face, &size);//initialize "size"
       FT_Activate_Size(size);//target call
       return 0;

In this test case the parameter "size" of target function FT_Activate_Size is initialized through the call of FT_New_Size function using its 2nd out-parameter. The first parameter "face" of FT_New_Size function is recursively initialized by the use of FT_New_Face function's 4th out-parameter. And finally the first parameter "alibrary" of FT_New_Face is initialized by the call of FT_Init_FreeType function on the 3rd recursion step. Other parameters of FT_New_Face function are initialized by intrinsic values.


Please send bug reports, feature requests and questions to the maintainer or post to the issue tracker.


LSB 4.0 certification test suites for Qt3 (9792 interfaces tested), Qt4 (10803 interfaces tested) and libxml2 (1284 interfaces tested) libraries were developed with the help of this tool, also known as AZOV Code Generator in the past (2007-2009).

Future Plans

  • Extend support of Splint and ACSL annotations
  • Make better distinction between pointer arguments (*x) and arrays (x[])
  • Add CUnit format support, see 2011 GSoC LSB projects
  • More kinds of generated test data files (images, fonts, ...)
  • Option for generating bound values for parameters


The tool was originally developed by Andrey Ponomarenko.


We would like to thank everyone who has contributed to the success of this project!


  1. wikipedia.org, “Sanity testing in software development”
  2. wikipedia.org, “Fuzz testing”
  3. R. S. Zybin , V. V. Kuliamin , A. V. Ponomarenko , V. V. Rubanov and E. S. Chernov, “Automation of broad sanity test generation”
Personal tools