About LSB Distribution Testkit Manager

From ISP_RAS
Jump to: navigation, search

LSB Distribution Testkit Manager (LSB DTK Manager) is a user interface front-end of the LSB Distribution Testkit, which wraps all the current LSB test suites into a user-friendly “single LSB test suite” with unified test management and analysis capabilities. This facilitates test execution and results analysis thus making the LSB tests more attractive for upstream developers as well as making certification process technically more inviting.

Start using LSB DTK Manager

The latest LSB DTK Manager release package can be downloaded from this page. The latest stable sources snapshot is at lsb/devel/dtk-manager in LF Bazaar. There are instructions on how to build LSB DTK Manager from sources.

Note: LSB DTK Manager uses the lsb-autotest scripts written by Jiri Dluhos. The current version of LSB DTK Manager includes a modified lsb-autotest and you cannot install it separately. In the future versions, lsb-autotest will be a completely independent component.

Please read LSB DTK Manager Getting Started to get an overview of the tool with screenshots for using it smoothly.

Key Features

The key features of LSB DTK Manager + lsb-autotest are:

  • Integrated user interface for all the test suites:
    • web based;
    • command line;
  • selection of tests to run (all, manually selected subset, predefined subsets, etc.);
  • automatic download of missing test suites from the LF FTP site;
  • integration with the Certification System;
  • 'one-button' tests execution;
  • global test summary (for all the tests);
  • unified reports in one place.

Detailed features of LSB DTK Manager can be found in LSB DTK Manager Release Notes.

Future Plans

The following features can be included in the future (note: this is a wish list only - it will be continually supplemented while it is not guaranteed that all the features below will be ever implemented):

  1. Calculate and display estimated execution time during test execution.
  2. Waiver detection should be synchronized with a centralized online DB of waivers.
  3. Build mapping between individual tests within LSB test suites and particular interfaces they test (or even particular assertions in the specifications). Given simple support in the test suites, this will allow the following:
    • Consolidated reports grouped by various criteria (by test suite, by target library, by specification section (libgroup), by interface, etc.).
    • Coverage reports:
      • list of tested/untested interfaces (with groupping and summaries);
      • list of tested/untested specification assertions (with groupping and summaries). This also can be presented as a colored specification where colors denote if each line in the specification is tested or not by the tests.
    • Links in the reports to the Specification’s text (e.g. pointing out for a fail which citation from the specification was actually violated).
  4. Exhaustive cross-linking of all the reports and related info like specification, documentation and online DB of waivers.
  5. Ability to re-execute only those test cases that failed at the previous run.
  6. Ability to visually compare test results of different runs at per test case basis.
Personal tools