Release Notes

Release Notes

0.0.0

Prelude

Added ability to use clouds.yaml with scenario tests

Create new cli test in Sahara Tempest plugin for node group templates

Clean resources created in CLI tests

Discovery of data sources with relative paths is now fixed.

Fix default resource discovery from the installed package.

Client tests have been imported from the Sahara repository.

Migrate auth system from keystoneclient to keystoneauth

Migrate sahara cli tests from saharaclient to sahara-tests

Create new cli tests in Sahara Tempest plugin for job types

Fixes and addition for the API and CLI tests.

Support for CDH 5.9 in scenario tests.

Less parameters required for scenario tests.

Yaml-files for deprecated plugins was removed

Tempest tests now support APIv2.

Scenario tests now support APIv2.

Removed the need of a .testr.conf file when calling the test runner.

Tests no longer depend on fake plugin to run

Ocata test templates are now available, while Liberty test templates have been removed.

Pike test templates are now available, while Mitaka test templates have been removed.

Queens test templates are now available, while Newton test templates have been removed.

Rocky test templates are now available.

Stein test templates are now available, while Ocata test templates have been removed.

Long overdue general updates of the test templates

Sahara Tests plugin now uses tempest stable interface

New Features

  • User can now use clouds.yaml in the format defined by os-client-config to specify the auth values wanted on a scenario test.

  • New basic scenario test template for CDH 5.13.0.

  • Now we can filter node group template with column flag by plugin and check if it was filtered successfully

  • The tests for the official Python clients have been moved here from the Sahara repository. They are based on the Tempest libraries even they do not follow the Tempest guidelines (as they need to test the Python clients, they do not use the native Tempest clients).

  • Now we can filter job types with column flag and also save config file of any job type to the specified file.

  • The Tempest-based tests have received an increase of the coverage for both API and CLI tests (job, plugins, templates; negative testing).

  • CDH 5.9 can be tested thanks to the addition of the specific templates.

  • Few parameters are not required anymore by the templates in scenario tests; a default value is provided (name of templates, etc).

  • The flavors used in templates are now parameters too.

  • If a flavor name is provided in addition to its specification and a flavor with that name exists, it is used and not created again.

  • The dependencies on non-public Tempest interfaces have been removed.

  • Sahara API tests have been imported from Tempest and made available using the Tempest Plugin Interface.

  • The Tempest plugin provides an APIv2 DataProcessing client and tempest tests can be executed against APIv2 too. The type of API used is driven by a tempest.conf configuration key (data_processing.use_api_v2 for API tests, data_processing.api_version_saharaclient for client and CLI tests)

  • The api_version_saharaclient variable now controls the Sahara API version used not only by the Tempest.lib-based clients tests, but also by the Tempest CLI tests.

  • Allow to enable boot_from_volume on node group templates when running scenario tests with APIv2.

  • sahara-scenario supports feature sets. When passing specific feature tags to sahara-scenario, additional job templates and EDP jobs marked with those tags will be loaded.

  • The fully generated YAML file is printed when the verbose mode is enabled.

  • sahara-scenario now supports testing the S3 API for job binaries and data sources, a feature introduced in Rocky. The code can be enabled using the “s3” feature and various templates now runs an S3-based job too when the feature is enabled from the command line.

  • Scenario tests can be executed against APIv2. The usage of APIv2 is enabled through a new command line argument for sahara-scenario (–v2, -2).

  • Added basic S3 API tests (job binaries and data sources) to the Tempest plugin. The tests are disabled by default and can be enabled using a new tempest.conf key (data-processing-feature-enabled.s3).

  • A folder with test templates with Ocata has been created and initialized starting from the templates in the main directory, following the status of the jobs tested on the Sahara CI.

  • A folder with scenario templates for Pike was added. It is a subset of the templates in the main directory.

  • Some requirements have been raised (especially Tempest).

  • A folder with scenario templates for Queens was added. It is a subset of the templates in the main directory.

  • The available templates now supports Spark 2.2 and Vanilla 2.8.2.

  • A folder with scenario templates for Rocky was added. It is a subset of the templates in the main directory, and includes all non-deprecated plugin/versions.

  • The available default test templates now supports also Spark 2.3, Storm 1.2, Vanilla 2.7.5, and the Ambari-based HDP 2.6 and 2.5.

  • A folder with scenario templates for Stein has been added. It is a subset of the templates in the main directory.

  • The public network field can be omitted from the configuration file of the scenario, enabling the testing when only the project network is used.

  • Capture and report the timestamp in scenario tests when an event starts and when an exception is triggered.

Upgrade Notes

  • Python 2.7 support has been dropped. Last release of sahara-tests to support python 2.7 is 0.9.1, the first version that can be used with OpenStack Train. The minimum version of Python now supported by sahara-tests is Python 3.5.

  • Migration from novaclient.v2.images to glanceclient

  • the name of the variables/parameters used for the name of the images in the scenario tests have been changes to follow a more consistent pattern.

  • Removed nova-network configuration. Nova network has been fully removed from the OpenStack codebase, in all releases supported by sahara-tests.

  • sahara-scenario now requires stestr.

Deprecation Notes

  • The saharaclient_version option in the data-processing group has been renamed to api_version_saharaclient.

  • The Liberty-specific job templates have been removed. This means that starting from this release Liberty is not supported (it has been EOL for a while).

  • The MapR 5.0.0 test template have been removed from the master branch as well.

  • The Mitaka-specific job templates have been removed. This means that starting from this release Mitaka is not supported (it has been EOL for a while).

  • The Newton-specific job templates have been removed. This means that starting from this release Newton is not supported (it has been EOL for a while).

  • The Ocata-specific job templates have been removed. This means that starting from this release Ocata is not supported (it is under Extended Maintenance now).

Critical Issues

  • The Tempest plugin was fixed after the removal of the data_processing plugin from the tempest repository. Most of the work was in place, only a small change was missing.

Bug Fixes

  • Fix bug in which after running CLI tests, projects and networks created for these tests were not deleted at the end of the run.

  • Datasources with relative paths are now properly found from the default resources.

  • The default set of resources (test templates for each plugin, etc) can now be properly discovered when the package is installed.

  • The node group template creation test for the CLI client does not fail anymore when the Sahara plugin selected for testing provides more than one version.

  • Fix the test template for storm 1.2.

  • Properly handle more use cases when only Keystone v3 is enabled and/or its service URI is missing the /v3 suffix.

  • The artifacts created during the execution of CLI tests are properly cleaned at the end of the tests.

  • Removed yaml-files for Kilo release

  • Removed unused yaml-files for master branch

  • Fix a strange error where the internally generated test does not start because sahara_tests.scenario is not found when running inside a virtualenv.

  • When passing the plugin/version parameters to sahara-scenario, users can now specify additional YAML templates which will be merged to the default YAMLs, instead of being ignored.

  • A .testr.conf file was previously required in the runner execution directory, now this is handled internally.

  • The CDH 5.11 test template, previously only available for queens, was added to main (unversioned) templates directory.

  • The default templates used by tests have been updated; added (MapR, Ambari and Storm, and some versions of CDH), or removed (obsolete versions of Vanilla and CDH).

Other Notes

  • The default timeout for cluster polling was raised from 1800 to 3600 seconds.

  • The documentation was improved (scenario tests) and extended (Tempest plugin).

  • OpenStack reno integration was added for managing release notes

  • Adapt Sahara Tests code to stop relying only on the fake plugin and use the default plugin available. However, it’s worth noting that - if available - the fake plugin will be used.

  • The dependency on tempest.scenario.manager.py has been removed. There are still dependencies on internal Tempest interfaces but they are more difficult to replace due to lack of tempest.lib alternatives, and scenario.manager.py is undergoing an heavy refactoring.

  • Updated the list of supported plugin/versions used by some Tempest and Tempest-based tests to cover all the combinations available in the Sahara releases supported by sahara-tests.

  • Sahara Tests plugin is adapted to use in-tree client, which was migrated from Tempest code. Also, there’s a new stable interface for Service Clients in Tempest, so this change adapts the code to use it.

Creative Commons Attribution 3.0 License

Except where otherwise noted, this document is licensed under Creative Commons Attribution 3.0 License. See all OpenStack Legal Documents.