Optional CI task: Benchmarks using rally
It would be good to have benchmarks directly in the CI pipeline.
Requirements:
- Manually triggered, not required for build/release#
- Runs as a weekly automated job. Use CI runner tags to separate nightly builds from benchmark.
- Uses OpenSearch Benchmark: https://github.com/opensearch-project/opensearch-benchmark
- Runs a baseline OpenSearch with standard security plugin against OpenSearch with Eliatra Suite with a predefined configuration using a representative configuration (using DLS, FLS, FM, etc)
- Runs on the same test runner as the integration tests do
- For the beginning, it just uses the standard rally http_logs track: https://esrally.readthedocs.io/en/stable/race.html#list-tracks
- Outputs the result as plain ASCII standard rally result, both to stdout and to a downloadable artifact
Edited by Nils Bandener