API fuzzing is a technique used to test the security and reliability of an application’s APIs.
Fuzzing involves sending a large number of malformed or unexpected inputs to an API to uncover potential vulnerabilities, such as input validation issues, buffer overflows, injection attacks or other types of
security flaws.
The main goal of API fuzzing is to identify vulnerabilities or weaknesses in the API implementation that an attacker could exploit. By injecting unexpected or malformed data, fuzzing can trigger unexpected behaviors or expose flaws in the API’s handling of input. This helps identify potential security vulnerabilities that an attacker could leverage to compromise the system.
The Benefits of API Fuzzing
- Security assessment. API fuzzing helps identify security vulnerabilities and weaknesses in the API implementation. By exposing these issues early in the development process, you can take corrective measures to mitigate potential risks.
- Error handling and resilience. Fuzzing can help assess how well an API handles unexpected or malformed input. By subjecting the API to various scenarios, you can identify error-handling weaknesses and ensure the system remains stable and resilient under stress or malicious inputs.
- Compliance and standards. Many industries and regulatory frameworks require thorough security testing to ensure compliance. API fuzzing helps you meet these requirements by actively testing the API for potential vulnerabilities and weaknesses.
- Third-party integration. APIs are often used to integrate third-party services or components into an application. API fuzzing allows you to assess the security posture of these integrations, ensuring that they don’t introduce vulnerabilities into your system.
- Cost-effectiveness. API fuzzing can help you identify vulnerabilities and weaknesses in a cost-effective manner. Compared to manual security testing, fuzzing can automatically generate a large number of test cases and quickly pinpoint potential issues.
Schemathesis: A Tool for API Fuzzing
Schemathesis is a specification-based testing tool designed explicitly for
OpenAPI and
GraphQL applications. It utilizes the robust
Hypothesis framework to generate test cases based on the provided API specification.
By leveraging the OpenAPI or GraphQL schema, Schemathesis automatically generates a wide range of test scenarios, covering different combinations of inputs, edge cases and potential vulnerabilities. This approach ensures thorough and systematic testing of the API implementation.
Schemathesis focuses on testing whether an API conforms to its specification. It verifies that the API responses match the expected schema, ensuring compliance with the defined contract. This type of testing helps uncover issues related to input validation, response structures, error handling and more.
By combining the benefits of the Hypothesis framework, which provides intelligent and property-based testing, with the OpenAPI or GraphQL specification, Schemathesis simplifies the process of testing and validating APIs. It streamlines the testing workflow, increases test coverage, and assists in identifying and resolving potential issues in the API implementation.
Key Features and Benefits of the Schemathesis Tool
In addition to its support for OpenAPI and GraphQL, which allows for testing a wide range of APIs, Schemathesis offers the following advantages:
- Positive and negative tests. With Schemathesis, you can create test cases that cover both valid and invalid inputs. This helps ensure your API handles unexpected or incorrect data gracefully.
- Stateful testing. Schemathesis enables the generation of sequences of API requests that build on each other. This allows you to test complex scenarios, where subsequent requests depend on the results of previous ones.
- Session replay. The tool provides a feature to store and replay test sessions. This makes it easier to investigate and debug issues by reproducing the exact sequence of API requests that led to a problem.
- Targeted testing. You can guide the data-generation process toward specific metrics, such as response time or size. This helps uncover performance or resource usage issues and optimize your API’s behavior under different conditions.
- Python integration. The tool seamlessly integrates with Python-built applications through native Asynchronous Server Gateway interface (ASGI) and Web Server Gateway Interface (WSGI) support. This ensures faster testing of your Python-based APIs.
- Customization. Schemathesis offers customization options, allowing you to fine-tune data generation, API response verification and the overall testing process to fit your specific needs.
- Continuous integration. Docker image and GitHub Action integration are supported, enabling you to run tests on every code change as part of your CI pipeline.
- Software as a Service platform. Schemathesis offers an all-in-one SaaS platform, which eliminates the need for setup or installation. This can be beneficial if you prefer a hosted solution for your testing needs.
- Commercial support. The open source tool is also available as a commercial and enterprise-level offering. The maker’s commercial support includes professional guidance to help you maintain an optimal testing workflow and address any issues or challenges you may encounter.
Getting Started with Schemathesis
By using advanced techniques like
swarm testing and
schema fuzzing, Schemathesis produces diverse and high-quality test data, enabling comprehensive testing and thorough bug detection.
The tool helps you prevent crashes, database corruption, and hangs by discovering API-breaking payloads. It also helps keep your API documentation up to date by validating examples from the OpenAPI or GraphQL schemas.
When issues occur, Schemathesis provides detailed failure reports and a single
cURL command to reproduce the problem instantly, simplifying the debugging process. By thoroughly testing your API with Schemathesis, you can have increased confidence in its stability and reliability.
Schemathesis covers a wide range of test scenarios, providing comprehensive testing coverage and uncovering potential vulnerabilities. Moreover, it saves time by automating the generation of test scenarios based on the schema.
You can use Schemathesis as a
CLI, Python library,
GitHub app or SaaS platform, making it accessible and adaptable to your testing needs.
Installation
To install Schemathesis, you can use either Python package installation or Docker image:
Python Package Installation
Run the following command in your terminal to install Schemathesis as a Python package:
python -m pip install schemathesis
This command will install the necessary dependencies and make the st entry point available for use.
Docker Image
Alternatively, if you prefer to use the
Docker image, you can pull it by executing the following command in your terminal:
docker pull schemathesis/schemathesis:stable
This command will download the Docker image of Schemathesis, allowing you to use it without installing it as a Python package.
Both installation methods provide access to the Schemathesis testing tool, and you can choose the one that suits your preferences and environment.
If you prioritize simplicity and ease of use, the Schemathesis CLI would be a suitable choice. The CLI provides a quick and straightforward way to get started with testing your API based on the schema. It offers a command-line interface where you can specify the necessary parameters, such as the API schema URL, and run the tests.
The Schemathesis CLI generates extensive tests based on the schema and reports any failures, along with reproduction instructions. This allows you to quickly identify and address issues in your API implementation.
On the other hand, if you prefer more control and customization within your codebase, the Schemathesis Python package would be a better fit. By integrating the Schemathesis library into your Python project, you have the flexibility to configure and fine-tune the testing process according to your specific needs. You can programmatically define test suites, set up custom hooks and further extend the functionality as required.
Regardless of whether you choose the CLI or the Python package, both options offer comprehensive testing capabilities and provide detailed reports of failures with reproduction instructions. You can select the option that aligns best with your workflow and preferences.
Additionally, Schemathesis offers a native GitHub app for convenient integration with pull requests, providing test result reports directly in your repositories.
How to Use Schemathesis with Different Configurations
Here are some examples of how to use the tool with different configs:
Running unit tests for all API operations with the not_a_server_error check:
export SCHEMA_URL="http://127.0.0.1:5000/api/openapi.json"
export PYTHONPATH=$(pwd)
st run $SCHEMA_URL
Selecting specific operations (POST) that have `booking` in their path:
st run -E booking -M POST $SCHEMA_URL
Running specific checks, such as status code conformance:
st run -c status_code_conformance $SCHEMA_URL
Including custom checks registered in the `test/hooks.py` module:
SCHEMATHESIS_HOOKS=test.hooks st run $SCHEMA_URL
Providing custom headers, such as an authorization token:
st run -H "Authorization: Bearer <token>" $SCHEMA_URL
Configuring Hypothesis parameters, such as running up to 1,000 examples per tested operation:
st run --hypothesis-max-examples 1000 $SCHEMA_URL
Running tests in multiple threads (eight threads, in this example):
Storing network log to a file for later replay:
st run --cassette-path=cassette.yaml $SCHEMA_URL
To replay requests from the stored log:
Running integration tests:
Make sure to set the `SCHEMA_URL` variable to the appropriate URL of your API’s schema file. You can also adjust other options and flags to customize your testing process.
Running Schemathesis from the Command Line
Using Python Package Installation
st run --checks all https://example.schemathesis.io/openapi.json
docker run schemathesis/schemathesis:stable \
run --checks all https://example.schemathesis.io/openapi.json
In both cases, the `run` command is used to initiate the test execution. The `–checks` all flag specifies that all available checks should be performed during the testing process. Replace https://example.schemathesis.io/openapi.json with the URL of your OpenAPI schema.
Choose either the Python package installation or the Docker image, based on your preferred method of installation and execution.
