Download: Beet implementation, replication package, Docker image, modified Daikon.
“Test oracle generation for REST APIs” by Juan C. Alonso, Michael D. Ernst, Sergio Segura, and Antonio Ruiz-Cortés. ACM Transactions on Software Engineering and Methodology, 2025.
The number and complexity of test case generation tools for REST APIs have significantly increased in recent years. These tools excel in automating input generation but are limited by their test oracles, which can only detect crashes, regressions, and violations of API specifications or design best practices. This article introduces AGORA+, an approach for generating test oracles for REST APIs through the detection of invariants — output properties that should always hold. AGORA+ learns the expected behavior of an API by analyzing API requests and their corresponding responses. We enhanced the Daikon tool for dynamic detection of likely invariants, adding new invariant types and creating a front-end called Beet. Beet translates any OpenAPI specification and a set of API requests and responses into Daikon inputs. AGORA+ can detect 106 different types of invariants in REST APIs. We also developed PostmanAssertify, which converts the invariants identified by AGORA+ into executable JavaScript assertions. AGORA+ achieved a precision of 80% on 25 operations from 20 industrial APIs. It also identified 48% of errors systematically seeded in the outputs of the APIs under test. AGORA+ uncovered 32 bugs in popular APIs, including Amadeus, Deutschebahn, GitHub, Marvel, NYTimesBooks, and YouTube, leading to fixes and documentation updates.
Download: Beet implementation, replication package, Docker image, modified Daikon.
BibTeX entry:
@article{AlonsoESRC2025,
author = {Juan C. Alonso and Michael D. Ernst and Sergio Segura and
Antonio Ruiz-Cort{\'e}s},
title = {Test oracle generation for {REST} {APIs}},
journal = {ACM Transactions on Software Engineering and Methodology},
year = {2025}
}
(This webpage was created with bibtex2web.)
Back to Michael Ernst's publications.