测试框架是什么
A testing framework is a set of guidelines or rules used for creating and designing test cases. A framework is comprised of a combination of practices and tools that are designed to help QA professionals test more efficiently.
These guidelines could include coding standards, test-data handling methods, object repositories, processes for storing test results, or information on how to access external resources.
测试框架是一组用于创建和设计测试用例的指南或规则。框架由旨在帮助 QA 专业人员更有效地测试的实践和工具的组合组成。 这些指南可能包括编码标准、测试数据处理方法、对象存储库、存储测试结果的过程或有关如何访问外部资源的信息。
测试框架的价值
Testing frameworks are an essential part of any successful automated testing process. They can reduce maintenance costs and testing efforts and will provide a higher return on investment (ROI) for QA teams looking to optimize their agile processes.
测试框架是任何成功的自动化测试过程的重要组成部分。它们可以降低维护成本和测试工作，并为寻求优化其敏捷流程的 QA 团队提供更高的投资回报率 (ROI)。
测试框架的收益
- Improved test efficiency 提高测试效率
- Lower maintenance costs 降低维护成本
- Minimal manual intervention 最少的人工干预
- Maximum test coverage 最大测试覆盖率
- Reusability of code 代码的可重用性
常见测试框架类型
| 框架 | 说明 |
|---|---|
| TDD | 代码风格 |
| DDT | 数据驱动风格 |
| ATDD | 验收测试驱动开发 |
| BDD | 行为驱动开发 (Behavior-driven development) |
| MBT | Model Based Testing 基于模型的测试 |
{style=“margin: auto”}
TDD
TDD 定义
Test-driven development (TDD) is a software development process relying on software requirements being converted to test cases before software is fully developed, and tracking all software development by repeatedly testing the software against all test cases.
测试驱动开发（TDD）是一个软件开发过程，在软件完全开发之前，将软件需求转换为测试用例，并通过针对所有测试用例重复测试软件来跟踪所有软件开发。这与首先 开发软件和稍后创建测试用例相反
TDD 流程
- 单元测试
- 重构
- 覆盖率
- 可测性提升
- 模型驱动设计
TDD 来源于 XP 极限编程
代表作 JUnit TestNG
@Test
void standardAssertions() {
assertEquals(2, calculator.add(1, 1));
assertEquals(4, calculator.multiply(2, 2),
"The optional failure message is now the last parameter");
assertTrue('a' < 'b', () -> "Assertion messages can be lazily evaluated -- "
+ "to avoid constructing complex messages unnecessarily.");
}TestNG
package example1;
import org.testng.annotations.*;
public class SimpleTest {
@BeforeClass
public void setUp() {
// code that will be invoked when this test is instantiated
}
@Test(groups = { "fast" })
public void aFastTest() {
System.out.println("Fast test");
}
@Test(groups = { "slow" })
public void aSlowTest() {
System.out.println("Slow test");
}
}代表作 Pytest UnitTest
# content of test_sample.py
def inc(x):
return x + 1
def test_answer():
assert inc(3) == 5import unittest
class TestStringMethods(unittest.TestCase):
def test_upper(self):
self.assertEqual('foo'.upper(), 'FOO')
def test_isupper(self):
self.assertTrue('FOO'.isupper())
self.assertFalse('Foo'.isupper())
def test_split(self):
s = 'hello world'
self.assertEqual(s.split(), ['hello', 'world'])
# check that s.split fails when the separator is not a string
with self.assertRaises(TypeError):
s.split(2)BDD Behavior Driven Development
BDD
behavior-driven development (BDD) is an agile software development process that encourages collaboration among developers, quality assurance experts, and customer representatives in a software project. It encourages teams to use conversation and concrete examples to formalize a shared understanding of how the application should behave. It emerged from test-driven development (TDD). Behavior-driven development combines the general techniques and principles of TDD with ideas from domain-driven design and object-oriented analysis and design to provide software development and management teams with shared tools and a shared process to collaborate on software development.
在软件工程中，行为驱动开发 (BDD) 是一种敏捷软件开发过程，它鼓励软件项目中的开发人员、质量保证专家和客户代表之间进行协作。它鼓励团队使用对话和具体示例来形成对应用程序应该如何运行的共同理解。它源于测试驱动开发 (TDD)。行为驱动开发将 TDD 的通用技术和原则与领域驱动设计和对象的思想相结合面向分析和设计，为软件开发和管理团队提供共享工具和共享流程，以便在软件开发方面进行协作。
BDD VS TDD
BDD
BDD 与 TDD
BDD 相关框架
- JBehave
- Cucumber
- Mspec
- Specflow
Cucumber
Cucumber is a tool that supports Behaviour-Driven Development(BDD).
Cucumber 测试用例 Scenario 场景
Scenario: Finding some cheese
Given I am on the Google search page
When I search for "Cheese!"
Then the page title should start with "cheese"Cucumber 测试用例步骤定义
public class ExampleSteps {
private final WebDriver driver = new FirefoxDriver();
@Given("I am on the Google search page")
public void I_visit_google() {
driver.get("https://www.google.com");
}
@When("I search for {string}")
public void search_for(String query) {
WebElement element = driver.findElement(By.name("q"));
// Enter something to search for
element.sendKeys(query);
// Now submit the form. WebDriver will find the form for us from the element
element.submit();
}
@Then("the page title should start with {string}")
public void checkTitle(String titleStartsWith) {
// Google's search is rendered dynamically with JavaScript
// Wait for the page to load timeout after ten seconds
new WebDriverWait(driver,10L).until(new ExpectedCondition<Boolean>() {
public Boolean apply(WebDriver d) {
return d.getTitle().toLowerCase().startsWith(titleStartsWith);
}
});
}
@After()
public void closeBrowser() {
driver.quit();
}
}
cucumber 项目结构
ATDD Acceptance Test Driven Development
ATDD 定义
Acceptance test–driven development (ATDD) is a development methodology based on communication between the business customers, the developers, and the testers. ATDD encompasses many of the same practices as specification by example (SBE), behavior-driven development (BDD), example-driven development (EDD), and support-driven development also called story test–driven development (SDD). All these processes aid developers and testers in understanding the customer’s needs prior to implementation and allow customers to be able to converse in their own domain language.
验收测试驱动开发 (ATDD) 是一种基于业务客户、开发人员和测试人员之间沟通的开发方法。ATDD 包含许多与示例规范 (SBE)、行为驱动开发 (BDD)、示例驱动开发 (EDD)、和支持驱动开发（也称为故事测试驱动开发（SDD）。所有这些流程都有助于开发人员和测试人员在实施之前了解客户的需求，并使客户能够使用他们自己的领域语言进行交流。
ATDD 相关工具
- FitNesse：The fully integrated standalone wiki and acceptance testing framework
- Robot Framework is a Python-based, extensible keyword-driven automation framework for acceptance testing, acceptance test driven development (ATDD), behavior driven development (BDD) and robotic process automation (RPA).
Robotframework
Robotframework 介绍
Robot Framework is a generic open source automation framework. It can be used for test automation and robotic process automation (RPA). Robot Framework has an easy syntax, utilizing human-readable keywords. Its capabilities can be extended by libraries implemented with Python, Java or many other programming languages. Robot Framework has a rich ecosystem around it, consisting of libraries and tools that are developed as separate projects.
Robotframework 测试用例
*** Settings ***
Documentation Simple example using SeleniumLibrary.
Library SeleniumLibrary
*** Variables ***
${LOGIN URL} http://localhost:7272
${BROWSER} Chrome
*** Test Cases ***
Valid Login
Open Browser To Login Page
Input Username demo
Input Password mode
Submit Credentials
Welcome Page Should Be Open
[Teardown] Close Browser
*** Keywords ***
Open Browser To Login Page
Open Browser ${LOGIN URL} ${BROWSER}
Title Should Be Login Page
Input Username
[Arguments] ${username}
Input Text username_field ${username}
Input Password
[Arguments] ${password}
Input Text password_field ${password}
Submit Credentials
Click Button login_button
Welcome Page Should Be Open
Title Should Be Welcome Page数据驱动风格
*** Settings ***
Test Template Login with invalid credentials should fail
*** Test Cases *** USERNAME PASSWORD
Invalid User Name invalid ${VALID PASSWORD}
Invalid Password ${VALID USER} invalid
Invalid User Name and Password invalid invalid
Empty User Name ${EMPTY} ${VALID PASSWORD}
Empty Password ${VALID USER} ${EMPTY}
Empty User Name and Password ${EMPTY} ${EMPTY}BDD 风格
*** Test Cases ***
Valid Login
Given login page is open
When valid username and password are inserted
and credentials are submitted
Then welcome page should be openATDD VS BDD
| TDD | ATDD | BDD | |
|---|---|---|---|
| 受众 | 开发 | 开发 测试 客户 | 开发 测试 客户 |
| 过程 | 代码 | DSL | 行为 |
| 目标 | 代码调用功能 | 验收测试 需求 | 需求 |
{style=“margin: auto”}
MBT Model Based Testing
- model-based testing
- GraphWalker, an open-source model-based testing tool
edge 代表步骤
An edge represents an action, a transition. An action could be an API call, a button click, a timeout, etc. Anything that moves your System Under Test into a new state that you want to verify. But remember, there is no verification going on in the edge. That happens only in the vertex.
一条边代表一个动作，一个过渡。 操作可以是 API 调用、按钮单击、超时等。任何将您的被测系统移动到您想要验证的新状态的任何事情。但请记住，边缘没有进行验证。这只发生在顶点。
vertex 代表断言
A vertex represents verification, an assertion. A verification is where you would have assertions in your code. It is here that you verify that an API call returns the correct values, that a button click actually did close a dialog, or that when the timeout should have occurred, the System Under Test triggered the expected event.
一个顶点代表验证，一个断言。 验证是您的代码中有断言的地方。在这里，您可以验证 API 调用是否返回正确的值、按钮单击是否确实关闭了对话框，或者在应该发生超时时，被测系统触发了预期的事件。
graph 代表测试用例集
A model is a graph, which is a set of vertices and edges From a model, GrapWalker will generate a path through it. A model has a start element, and a generator which rules how the path is generated, and associated stop condition which tells GraphWalker when to stop generating the path.
模型是一个图，它是一组顶点和边 从模型中，GrapWalker 将生成一条通过它的路径。一个模型有一个起始元素，一个规则如何生成路径的生成器，以及告诉 GraphWalker 何时停止生成路径的相关停止条件。
测试用例样板生成
@GraphWalker(value = "random(edge_coverage(100))")
public class OwnerInformationTest extends ExecutionContext implements OwnerInformation {
private static final Logger log = LoggerFactory.getLogger(OwnerInformationTest.class);
@Override
public void v_OwnerInformation() {
$(By.tagName("h2")).shouldHave(text("Owner Information"));
setAttribute("numOfPets", Value.asValue($$x("//table/tbody/tr/td//dl").size()));
log.info("Number of pets: " + getAttribute("numOfPets"));
}
@Override
public void e_UpdatePet() {
$("button[type=\"submit\"]").click();
}
@Override
public void v_FindOwners() {
$(By.tagName("h2")).shouldHave(text("Find Owners"));
$(By.tagName("h2")).shouldBe(visible);
}
@Override
public void e_EditPet() {
$(By.linkText("Edit Pet")).click();
}
@Override
public void e_AddNewPet() {
$(By.linkText("Add New Pet")).click();
}
@Override
public void e_AddVisit() {
$(By.linkText("Add Visit")).click();
}
@Override
public void e_FindOwners() {
$("[title='find owners']").click();
}
@Override
public void e_AddPetSuccessfully() {
Date date = new Faker().date().past( 365 * 20, TimeUnit.DAYS);
SimpleDateFormat sdf;
sdf = new SimpleDateFormat("yyyy-MM-dd");
String birthData = sdf