State Testing

What are State Tests?

AFib state tests allow you to validate the core of your app's functionality programatically.

In a sense, state tests substitute the things that are 'external' to your app -- the user, and external persistent data store and device APIs -- with test code and known test data respectively. Specifically:

1. AFib state tests replace UI widgets with code that validates and manipulates your app at the SPI level. Because the SPI exposes simplified data structures used in rendering the UI, it is useful for validating the business state of your app. Because the SPI exposes high-level event handlers for user-actions, it is useful for manipulating the state of your App.

   So, your test code effectively replaces the user by seeing the data the user sees, and manipulating the same event handlers the UI manipulates.

2. AFib state tests eliminate the startAsync stage of your queries. Instead, test code supplies results to your finishAsyncWithSuccess method (or finishAsyncWithError if desired). The results can be pulled from your static test data, or can be dynamically generated.

   So, your test code effectively replaces the external systems that add persistant data storage to your app.

Why are State Tests Useful?

State Tests make it easy to validate data

You SPI should present your UI with a simple view of the data it needs to render. That simple view of the data is typically very convenient for programatic validation.

Validating data at the UI level is often far less convenient. For example, when you validate the children of a ListView widget, you may have problems with a specific child widget being missing because it is off screen and not rendered, despite the fact that it is in the underlying list. This failure to render the off-screen widget occurs because ListView is efficiently implemented, but it is inconvenient for testing purposes.

To work around this issue, you need to scroll the list to a position where the widget you care about gets rendered. Scrolling involves animations and asynchronicity which introduce additional complexity. None of this complexity is present when you validate the data from an SPI that drives the creation of the list.

State Tests make it easy to manipulate your app

In my experience, the event handler methods exposed by your SPI are a very efficient level at which to manipulate your app's state. They allow you to drive your app just barely below the UI level, without requiring you to duplicate code, and without requiring you to do extraneous work looking up and manipulating widgets.

State Tests execute synchronously

In a properly designed AFib app, all your asynchonous logic will be encapsulated within the startAsync method of your queries. Because state tests do not invoke that method, they can be fully synchronous.

Synchronous tests are far easier to debug, and are far less susceptible to difficult to reproduce errors.

State Tests can be debugged via the UI

Although state tests eliminate the UI, AFib can still render the final state of a state test into a full UI. Although UI widgets are often inconvenient for programatic validation, they are purpose built to enable human beings to understand the state of your app. As a result, the ability to convert the terminal state of a state test into a UI can be extremely helpful in debugging.

Often, the source of a test failure is immediately apparent when you see the UI -- you aren't in the state you thought you were. Recognizing that fact in the debugger would have been possible, but far more time-consuming.

The Three Stages of a State Test: Define, Start, Execute

A state test has three stages.

First, it defines results for all the queries that will be executed during the state test. This step effectively replaces the external systems that your app would normally communicate with.

Second, it calls executeStartup(), which executes your StartupQuery, just as AFib does when your app starts in debug or production. Your StartupQuery typically executes additional queries, whose results determine the initial state of your app for the test. For example, your StartupQuery might execute a CheckSigninQuery, the results of which we specified in the first step.

If your state test returns a 'not signed in' result for that query, you app would likely display a signin page. If your state test returns an 'already signed in' result for that query, then you app would likely execute additional data retrieval queries and then show your app's home screen. As a result, after you call executeStartup() your app will be in some known state with a screen "showing", it will then wait for something to happen, just as your production app does.

Third, the state test makes a number of 'execute...' calls, gaining access to SPIs for screens and widgets, which it uses to validate and manipulate the state of your app. This step effectively replaces the user of your app, who would normally view your UI and then manipulate it.

Example: A very simple state test

The code below shows a very simple initial state test, loosely derived from the app-starter-signin-firebase project style:

void defineStartupStateTest(AFStateTestDefinitionContext definitions) {

  //--------------------------------------------
  // Preamble, just minor setup code
  //--------------------------------------------

  // you can access your test data easily in your test.
  final westCoastUser = definitions.find<User>(STFBTestDataID.userWestCoast);


  //--------------------------------------------
  // Start of the substantive test
  //--------------------------------------------

  definitions.addTest(STFBStateTestID.readyForLoginWestCoast, extendTest: null, body: (testContext) {
      // these shortcuts are maintained by afib and hide IDs and data types which would otherwise
      // be duplicated throughout your test.
      // Note that the testContext gets passed into the shortcuts object.  It manipulates the testContext
      // as you make calls below.
      final signinShortcuts = AFSIStateTestShortcuts(testContext);
      final signinScreen = signinShortcuts.createSigninScreen();

      final stfbShortcuts = STFBStateTestShortcuts(testContext);
      final homePageScreen = stfbShortcuts.createHomePageScreen();

      //-------------------------------------------
      // STEP 1: Define query responses
      //-------------------------------------------

      // the startup query has a result type of AFUnused, this is a shortcut for
      // supplying that.
      testContext.defineQueryResponseUnused<StartupQuery>();

      // we can specify a query result directly
      testContext.defineQueryResponseFixed<CheckSigninListenerQuery>(UserCredentialRoot.createNotSignedIn());

      // or by specifying a test data id.
      testContext.defineQueryResponseFixed<ReadOneUserListenerQuery>(STFBTestDataID.userWestCoast);

      // or we can dynamically define query result with code.
      testContext.defineQueryResponseDynamic<SigninQuery>(body: (context, query) {
        final email = query.email;
        if(email != westCoastUser.email) {
          context.onError((AFQueryError(message: "Please enter ${westCoastUser.email} as the email")));
        } else {
          context.onSuccess(UserCredentialRoot(
            userId: westCoastUser.id,
            token: '--', 
            storedEmail: westCoastUser.email
          ));
        }
      });

      //-------------------------------------------
      // STEP 2: Start the app by executing the StartupQuery
      //-------------------------------------------

      // This will execute the StartupQuery, its result will be AFUnused. It will in turn 
      // issue a CheckSigninListenerQuery.   The result for that will be not-signed in, as 
      // specified above. As a result, that query will redirect us to the signin page, 
      // which is where we will be if you uncommented the executeDebugStopHere() line after
      // this and navigate into this test in prototype mode.
      testContext.executeStartup();

      // testContext.executeDebugStopHere();


      //-------------------------------------------
      // STEP 3: Make execute calls to gain access to SPIs and validate/manipulate them.
      //-------------------------------------------

      // note that this line does not take us to the signin screen.  We must already be on the 
      // signin screen or this will produce an error.  Instead, it allows us to repeatedly
      // access that screen's SPI.  Note that this screen's SPI is actually from a third party
      // component - AFib Signin.
      signinScreen.executeScreen((e, screenContext) { 
        screenContext.executeBuild((spi) => spi.onChangedEmail(westCoastUser.email));
        screenContext.executeBuild((spi) => spi.onChangedPassword("test"));

        // if you uncomment this line, then when you enter this test in prototype mode, you will see that the
        // signin screen has been filled in, but the test will stop there.
        // screenContext.executeDebugStopHere();

        // when this line executes, it will execute the SigninQuery.   The test code for that 
        // query will execute, yielding a result that ultimately causes that query to redirect
        // us to the home page.
        screenContext.executeBuild((spi) => spi.onPressedSignin());

      });

      // again, this line does not take us to the home page, it will fail if we are not already on it.
      homePageScreen.executeScreen((e, screenContext) { 
        screenContext.executeBuild((spi) { 
          // here, we can use data from the SPI to validate the state of our app.
          // Although state tests don't validate that the firstName is showing up in the right
          // place in the UI, they do validate that any UI that refers to the activeUser is
          // going to find the expected value in the firstName field.  I find the vast majority
          // of my bugs are at this 'business logic' level of the app.
          e.expect(spi.activeUser.firstName, ft.equals(westCoastUser.firstName));
        });
      });
  });
}

Note that in the actual app-starter-signin-firebase project style, the substance of this test is divided among several tests using AFib's state test extension functionality (described below). Doing so makes the substance of this test much more concise. I recommend you start any signin-based app with either app-starter-signin or app-starter-signin-firebase, because they both provide a good initial test organization for handling various signin scenarios.

Running State Tests

Running state tests from the command line

State tests will automatically run from the command line when you run:

dart bin/xxx_afib.dart test

This command produces output which shows the full test id for all tests. You can run a specific state test by adding its ID after test above.

Running state tests from the prototype UI

As a state tests executes, it generates AFib states that fully describe a potential user interface, but it does not actually render that user interface. Attempting to render the user interface as the state test executes would require asynchronous execution and make the tests much more difficult to debug. But, rendering the UI for the terminal state of a state test is practical, and AFib can do that for you in prototype mode.

In prototype mode, you can click the "State Tests" section to view a list of all of your state tests. State tests appear here automatically. When you click on a state test, the entire state test will execute, then AFib will build the UI for its terminal state.

Adding State Tests to Your Prototype Mode Favorites

If you have specified/simulated results for all your queries, the resulting app is typically very deeply functional. You can navigate around and interact with and modify the state.

I do the vast majority of my development in state tests, which are much more functional than UI prototypes, but allow me to work from known test data. When I develop state tests, I am often simultaneously developing the test and developing an app state which will be useful for ongoing interactive development.

You can add a state test to your prototype mode favorites in lib/initialization/environments/prototype.dart.

Starting Directly into a State Test

If you will be prototyping in a single state test, you can start directly into it by choosing AFEnvironment.startupInStateTest in your xxx_config.g.dart file. Then, specify the specific state test to startup into in lib/initialization/environments/prototype.dart.

Creating State Tests

Generating a State Test

State tests are generally organized by an overall product area or workflow type. Despite the fact that a state test does not correspond to a specific class, we still use a class-like syntax for generating one for consistency with the other generate commands:

dart bin/xxx_afib.dart generate test SettingsStateTest

This will result in a state test file in the expected location:

lib/test/state_tests/settings_state_test.dart

Step 1: Defining Query Results

As I mentioned above, your first goal when creating a state test is to define responses to the queries which will execute during the course of the test. Note that one state test can extend another (described below), so you will often have a small number of root state tests which define most query results, and a larger number of leaf tests that manipulate and validate your app.

Specifying results from test data

For many apps, you will specify fixed results from your test data for the queries that occur immediately after signin. Doing so allows you to establish a known state for your app. For example, this line might initialize your app with a particularly large set of todo items:

  context.defineQueryResponseFixed<TodoItemsListenerQuery>(TDLETestDataID.longTodoList);

Note that the data associated with TDLETestDataID.longTodoList will be defined in your test_data.dart file, and will need to be consistent with the result data type expected by TodoItemsListenerQuery.

Specifying that a query returns no response, a null response, or an AFUnused

Sometimes a listener query starts, but initially recieves no response at all. You can use this syntax to handle that scenario.

    context.defineQueryResponseNone<UserSettingsListenerQuery>();

Other times, the result data type for a query may be nullable, and you may wish to invoke finishAsyncWithResponse with null. You can do that with this syntax:

    context.defineQueryResponseNull<UserSettingsListenerQuery>();

Finally, if your query does not have a result type, you can use the type AFUnused. You can force finishedAsyncWithResponse to be executed with an AFUnused value using:

    context.defineQueryResponseUnused<StartupQuery>();

Specifying Query Results Dynamically

When a state test is starting up, it usually makes sense to specify query responses from the test data, since you are trying to establish a well-understood test state that is useful for testing.

As you get further into your state tests, you will want to enter data dynamically through the user interface, and dynamically simulate query results in response to it. You can do this using the context.defineQueryResponseDynamic method. That method allows you to define code that substitutes for the startAsync method, simulating some version of what that method does in pracice.

For example, suppose a TodoList app has a screen which adds a TodoItem using a WriteTodoItemQuery. That query's startAsync method writes the item to a persistent data store, allocating a unique id for it, and then passes it to context.onSuccess and finishAsyncWithResponse. In a state test, startAsync never gets called, so we need to simulate its behavior in a state test, we can do so like this:

    context.defineQueryResponseDynamic<WriteTodoItemQuery>(body: (context, query) {
      // query is the query being executed.
      final item = query.item;
      var savedItem;
      if(item.isNewId) {
        // if the item is we need to create a unique test id for it.  I often do this by
        // taking some unique data in the item itself and turning it into a string id.
        final testId = item.createTestId();
        savedItem = item.copyWith(id: testId);
      } else {
        // if the item has already been saved, we won't changed its id, but we still need
        // to make a new copy of it, as that will be behavior of the real query.  In some cases,
        // preserving this 'new pointer' behavior might be important to getting the UI to actually 
        // update.
        savedItem = item.copyWith();
      }

      // note that the result of the query in question is specified here.  This syntax directly
      // parallels the normal context.onSuccess/context.onError synax found in startAsync.  
      // Essentially, this closer is just a test-mode implementation of startAsync.
      context.onSuccess(savedItem);
    });    

Specifying a dynamic update to a listener query

When you call a context.define... method, you are defining a universal response for that query. As we will see later, if you redefine the result of a query in a child state test, you override the response that was defined in its parent from the beginning of the state test.

Sometimes, you may want to simulate an update dynamically arriving for a listener query. You can do so with this call:

    context.executeInjectListenerQueryResponse<TodoListItemQueryListener>(TDLETestDataID.moreTodoItems);

Note that this call does not begin with 'define'. It will not override any previously defined static response to a query in a parent state test. Instead, it simulates a new response arriving asynchonously to a listener query at exactly this moment in the test.

Specifying a push to a listener query in response to a simple query

Occasionally, a simple query will not return a result directly, but will yield a response to a separate listener query (this is true of mobile device purchase APIs).

In that case, see the documentation for:

    context.defineDynamicCrossQueryResponse(...)

Running a live query

Sometimes, it may be useful to allow a query's startAsync method to run normally in a state test which you are executing in the UI as a development environment. For example, I sometimes do this when an app is loading constant, static JSON from a server. It can also be useful when dealing with streaming media from a server.

You can do so by calling:

   context.defineQueryResponseLive<LoadStaticHelpTopicJsonQuery>();

In this case, the full query will execute. If I am using this particular state test as a prototype environment, this live-query functionality can be quite useful.

Step 2: Starting Your App

When an AFib app starts, the first thing it does is execute your startup query. Up to this point in our state test, we have been defining future responses to queries, but nothing has actually been executed. When we have setup our context adequately, we are ready to start the app by running.

    context.executeStartup();

This call will execute our startup query, and any queries it executes. This is exactly what happens when your app starts in debug or production mode. However, the result values for the queries which execute will be those you defined in your state test, not those produced by running startAsync, which is not executed.

Step 3: Validating and Manipulating Your App

Now, you can begin validating and manipulating the state of your app.

Use the shortcut object to access screens, dialogs, widgets, etc

As shown above, AFib automatically maintains a 'shortcuts' class for you:

final stfbShortcuts = STFBStateTestShortcuts(testContext);
final homePageScreen = stfbShortcuts.createHomePageScreen();

Use executeScreen to begin manipulating a screen

As shown above, the executeScreen call does not move you to a screen, it just helps the test know what context you expect to be in. A section like this:

signinScreen.executeScreen((e, screenContext) { 
  screenContext.executeBuild((spi) => spi.onChangedEmail(westCoastUser.email));
  screenContext.executeBuild((spi) => spi.onChangedPassword("test"));
...

First, e is an execution context which is mainly used for e.expect. Remember to use e.expect rather than Flutter Test's native expect, as it yields nicer reporting on the command line.

Second, screenContext is a context useful for repeatedly building the SPI for the screen.

Use screenContext.executeBuild to validate and manipulate an SPI

In the example above, we use screenContext.executeBuild to repeatedly get access to the SPI. This is the same SPI value that AFib would have passed to your screen's buildWithSPI function. You can use it to validate the SPI-based data your UI code would have used to render the UI. You can also invoke an event handler on it, manipulating the state of your app.

Keep in mind that the code you are writing in the state test replaces the UI build and user with your test code. That code uses the SPI in a way that simulates when the UI build and user would have done.

Use execute...Build variants if you only need the SPI once

The executeScreen/executeBuild syntax is useful when you visit a screen and then access its SPI multiple times. Sometimes, however, you only need to do a single thing on a screen. For example, perhaps you have opened a drawer and just need to tap a single menu.

All the execute... functions on the shortcuts have a method suffixed with Build which gives you direct access to the SPI, eliminating the need to make a call to executeBuild, like this:

menuDrawer.executeDrawerBuild((e, spi) => spi.onPressedSettingsMenu());

Use e.expect to verify values

The correct use of expect is shown here:

import 'package:flutter_test/flutter_test.dart' as ft;
...

homePageScreen.executeScreen((e, screenContext) { 
  screenContext.executeBuild((spi) { 
    e.expect(spi.activeUser.firstName, ft.equals(westCoastUser.firstName));
  });
});

Shortcut for validating active screen.

Validating the active screen is pretty common, so we could also use:

    context.executeVerifyActiveScreenId(TODOScreenID.signin);

Validating the full state directly

Validating the data structures exposed via the SPI is usually sufficient.

However, you can use this syntax at the root of your state test definition to verify any part of the public state directly:

    context.executeVerifyState((e, verifyCtx) {
      // validate that the user is not signed in.
      final state = verifyCtx.accessComponentState<TODOState>();
      final signinState = state.signinState;
      e.expect(signinState.userId, ft.isNull);

      // validate that we are on the signin screen.
      e.expect(verifyCtx.route.activeScreenId, ft.equals(TODOScreenID.signin));
    });

Implementing State Tests in Detail

Warning: You must 'rebuild' the SPI on every change

When your app runs in production, every state change causes any impacted SPI to be re-built as part of the rendering process. Because SPIs and the route parameters and state views that they reference are immutable, you must always rebuild the SPI in your state tests after each state change.

For example, this code does not work, because it attempts to make two state changes on the same SPI:

      screenContext.executeBuild((spi) {
        // the route parameter's email and password are empty here, then the route parameter gets
        // revised with a valid email.  But that change won't propagate until a new SPI is built
        // with that revised route parameter.   The SPI is immutable.  It cannot be altered, only 
        // rebuilt with a new state.
        spi.onChangedEmail(user.email));

        // Because no rebuild occurred, this immutable SPI still has an empty the email value.
        // When the route parameter gets revised here, the revised copy with have an empty email
        // and will now have a non-empty password.
        spi.onChangedPassword("testpass"));      
      });

The spi.onChangedEmail will revise the route parameter containing an empty email and password, yielding a route parameter with a new email, and call context.updateRouteParam inside the SPI. The route parameter attached to the SPI is immutable, and will continue to have an empty email and password. In the real UI, that will cause the UI to re-render and the SPI to be rebuilt with the new route parameter containing the updated email value. Only then could the user change the password.

However, in the incorrect code above, the SPI is never rebuilt. Consequently, when spi.onChangedPassword is called, it will revise the original route parameter, which still has an empty email value. The upshot is that it will appear only your final change is applied.

In order to write state tests properly, you need to internalize that any time any state change occurs, a new build cycle must also occur, regenerating the SPI. This rebuild cycle happens automatically in the user interface. In a state test, you manually implement this build by calling screenContext.executeBuild after you make a change.

This version of the code above will work correctly, note that the SPI is rebuilt after each change:

      screenContext.executeBuild((spi) {
        spi.onChangedEmail(user.email));
        // note that right here, the spi does not yet contain the updated email value.  That value 
        // is immutable.
      });

      // but now we re-build the SPI after the change above, the new SPI has the updated email
      // value.  This is how the UI works in production.
      screenContext.executeBuild((spi) {
        // this re-built spi has the updated email from above, so now we can apply another change.
        spi.onChangedPassword("testpass"));      
      });

Testing Dialogs, Drawers and Bottom Sheets

From a state testing perspective, these three UI elements are no different from screens. If you interact with an SPI in a way that causes a dialog to appear, you can simply use the shortcut for that dialog to manipulate its SPI. For example:

    todoItemDialog.executeDialog((e, context) {
      final revisedName = "Goodbye!";
      context.executeBuild((spi) {
        // note that although you cannot make to state changes to an SPI, you can validate
        // an SPI and then make a change.
        e.expect(spi.todoName, ft.equals("Hello!"));
        spi.onEditTodoName(revisedName));
      });
      context.executeBuild((spi) {
        e.expect(spi.todoName, ft.equals(revisedName));
        spi.onPressedSave();
      });
    });

Testing AFib-aware Widgets

Widgets are different from other AFib UI components because they are not created by AFib itself. Instead, they are created as part of your UI rendering code. In state tests, your UI rendering code doesn't execute, requiring some special handling described below.

Widget shortcuts are attached to the screen they reside on

When you create a shortcut for a specific widget, it must be attached to a particular screen. Note that widgets are often created so that they can be shared across multiple screens. In that case, you may end up with one widget shortcut for each screen on which that widget resides.

  final listTodoItemsScreen = shortcuts.createListTodoItemsScreen();
  // note that when the shortcut is created, it takes the screen it resides on as a parameter.
  // however, this is not yet referring to a specific todo item.  Its just a shortcut used to 
  // reference any Todo item widget on that screen.
  final todoItemWidget = shortcuts.createEditTodoItemWidget(listTodoItemsScreen);

You will usually nest widgets within their screen's SPI scope

When working with screens, the launch parameter for the screen/dialog/etc is passed as part of the navigation action. The navigation action typically occurs within an SPI event handler, and is thus hidden from your state test code.

With widgets, the launch param is created as part of the UI rendering code, and that code is not executed as part of a state test. Consequently, you must specify the launch parameter that would have been created by the rendering code in your state test.

For example, your rendering code might look like this:

  ...
  return RegistrationDetailsWidget(
    launchParam: RegistrationDetailsWidgetRouteParam(
      userToEdit: userToEdit,
    )
  );
  ...

However, instead of embedding the launch parameter in the UI code, where it cannot be re-used, you should create it from the spi, like this:

  ...
  return RegistrationDetailsWidget(
    wid: XXXWidgetID.widgetRegistrationDetails,
    launchParam: spi.createRegistrationDetailsLaunchParam(userToEdit),
  );
  ...

Then, you call the SPI method that creates the launch parameter from your state test as well, by nesting access to the widget within access to the screen's SPI:

    listTodoItemsScreen.executeScreen((e, screenContext) {
      screenContext.executeBuild((spiScreen) { 
        // we do it inside the screen's SPI context because the screen SPI usually has a method
        // for creating the launch parameter.
        todoItemWidget.executeWidget(screenContext, launchParam: spiScreen.createRegistrationDetailsLaunchParam(userToEdit), body: (widgetContext) {
          widgetContext.executeBuild((spi) => spi.onChangedFirstName("Chris"));
        });
      });      

As I mentioned above, I prefer creating child route parameters as part of the screen's navigatePush function, passing them in to the children parameter of AFNavigatePushAction. If you do that, you will still need to specify a launch parameter in the UI code, but it will be an AFRouteParamRef referring to the existing child parameter.

In that case, the SPI's createRegistrationDetailsLaunchParam method will still exist, and will still be referenced from both the UI code and the state test, but its implementation will look like this:

screen's navigation, it might simply refer to the existing route parameter using AFRouteParamRef:

// we don't need a user to edit in this case, as it was already specified
// when the RegistrationDetailsWidgetRouteParam was created during the navigation
// to the parent screen
AFRouteParam createRegistrationDetailsLaunchParam() {
  return AFRouteParamRef(
    screenId: screenId,
    wid: XXXWidgetID.widgetRegistrationDetails,
    routeLocation: AFRouteLocation.screenHierarchy,
  );
}

Regardless, the specific details of how the launch parameter is configured are hidden from both the UI code and the state test by the SPI.

Extending an Existing State Test

One state test can extend another state test. This feature is very useful for building a hierarchy of state tests which do not need to repeat configuration steps, and for adding new tests with enhanced data without breaking previous tests that assume simpler data.

The exact meaning of one state test extending another is nuanced. It is important for you to understand in order to use state tests successfully.

1. An extended state test can override the query results specified in the parent test. So, for example, suppose the TODOStateTestID.alreadySignedInChris defines a return value with 10 todo items for the query ReadManyTodoItemsQuery. Note that the test below extends that initial test, and specifies a much larger set of todo items as a response to the ReadManyTodoItemsQuery:

   testDefs.addTest(TODOStateTestID.signedInChrisWith1000Todos, extendTest: TODOStateTestID.alreadySignedInChris, body: (testContext) {
   
     testContext.defineQueryFixedResponse<ReadManyTodoItemsQuery>(TODOTestDataID.todoItemsList1000);
     ...  
   

   When the TODOStateTestID.signedInChrisWith1000Todos executes the ReadManyTodoItemsQuery, it will get the larger set of todo items, not the 10 specified in the parent state test. When you re-define the query results in an extended test, you replace them from the beginning. However, any queries which were defined in the parent test and not-overridden will continue to have the same resuls as the parent test.

2. The executable portion of your tests runs linearly, starting with the executeStartup call in the root test, then any executable statements in that root test, then executable statements in the next child test down, until it eventually executes the statements in your leaf test.

3. Because the results returned by queries may be different than the parent tests originally anticipated, all 'expect' calls in the executable code for parent tests are ignored. Only the expect calls in your leaf test are active. Because AFib counts each successful expect call in its test output, this also yields more sensible success counts for extended tests.

Why Does Test Extension Work This Way?

I find the ability to override query results from the beginning while otherwise maintaining the original execution sequence (less e.expect) often allows me to generate interesting test states efficiently. For example:

1. If a root test defines all the query results necessary to populate the home page of your app (for example, a ReadOneUserQuery and a ReadTodoItemsListenerQuery), then you can easily create one child test which returns an 'already signed in' result in response to a CheckSigninQuery, causing your app to start from a populated home screen. You can create a second child test that returns a 'not signed in' result, causing your app to start from a signin screen. However, if you actually click 'signin' (either programatically, or in prototype mode), the query results from the root test will cause the signin process to work, and place you on the same populated home screen.

2. If a parent test defines a simple set of test data, and your app introduces new functionality which requires more complex data, then a child test can extend the parent test and override query results to return the more complex data. However, the parent test will continue to see its original test data, and as a result its validations won't be disturbed by the new, more complex data in the child test.

As you play with the state test extension mechanism yourself, it will probably become more clear how the extension mechanism makes your tests easier to write, more flexible and more maintainable.