Streamline Your Flutter CI/CD: Integration Testing with Firebase Test Lab on GitHub Actions (Part 1)

Amarjit MallickAmarjit Mallick
9 min read

Table of contents

Putting it Together: How Integration Testing Works

Imagine your Flutter app is like a big machine with many parts. Integration testing is like running a test on the entire machine, not just individual parts.

  • Regular tests (unit tests and widget tests): These are like checking each screw, bolt, and gear in the machine to make sure they're working properly on their own.

  • Integration testing: This is like running the machine and seeing if all the parts work together smoothly. You press buttons, turn knobs, and see if the machine does what it's supposed to do.

Integration tests help catch problems that might happen when different parts of your app interact. For instance, imagine a button that adds items to a shopping list. An integration test would press the button and check if the item is actually added to the list, making sure these two parts of the app are talking to each other correctly.

In short, integration testing helps you build a more reliable and well-oiled machine, which is your awesome Flutter app!

Why Firebase Test Lab Matters for Flutter Apps

Local testing with Flutter's integration_test package is great, but it has limitations:

  • Limited Devices: You're stuck with what you have (emulators or a few physical devices).

  • Manual & Time-Consuming: Running tests on each device takes time and effort.

  • Uncontrolled Network: Local network might not reflect real-world conditions.

Firebase Test Lab tackles these issues:

  • Wider Device Coverage: Tests run on a vast array of real devices with various configurations.

  • Scalability & Automation: Runs tests in parallel on multiple devices, saving time and integrating with CI pipelines.

  • Controlled Network Environment: Simulates real-world network conditions for better testing.

In short, Firebase Test Lab offers a more comprehensive testing experience by providing real-world device variety, automation, and controlled network environments.

Important Note:

The sample project uses:

  • Flutter 3.22.1

  • Dart 3.4

  • AGP 8.4.0

If you are using an old version of Flutter or some other version of AGP, you may face some error. So it is suggested that if you are creating a new project or working on an existing project, please upgrade your flutter version to 3.22.0 to avoid the errors.

Let's Setup the Flutter Project

Create a Flutter Project

Assuming you have already installed the Flutter SDK & IDE in your system (if not you can check the official documentation and then continue), open a command prompt, terminal, or PowerShell window depending on your operating system. Then navigate to the directory where you want to create your Flutter project.

Use the following command to create a new Flutter project named my_app:

flutter create my_app

This command will create the basic project structure for your Flutter app in the my_app directory.

Alternatively you can create a new flutter project directly from your IDE also.

Write the code

In this app we will be creating a login page and upon successful authentication we will navigate to the home page.

  • In the main.dart file we will write the code for Login Page and Home Page
import 'package:flutter/material.dart';

void main() {
  runApp(const MyApp());
}

class MyApp extends StatelessWidget {
  const MyApp({super.key});

  @override
  Widget build(BuildContext context) {
    return const MaterialApp(
      title: "Firebase Test Lab Sample",
      home: LoginPage(),
    );
  }
}

class LoginPage extends StatefulWidget {
  const LoginPage({super.key});

  @override
  State<LoginPage> createState() => _LoginPageState();
}

class _LoginPageState extends State<LoginPage> {
  final _formKey = GlobalKey<FormState>();
  final _emailController = TextEditingController();
  final _passwordController = TextEditingController();

  @override
  void dispose() {
    _emailController.dispose();
    _passwordController.dispose();
    super.dispose();
  }

  @override
  Widget build(BuildContext context) {
    return Scaffold(
      appBar: AppBar(
        title: const Text('Login'),
      ),
      body: Padding(
        padding: const EdgeInsets.all(16.0),
        child: Form(
          key: _formKey,
          child: Column(
            children: [
              TextFormField(
                key: const Key('email_field'),
                controller: _emailController,
                decoration: const InputDecoration(
                  labelText: 'Email',
                ),
                validator: (value) {
                  if (value!.isEmpty) {
                    return 'Please enter your email address';
                  } else if (!RegExp(
                          r"^[a-zA-Z0-9.a-zA-Z0-9.!#$%&'*+-/=?^_`{|}~]+@[a-zA-Z0-9]+\.[a-zA-Z]+")
                      .hasMatch(value)) {
                    return 'Please enter a valid email address';
                  }
                  return null;
                },
              ),
              const SizedBox(height: 16.0),
              TextFormField(
                key: const Key('password_field'),
                controller: _passwordController,
                decoration: const InputDecoration(
                  labelText: 'Password',
                ),
                obscureText: true,
                validator: (value) {
                  if (value!.isEmpty) {
                    return 'Please enter your password';
                  } else if (!RegExp(r"^(?=.*[A-Za-z])(?=.*\d)[A-Za-z\d]{8,}$")
                      .hasMatch(value)) {
                    return 'Password must be at least 8 characters long and contain at least one letter and one number';
                  }
                  return null;
                },
              ),
              const SizedBox(height: 16.0),
              ElevatedButton(
                onPressed: () async {
                  if (_formKey.currentState!.validate()) {
                    Navigator.push(
                      context,
                      MaterialPageRoute(
                        builder: (BuildContext context) {
                          return const HomePage();
                        },
                      ),
                    );
                  }
                },
                child: const Text('Login'),
              ),
            ],
          ),
        ),
      ),
    );
  }
}

class HomePage extends StatelessWidget {
  const HomePage({super.key});

  @override
  Widget build(BuildContext context) {
    return Scaffold(
      appBar: AppBar(),
      body: const Center(
        child: Text(
          key: Key('Home Page'),
          "Home Page",
        ),
      ),
    );
  }
}

Set up Integration Testing Dependencies

  • Open the pubspec.yaml file.

  • Under the dev_dependencies section of the pubspec.yaml file, add the integration_test package and save the file.

dev_dependencies:
  flutter_test:
    sdk: flutter
  integration_test:
    sdk: flutter
  • Open your terminal and navigate to your project directory (if you're not already there) and run the following command to install the newly added dependencies:
flutter pub get

Write the Integration Tests

  • Inside your project's root directory, create a new directory named integration_test. This directory will hold your integration test files.

  • Inside the integration_test directory, create a new Dart file (e.g., integration_test.dart). This file will contain your integration test code.

  • Inside the integration_test.dart file we will write our integration test code like this:

import 'package:flutter/material.dart';
import 'package:flutter_test/flutter_test.dart';

import 'package:my_app/main.dart';
import 'package:integration_test/integration_test.dart';

void main() {
  IntegrationTestWidgetsFlutterBinding.ensureInitialized();

  testWidgets('Login and navigate to home page', (WidgetTester tester) async {
    // Build the app and trigger a frame.
    await tester.pumpWidget(const MyApp());

    // Enter the email and password.
    await tester.enterText(
        find.byKey(const Key('email_field')), 'test@example.com');
    await tester.enterText(
        find.byKey(const Key('password_field')), 'Password123');

    // Tap the login button.
    await tester.tap(find.byType(ElevatedButton));
    // Wait for the home page to load.
    await tester.pumpAndSettle();

    // Verify that the home page is displayed.
    expect(find.text('Home Page'), findsOneWidget);
  });
}

Preparing to run tests from an APK file

To check if the app has the same behavior across multiple devices and configurations, we’re going to use Firebase Test Lab for. Also, to automate the process of running the tests whenever a new commit is pushed to a git repository, we’re going to use GitHub Actions.

In order to run the tests purely from an APK file, we need to do a little bit of a setup for the native Android project of the Flutter app. In the native Android world, there are instrumentation tests and we are basically going to hook up the multiplatform Flutter integration tests to run as if they were native instrumentation tests on Android.

  • Inside android/app/src create two new folders androidTest/java. Then create nested folders with names according to YOUR project’s package name. You will be able to find the package name inside the MainActivity.java or MainActivity.kt file. In our case the package name is com.example.my_app .

  • Now we will create MainActivityTest.java file. The path of the file will look like androidTest/java/com/example/my_app/MainActivityTest.java .

  • Inside this MainActivityTest.java file we will write some code as follows:

package com.example.my_app; // Change the package name according to your package name

import androidx.test.rule.ActivityTestRule;
import dev.flutter.plugins.integration_test.FlutterTestRunner;
import org.junit.Rule;
import org.junit.runner.RunWith;
import com.example.my_app.MainActivity; // The MainActivity should be imported from your package

@RunWith(FlutterTestRunner.class)
public class MainActivityTest {
 @Rule
 public ActivityTestRule<MainActivity> rule = new ActivityTestRule<>(MainActivity.class, true, false);
}

Note:

  • Update the package name

  • Import the MainActivity from the same package

After this your project structure should look like this:

  • Now open the app level build.gradle file (android/app/build.gradle) and update the code as follows:
plugins {
    id "com.android.application"
    id "kotlin-android"
    id "dev.flutter.flutter-gradle-plugin"
}

def localProperties = new Properties()
def localPropertiesFile = rootProject.file("local.properties")
if (localPropertiesFile.exists()) {
    localPropertiesFile.withReader("UTF-8") { reader ->
        localProperties.load(reader)
    }
}

def flutterVersionCode = localProperties.getProperty("flutter.versionCode")
if (flutterVersionCode == null) {
    flutterVersionCode = "1"
}

def flutterVersionName = localProperties.getProperty("flutter.versionName")
if (flutterVersionName == null) {
    flutterVersionName = "1.0"
}

android {
    namespace = "com.example.my_app"
    compileSdk = 34
    ndkVersion = "26.1.10909125"

    compileOptions {
        sourceCompatibility = JavaVersion.VERSION_1_8
        targetCompatibility = JavaVersion.VERSION_1_8
    }

    defaultConfig {
        applicationId = "com.example.my_app"
        minSdk = 21
        targetSdk = 34
        versionCode = flutterVersionCode.toInteger()
        versionName = flutterVersionName
        // Add these lines
        multiDexEnabled true
        testInstrumentationRunner "androidx.test.runner.AndroidJUnitRunner" 
    }

    buildTypes {
        release {
            signingConfig = signingConfigs.debug
        }
    }
}

flutter {
    source = "../.."
}

dependencies {
     // Add these dependencies
     implementation 'androidx.multidex:multidex:2.0.1'
     androidTestImplementation "androidx.test:runner:1.2.0"
     testImplementation 'junit:junit:4.13.2'
     androidTestImplementation 'androidx.test.espresso:espresso-core:3.2.0'
     androidTestImplementation 'androidx.test:rules:1.2.0'
     androidTestImplementation 'com.android.support.test:rules:1.0.2'
}

Build the apk files

  • Open the terminal in your project directory and run the following commands:
flutter clean
flutter pub get
pushd android
flutter build apk
./gradlew app:assembleAndroidTest
./gradlew app:assembleDebug -Ptarget=integration_test/integration_test.dart
popd

Note:

If you have your integration testing code in a file other thanintegration_test.dart, then in./gradlew app:assembleDebug -Ptarget=integration_test/integration_test.dartin this command after-Ptartget=provide your integration test file path

Once the command is run successfully you will get two apk files (inside the build folder of your project directory) at build/app/outputs/apk/debug/app-debug.apk and build/app/outputs/apk/androidTest/debug/app-debug-androidTest.apk paths respectively.

Let's Setup Firebase Testlab

  • If you already have a firebase project, you can use that or you can create a new project.

    [ Note: Running tests on Firebase Test Lab doesn't require integrating Firebase functionalities within your Flutter app. You only need a Firebase project to get started. Test Lab operates independently of whether you use other Firebase services. ]

  • Once your project setup is done, inside the firebase console go to the All Products section on the left panel and then select Test Lab from the options.

  • If you open the test lab for the first time you may see something like this (Otherwise you can skip this step):

  • now here we have to run a Robo test first. Click on the Browse button and select thebuild/app/outputs/apk/debug/app-debug.apkfile from the file picker and start the robo test. You can wait till the test is finished, or you can also cancel the robo test.

Run Instrumentation Test

  • Now if you open the Test Lab you will see the previous test(s). Click on "Run a test" button and select "Instrumentation". Then you will be asked to upload the two apk files that we have generated earlier.

  • Click on the Browse button for the App APK or AAB and select the build/app/outputs/apk/debug/app-debug.apk file and then click on the Browse button for the Test APK and select the build/app/outputs/apk/androidTest/debug/app-debug-androidTest.apk file.When the files are uploaded successfully, click on Continue.

  • If you haven't selected any devices, you will be asked to select the devices you want to run your tests on. Click on Customize and select the devices and then Confirm the selection.

  • Click on "Start 1 test".

The test will be started. If the test is fails for some reason you will see the status of the test as failed. Otherwise if the test is successful, you will see the test details and report as below:

  • Click on "Full run video" and you will be able see the options to view Test Issues, Logs, Videos, Performance metrics, Test Cases.

Congratulations!!! you have run your Flutter Integration Test successfully on Firebase Test Lab

Summary

This blog post teaches you how to run integration tests on your Flutter app using Firebase Test Lab. Integration testing helps catch bugs between different parts of your app. Firebase Test Lab offers a wider range of devices, automatic testing, and a controlled network environment compared to local testing.

In the Second Part of this blog post, we will learn how to automate this whole process using GitHub Actions.

If you found this blog helpful, let me know in the comments below! Your feedback is always appreciated.

Want to learn more about integration testing in Flutter? This blog post just scratches the surface. Feel free to ask questions or suggest future topics in the comments!

To know more about topics like this, you can follow me on LinkedIn & X.

Thank You

Happy Coding

0
Subscribe to my newsletter

Read articles from Amarjit Mallick directly inside your inbox. Subscribe to the newsletter, and don't miss out.

test labFirebase Test LabFlutterFirebaseGitHubgithub-actionsci-cdautomationautomation testing Real Device Testingflutter-firebase

Written by

Amarjit Mallick
Amarjit Mallick

I am a passionate Flutter developer, enthusiastic about creating delightful and performant mobile applications. My journey involves exploring the intricacies of UI/UX design and crafting seamless user experiences.