Exploring Cloud-Based Testing with the Elastic Execution Grid

You know those regression packs that used to finish while you grabbed coffee? Are they now taking hours? And that testing box you requisitioned six months ago? Is it already maxed out? And do you find yourself complaining about how resources are idling 90% of the day?

Yes, it’s time to look at cloud-based testing. Which is exactly what I recently started doing. I wanted to find a testing solution that was fast, easy, and gave me flexible capacity. And one that took minimal effort for me to maintain.

My first trial was the Tricentis Elastic Execution Grid (E2G). In this article I’ll cover what it is, what it does, and what I thought.

What is the Elastic Execution Grid?

The Tricentis Elastic Execution Grid (E2G) is “a cloud-based environment where you can run and track tests over time.”

E2G lets you run tests on cloud infrastructure that spins up when you need to run your tests … and spins down when you don’t. With E2G you aren’t required to maintain your own testing infrastructure or pay for idling computing resources.

Note that E2G is flexible. It allows you to run your tests on your infrastructure or on Tricentis cloud resources. Team agents allow you to use your own infrastructure — whether that’s virtual, physical, or private cloud. In this article, however, I’m concerned only with the cloud agents that run on Tricentis resources. I want to be able to run my tests without providing my own hardware. In this article, I’ll explore these cloud agents, but don’t forget that E2G lets you choose to run on your own infrastructure as well.

E2G is framework-agnostic. You can use it with Tosca, but also with other frameworks like PowerShell. At its core, E2G is a control plane that:

1. Spins up short‑lived “agents” when a run is queued.

2. Routes each test to an agent that has the required tooling, then streams logs back in real-time.

I love this idea of zero-footprint testing. With the cloud agents, I can just define a few properties and E2G automatically provisions (and scales) the compute resources necessary to run my test suite.

How does the Elastic Execution Grid work?

Let’s look at the architecture to understand how E2G works. E2G consists of four components: agents, agent services, characteristics, and runners.

• Agents are the machines where you run your tests. They host the Agent Service and the Runner.

• Agent services receive the tests from the server and hand them to the Runner

• Runners are the actual test frameworks that execute the tests.

Characteristics define what applications your tests can work with. For example, if your tests work with MS Excel, your agent would have an “Excel” characteristic.

Here’s how it all works together:

1. Push your tests to E2G - You design tests in your framework (Tosca, PowerShell, open‑source tools) and push them to E2G. The service stores your artifacts, queues and runs.

2. Characteristics - Each E2G Agent has characteristics as defined above (Excel, SAP, specific browser versions, etc). When you tag a test with its characteristics, E2G dispatches it only to agents that match. This guarantees the right tooling is present without over‑provisioning every box.

3. Execute agents - The Agent Service pulls the next test and hands it to the Runner, which pretends to be a user and executes the test. The Runner inherits the host machine’s permissions, so whatever apps or files your tests touch must already exist (for example, Excel must be installed if the test writes to a spreadsheet).

4. Collect results - When execution finishes, the agent sends back logs, screenshots, and pass/fail status for your review.

E2G in Action

Let’s try this all out. Once you have your free trial you’re ready. It’s a pretty simple process.

To run a test playlist on a cloud agent, open the Agent Characteristics tab of the Playlist details for the playlist you want to run and check the “Cloud-agent” box. Tosca will know that you need a cloud agent provisioned.

Click to start your tests and Tosa will schedule them in the Test Run tab of Tosca. You can track the progress of your test suite from here. E2G will scale the cloud resources as needed—I don’t have to worry about any bottlenecks.

The tests shown here are some of the samples that were provisioned for my trial account.

E2G can also split a test suite across multiple agents to speed up runs. You can indicate tests that can run in parallel on the playlist. You can also mark tests as dependent on each other to force them to run in a certain sequence.

What happens when things go wrong?

One concern I have with cloud-based testing is debugging. While ephemeral testing infrastructure is easier to manage than dedicated testing machines, using it also means that debugging can be more difficult.

In E2G, it’s just like debugging local tests. When a playlist is sent, whether through cloud agents or team agents, the outcome is reported in the test run interface. Here you can see pass/fail, progress, and eventual outcome of the playlist.

You can click in here on a specific test and see details of why your tests passed or failed, including any logs or screen recordings.

The ability to click straight into the failing test case makes remediation just about the same as it usually is. I can see situations where I might need more detail … but as a back-up I can always run the tests on my local machine using Tosca’s personal agent.

How secure is E2G?

Another concern I have with cloud-based testing infrastructure is regulatory or policy requirements. Here’s what I found for E2G:

• Tricentis is SOC/ISO compatible.

• The VMs used for cloud testing are temporary. As soon as your tests are over, they are terminated. Since they no longer exist, they don’t exist as attack vectors or as a gateway to your infrastructure.

• All data is encrypted and transferred via HTTPS

• Uploads (test artifacts, results) are stored for 28 days, then deleted

My remaining questions

There are a couple areas where I still had concerns:

• You are default limited to 50 concurrent agents and 50MB artifacts. This could limit your ability to run a large number of tests in parallel or could limit the size of your data files. However, you can request a bump if you need more.

• I feel like there are still situations where I will want (or need?) to run a test locally. Just to have that minute control and information. Of course, I can just choose the personal agent with E2G and do just that! So this really is just me getting comfortable with cloud testing. We’ll see if I get more comfortable as time goes on.

Give it a try!

Cloud-based testing is here and I’m a fan. I don’t expect a magic bullet for all my problems—but I’m excited to finally have a way to manage spikes and wasted resources. Renting 20 agents for 10 minutes seems better than owning two agents forever. And E2G feels like a solid solution. I like the framework flexibility and ease of switching between cloud and local tests in one place. I look forward to exploring it more.

Have a really great day!