Overview: test with Silverstream

Silverstream is an early stage startup based in SF 🌁 We a re a small team of four co-founders. Behind every email or chat there’s one of us: we really care about your feedback on what we’re building, so please shot any idea or suggestions!

1️⃣ Accept the invitation

Access to the testing platform is currently invite-only. We're keeping numbers low to ensure we can hear feedback from everyone and truly understand how to improve this product.

The invitation from invitations@silverstream.ai should look like this:

Set a new password or create an account by logging in with Google. We currently don't support team or organization accounts—the service is for single users only. Feel free to share the account with a colleague if you both want visibility on the same tests. If multi-account is a priority, let us know through the Feature request board

2️⃣ Download the library

Install the silverriver library via pip. You can always find the most up-to-date beta version at this link. This library also installs the latest releases of Playwright and Chromium. It allows you to record a trace of the behavior you'd like to test in an instrumented Chromium session. The upload step will fail if the test wasn’t recorded with the latest version of the library.

pip install silverriver==0.1.37b7

3️⃣ Record the test

Now let’s record a flow we would like to test:

silverriver record URL [-o OUTPUT] [--upload]

URL: The URL of the webpage to trace (must start with http:// or https://) - If you encounter an error, check the url spelling or add “www”
o OUTPUT, -output OUTPUT: The output filename for the trace (default: 'silverriver_trace')
-upload: Upload the trace after recording

This command launches a browser session where you can demonstrate the intended behavior. The custom browser captures interactions in real-time. Don't forget to close the browser session to save the trace.

TRACE RECORDING BEST PRACTICES

Staging Url vs Production Url
During the test run, the agent will perform actions on the webpage like a real user. We suggest using the staging/development environment to test the flow, avoiding any impact on production.
Authenticated Flow
If you would like to test flows that are behind authentication (like in a web application), don't forget these two steps:
1. Create an account for the agent with the same permissions you'd like to have for the user.
2. Record the login step during the trace recording.
Keep it short
We recommend keeping the trace recording brief. An ideal flow could include actions like submitting a contact form, adding products to a cart, or adjusting settings. These concise scenarios effectively demonstrate key functionalities.

Please let us know if you have any doubts!

4️⃣ Upload the test & Check the user story

To add a new test, go back to the dashboard at dashboard.silverstream.ai. Click on the blue button in the top right “+ New Test”.

Upload the trace file you generated in the previous step. After uploading, click the "Generate test" button to initiate the user story generation.

From the trace, we generate a user story: it’s a set of instructions and context of the agent. You can review and edit the story. Don’t forget to click on “Let’s create this test”: this action triggers the generation a corresponding test sequence for the agent to follow.

5️⃣ Navigate Test Status

A test will go through several status. Let’s analyze them together:

🕓 Not Ready
Automatic test generation is in progress. This is a one-time process for each test. It’s usually very quick, but feel free to grab a coffee—it might take a few moments to complete.
🕓 Ready to run
The environment is ready and the test is ready to be launched: just click on “Run test”
Running
The run is in progress, and the agent is working to resolve the task. You can now view the start date and updated run duration.
✅ Success
The agent successfully completed the workflow. Everything worked as expected, and the acceptance criteria were met. You can review the logs anytime to see how the agent interacted with your web page.
Skipped
The agent identified instability in completing the task, and prevented disruptions by skipping the test. We try to always say upfront if a test is not feasible for the agent, to avoid flaky test. We recommend to ask help to a human friend for this test :)
❌ Failure
The agent couldn’t complete the task and it found a bug. This means it’s time to investigate the logs to identify what went wrong. See step 7 to understand how to read the logs.
❌ Error
It’s not you, it’s us.This is one of the rare cases where the agent encounters an issue during the run. Our team will prioritize this test to understand what caused the agent to crash.

6️⃣ Run and Rerun

You can run a test as many times as you want by clicking ‘Run Test.’ When the test is starting, you will see this popup in the bottom-right of your screen.

Currently, tests are only run on demand, but we’re working on implementing scheduled runs and triggers. 🕓

At the moment, each run overwrites the previous test results, and logs are not stored historically, so be sure to download the logs after each run. Run history is a work in progress; let us know if these features are high priority for you through the Feature Request board.

7️⃣ Download and Analyze logs

It’s time to see what happened during the run! Few moments after the run, the logs column status will shift from “Logs not available”

Let’s download the log by clicking on “Download Logs”.

It’s a playwright trace that you need to upload on trace.playwright.dev

If you are not familiar with Playwright Trace Viewer, here’s the complete official documentation and a video tutorial from Playwright (min 1:55s).

Here's a step-by-step guide:

Timeline Overview At the top, you’ll see a timeline showing key events, actions, and any errors during the test. Hover over individual events for a quick snapshot of each action, providing a visual summary of the test flow..
Action Tab
Explore the list of actions performed in sequence. Each action displays:
• Name of the action (e.g., click, fill, navigate)
• Duration (time taken for each action)
• Selector used for each interaction.
Click on any action to view more details, including DOM snapshots before and after the action.
Screenshot Panel If screenshot recording was enabled, a filmstrip will appear along the timeline. This feature provides snapshots that correspond to each significant action, helping you visualize changes as the test progresses
Log and Error Details The Log tab lists all actions sequentially, offering an in-depth view of what occurred at each step. The Errors tab highlights any failed actions, making it easy to locate problems
Console and Network Information
The Console tab shows console logs generated during the run, while the Network tab displays network requests made during each action, including timing and responses. This information is essential for diagnosing frontend and backend issues.