Selenium WebDriver

Selenium WebDriver is a powerful tool for automating web browser interactions. Nostos provides built-in functions to control browsers through Selenium, making it easy to write end-to-end tests for your web applications, automate repetitive browser tasks, or scrape dynamic web content. In this chapter, you'll learn how to use Nostos to drive a web browser programmatically.

What is Selenium?

Selenium WebDriver is an industry-standard tool for browser automation. It allows you to:

Open web pages in a real browser (Chrome, Firefox, etc.)
Find elements on the page using CSS selectors or other methods
Click buttons, fill forms, and interact with the page
Read text content from elements
Wait for elements to appear (useful for dynamic/JavaScript-heavy pages)
Take screenshots and verify page state

Nostos integrates with Selenium through ChromeDriver, providing a simple, high-level API that lets you write browser automation scripts in pure Nostos code.

Prerequisites

Before you can use Selenium with Nostos, you need to set up ChromeDriver:

1. Install Chrome Browser

Make sure you have Google Chrome installed on your system. Selenium will use this browser for automation.

2. Download ChromeDriver

ChromeDriver is a separate executable that Selenium uses to communicate with Chrome. Download it from chromedriver.chromium.org. Make sure to download the version that matches your Chrome browser version.

# Check your Chrome version (Help -> About Google Chrome)
# Download matching ChromeDriver and place it in your PATH

# On Linux/Mac:
chmod +x chromedriver
sudo mv chromedriver /usr/local/bin/

# On Windows:
# Add chromedriver.exe to a directory in your PATH

3. Start ChromeDriver

ChromeDriver runs as a local server that accepts commands from your Nostos scripts. Start it on port 4444:

chromedriver --port=4444

Keep this running in a terminal while you run your Selenium tests. You'll see log output as browsers are opened and commands are executed.

Tip: Headless Mode

By default, Selenium opens a visible browser window. For CI/CD pipelines or servers without displays, you can run Chrome in headless mode (no visible window). ChromeDriver will automatically use headless mode when the DISPLAY environment variable is not set.

The Selenium API

Nostos provides the following built-in Selenium functions. All functions are in the Selenium namespace:

Function	Description
`Selenium.connect(url)`	Connect to ChromeDriver at the given URL. Returns a driver handle.
`Selenium.goto(driver, url)`	Navigate the browser to the specified URL.
`Selenium.waitFor(driver, selector, timeout)`	Wait up to `timeout` milliseconds for an element matching the CSS selector to appear.
`Selenium.click(driver, selector)`	Click on the element matching the CSS selector.
`Selenium.text(driver, selector)`	Get the text content of the element matching the CSS selector.
`Selenium.type(driver, selector, text)`	Type text into an input element matching the CSS selector.
`Selenium.close(driver)`	Close the browser and end the session.

Your First Selenium Script

Let's start with a simple script that opens a web page and reads some text:

# my_first_selenium.nos
# Run with: ./target/release/nostos my_first_selenium.nos

main() = {
    println("Starting Selenium test...")

    # Connect to ChromeDriver (must be running on port 4444)
    driver = Selenium.connect("http://localhost:4444")
    println("Connected to ChromeDriver!")

    # Navigate to a website
    Selenium.goto(driver, "https://example.com")
    println("Navigated to example.com")

    # Wait for the heading to appear (up to 5 seconds)
    Selenium.waitFor(driver, "h1", 5000)

    # Get the text of the heading
    heading = Selenium.text(driver, "h1")
    println("Page heading: " ++ heading)

    # Close the browser
    Selenium.close(driver)
    println("Done!")

    0
}

Running this script will:

Open a Chrome browser window
Navigate to example.com
Wait for the <h1> element to appear
Print the heading text ("Example Domain")
Close the browser

Understanding CSS Selectors

Selenium uses CSS selectors to find elements on the page. Here are the most common patterns:

# By tag name
Selenium.text(driver, "h1")              # First <h1> element

# By ID (use # prefix)
Selenium.click(driver, "#submit-button") # Element with id="submit-button"

# By class (use . prefix)
Selenium.text(driver, ".error-message")  # Element with class="error-message"

# By data attribute (very useful for testing!)
Selenium.click(driver, "[data-testid='login-btn']")

# Combining selectors
Selenium.text(driver, "div.container h2")  # <h2> inside div.container
Selenium.click(driver, "form#login button[type='submit']")

# By attribute
Selenium.type(driver, "input[name='email']", "user@example.com")

Best Practice: Use data-testid Attributes

For robust tests, add data-testid attributes to your HTML elements. This decouples your tests from styling and structure changes. Nostos provides convenient named parameters for this:

# In your Nostos RWeb app - use dataTestid and dataAction parameters
button("Place Order", dataAction: "submit", dataTestid: "submit-order")
span("$99.99", dataTestid: "total-price")
div([...], dataTestid: "cart-items")

# In your Nostos test - select by data-testid
Selenium.click(driver, "[data-testid='submit-order']")
price = Selenium.text(driver, "[data-testid='total-price']")

Waiting for Elements

Modern web applications often load content dynamically with JavaScript. The Selenium.waitFor function is crucial for handling this:

# Wait up to 5 seconds for an element to appear
Selenium.waitFor(driver, "[data-testid='results']", 5000)

# Now it's safe to interact with the element
results = Selenium.text(driver, "[data-testid='results']")
println("Search results: " ++ results)

The timeout is specified in milliseconds. If the element doesn't appear within the timeout, an error is raised.

For elements that take time to update after an action (like clicking a button), you can also use the sleep function for simple delays:

# Click a button that triggers an AJAX update
Selenium.click(driver, "[data-testid='load-more']")

# Wait a bit for the update to complete
sleep(500)  # 500 milliseconds

# Now read the updated content
content = Selenium.text(driver, "[data-testid='item-list']")

Complete Test Example: Counter App

Let's write a complete test for a simple counter web application. First, here's the counter app (using Nostos RWeb):

# counter_app.nos - A simple counter web app
use stdlib.rweb
use stdlib.rhtml

reactive Counter = { value: Int }

session() = {
    counter = Counter(0)

    (
        # Render function
        () => RHtml(div([
            h1("Counter Demo"),
            component("display", () => RHtml(
                div([
                    span("Count: "),
                    span(show(counter.value))
                ], dataTestid: "count")
            )),
            div([
                button("-", dataAction: "decrement", dataTestid: "decrement"),
                button("+", dataAction: "increment", dataTestid: "increment"),
                button("Reset", dataAction: "reset", dataTestid: "reset")
            ])
        ])),

        # Action handler
        (action, params) => match action {
            "increment" -> { counter.value = counter.value + 1 }
            "decrement" -> { counter.value = counter.value - 1 }
            "reset" -> { counter.value = 0 }
            _ -> ()
        }
    )
}

main() = startRWeb(8095, "Counter Demo", session)

Now let's write a comprehensive Selenium test for this counter:

# counter_test.nos - Selenium tests for the counter app
#
# To run:
#   1. Start ChromeDriver: chromedriver --port=4444
#   2. Start the counter app: ./nostos counter_app.nos
#   3. Run the test: ./nostos counter_test.nos

# Helper function to run a test and report results
runTest(name, got, expected) = {
    println("Test: " ++ name)
    println("  Got: " ++ got)
    result = String.contains(got, expected)
    if result then println("  [PASS]") else println("  [FAIL] Expected: " ++ expected)
    if result then 1 else 0
}

main() = {
    println("=== Counter App Selenium Tests ===")
    println("")

    # Connect to ChromeDriver
    println("Connecting to ChromeDriver...")
    driver = Selenium.connect("http://localhost:4444")
    println("Connected!")

    # Navigate to the counter app
    println("Navigating to counter app...")
    Selenium.goto(driver, "http://localhost:8095")

    # Wait for the page to load
    Selenium.waitFor(driver, "[data-testid='count']", 5000)
    println("Page loaded!")
    println("")

    # Test 1: Initial count should be 0
    text1 = Selenium.text(driver, "[data-testid='count']")
    t1 = runTest("Initial count should be 0", text1, "Count: 0")

    # Test 2: Increment should increase count
    println("")
    Selenium.click(driver, "[data-testid='increment']")
    sleep(300)
    text2 = Selenium.text(driver, "[data-testid='count']")
    t2 = runTest("After increment", text2, "Count: 1")

    # Test 3: Multiple increments
    println("")
    Selenium.click(driver, "[data-testid='increment']")
    sleep(200)
    Selenium.click(driver, "[data-testid='increment']")
    sleep(200)
    Selenium.click(driver, "[data-testid='increment']")
    sleep(200)
    Selenium.click(driver, "[data-testid='increment']")
    sleep(300)
    text3 = Selenium.text(driver, "[data-testid='count']")
    t3 = runTest("After 4 more increments", text3, "Count: 5")

    # Test 4: Decrement should decrease count
    println("")
    Selenium.click(driver, "[data-testid='decrement']")
    sleep(300)
    text4 = Selenium.text(driver, "[data-testid='count']")
    t4 = runTest("After decrement", text4, "Count: 4")

    # Test 5: Reset should set count to 0
    println("")
    Selenium.click(driver, "[data-testid='reset']")
    sleep(300)
    text5 = Selenium.text(driver, "[data-testid='count']")
    t5 = runTest("After reset", text5, "Count: 0")

    # Close the browser
    println("")
    println("Closing browser...")
    Selenium.close(driver)

    # Print summary
    passed = t1 + t2 + t3 + t4 + t5
    println("")
    println("=== Test Summary ===")
    println("Passed: " ++ show(passed) ++ " / 5")

    # Return 0 if all tests passed, 1 otherwise
    if passed == 5 then 0 else 1
}

Running the test produces output like:

=== Counter App Selenium Tests ===

Connecting to ChromeDriver...
Connected!
Navigating to counter app...
Page loaded!

Test: Initial count should be 0
  Got: Count: 0
  [PASS]

Test: After increment
  Got: Count: 1
  [PASS]

Test: After 4 more increments
  Got: Count: 5
  [PASS]

Test: After decrement
  Got: Count: 4
  [PASS]

Test: After reset
  Got: Count: 0
  [PASS]

Closing browser...

=== Test Summary ===
Passed: 5 / 5

Example: Testing a Shopping Cart

Here's a more complex example testing a shopping cart with multiple items, quantities, and discounts:

# cart_test.nos - Shopping cart integration tests

runTest(name, got, expected) = {
    println("Test: " ++ name)
    println("  Got: " ++ got)
    result = String.contains(got, expected)
    if result then println("  [PASS]") else println("  [FAIL] Expected: " ++ expected)
    if result then 1 else 0
}

main() = {
    println("=== Shopping Cart Selenium Tests ===")
    println("")

    driver = Selenium.connect("http://localhost:4444")
    Selenium.goto(driver, "http://localhost:8097")
    Selenium.waitFor(driver, "[data-testid='qty-a']", 5000)
    println("Cart app loaded!")
    println("")

    # Test 1: Initial quantity is 0
    t1 = runTest("Initial qty A",
                 Selenium.text(driver, "[data-testid='qty-a']"), "0")

    # Test 2: Add Item A twice (2 x $10 = $20)
    println("")
    Selenium.click(driver, "[data-testid='inc-a']")
    sleep(300)
    Selenium.click(driver, "[data-testid='inc-a']")
    sleep(300)
    t2 = runTest("Qty A after 2 increments",
                 Selenium.text(driver, "[data-testid='qty-a']"), "2")

    # Test 3: Check subtotal A
    println("")
    t3 = runTest("Subtotal A (2 x $10)",
                 Selenium.text(driver, "[data-testid='subtotal-a']"), "$20")

    # Test 4: Add Item B three times (3 x $25 = $75)
    println("")
    Selenium.click(driver, "[data-testid='inc-b']")
    sleep(300)
    Selenium.click(driver, "[data-testid='inc-b']")
    sleep(300)
    Selenium.click(driver, "[data-testid='inc-b']")
    sleep(300)
    t4 = runTest("Subtotal B (3 x $25)",
                 Selenium.text(driver, "[data-testid='subtotal-b']"), "$75")

    # Test 5: Check total subtotal ($20 + $75 = $95)
    println("")
    t5 = runTest("Total subtotal",
                 Selenium.text(driver, "[data-testid='subtotal']"), "$95")

    # Test 6: Apply 50% discount ($95 * 50% = $47.50, rounds to $47)
    println("")
    Selenium.click(driver, "[data-testid='discount-50']")
    sleep(500)
    t6 = runTest("Total with 50% discount",
                 Selenium.text(driver, "[data-testid='total']"), "$47")

    # Test 7: Clear cart
    println("")
    Selenium.click(driver, "[data-testid='clear-cart']")
    sleep(500)
    t7 = runTest("Subtotal after clear",
                 Selenium.text(driver, "[data-testid='subtotal']"), "$0")

    println("")
    println("Closing browser...")
    Selenium.close(driver)

    passed = t1 + t2 + t3 + t4 + t5 + t6 + t7
    println("")
    println("=== Test Summary ===")
    println("Passed: " ++ show(passed) ++ " / 7")

    if passed == 7 then 0 else 1
}

Typing Text into Forms

Use Selenium.type to enter text into input fields:

# Example: Testing a login form
main() = {
    driver = Selenium.connect("http://localhost:4444")
    Selenium.goto(driver, "http://localhost:8080/login")
    Selenium.waitFor(driver, "input[name='email']", 5000)

    # Type into form fields
    Selenium.type(driver, "input[name='email']", "user@example.com")
    Selenium.type(driver, "input[name='password']", "secretpassword")

    # Submit the form
    Selenium.click(driver, "button[type='submit']")

    # Wait for redirect and verify login success
    Selenium.waitFor(driver, "[data-testid='welcome-message']", 5000)
    welcome = Selenium.text(driver, "[data-testid='welcome-message']")
    println("Welcome message: " ++ welcome)

    Selenium.close(driver)
    0
}

Error Handling

Selenium operations can fail for various reasons: element not found, timeout exceeded, browser crashed, etc. You can use Nostos's error handling to deal with these situations:

# Example with error handling
main() = {
    println("Attempting to connect to ChromeDriver...")

    result = try {
        driver = Selenium.connect("http://localhost:4444")
        Selenium.goto(driver, "http://localhost:8080")

        # Try to find an element that might not exist
        Selenium.waitFor(driver, "[data-testid='optional-element']", 2000)
        text = Selenium.text(driver, "[data-testid='optional-element']")

        Selenium.close(driver)
        Ok(text)
    } catch e {
        Err(e)
    }

    match result {
        Ok(text) -> println("Found element with text: " ++ text)
        Err(e) -> println("Error occurred: " ++ e)
    }

    0
}

Organizing Your Tests

For larger test suites, organize your tests into separate files and create helper modules:

# test_helpers.nos - Reusable test utilities

# Standard test runner with formatted output
runTest(name, got, expected) = {
    println("Test: " ++ name)
    println("  Got: " ++ got)
    result = String.contains(got, expected)
    if result then println("  [PASS]") else println("  [FAIL] Expected: " ++ expected)
    if result then 1 else 0
}

# Run a test with exact match
runExactTest(name, got, expected) = {
    println("Test: " ++ name)
    println("  Got: " ++ got)
    result = got == expected
    if result then println("  [PASS]") else println("  [FAIL] Expected exactly: " ++ expected)
    if result then 1 else 0
}

# Create a fresh browser session
withBrowser(url, testFn) = {
    driver = Selenium.connect("http://localhost:4444")
    Selenium.goto(driver, url)
    Selenium.waitFor(driver, "body", 10000)

    result = testFn(driver)

    Selenium.close(driver)
    result
}

Then use the helpers in your test files:

# login_tests.nos
use test_helpers

main() = withBrowser("http://localhost:8080", driver => {
    println("=== Login Tests ===")

    # Navigate to login page
    Selenium.click(driver, "[data-testid='login-link']")
    Selenium.waitFor(driver, "[data-testid='login-form']", 5000)

    # Test empty form submission
    Selenium.click(driver, "[data-testid='submit']")
    sleep(300)
    t1 = runTest("Shows error for empty form",
                 Selenium.text(driver, "[data-testid='error']"),
                 "Email is required")

    # Test invalid email
    Selenium.type(driver, "input[name='email']", "notanemail")
    Selenium.click(driver, "[data-testid='submit']")
    sleep(300)
    t2 = runTest("Shows error for invalid email",
                 Selenium.text(driver, "[data-testid='error']"),
                 "Invalid email format")

    println("")
    println("Passed: " ++ show(t1 + t2) ++ " / 2")

    if t1 + t2 == 2 then 0 else 1
})

Best Practices

1. Use Stable Selectors

# GOOD: Stable, testing-specific selectors
Selenium.click(driver, "[data-testid='add-to-cart']")

# BAD: Brittle selectors that break with styling changes
Selenium.click(driver, "div.product-card > div.actions > button.btn-primary")
Selenium.click(driver, "button:nth-child(3)")

2. Always Wait for Elements

# GOOD: Wait for dynamic content
Selenium.click(driver, "[data-testid='load-data']")
Selenium.waitFor(driver, "[data-testid='data-table']", 10000)
data = Selenium.text(driver, "[data-testid='data-table']")

# BAD: Assumes content is immediately available
Selenium.click(driver, "[data-testid='load-data']")
data = Selenium.text(driver, "[data-testid='data-table']")  # Might fail!

3. Clean Up After Tests

# GOOD: Always close the browser, even on error
main() = {
    driver = Selenium.connect("http://localhost:4444")

    result = try {
        # ... run tests ...
        0
    } catch e {
        println("Test error: " ++ e)
        1
    }

    # Always clean up
    Selenium.close(driver)
    result
}

4. Use Meaningful Delays

# GOOD: Use waitFor when possible, short sleeps for animations
Selenium.waitFor(driver, "[data-testid='modal']", 5000)
sleep(300)  # Wait for animation to complete
Selenium.click(driver, "[data-testid='modal-close']")

# BAD: Long arbitrary sleeps
sleep(5000)  # Why 5 seconds? Is this enough? Too much?

Running Tests in CI/CD

To run Selenium tests in a CI/CD pipeline, you'll need ChromeDriver running in headless mode. Here's a typical setup:

# .github/workflows/selenium-tests.yml
name: Selenium Tests

on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3

      - name: Install Chrome and ChromeDriver
        run: |
          sudo apt-get update
          sudo apt-get install -y google-chrome-stable
          CHROME_VERSION=$(google-chrome --version | grep -oE '[0-9]+' | head -1)
          wget https://chromedriver.storage.googleapis.com/LATEST_RELEASE_$CHROME_VERSION -O /tmp/chromedriver_version
          wget https://chromedriver.storage.googleapis.com/$(cat /tmp/chromedriver_version)/chromedriver_linux64.zip
          unzip chromedriver_linux64.zip
          sudo mv chromedriver /usr/local/bin/

      - name: Start ChromeDriver
        run: chromedriver --port=4444 &

      - name: Start application
        run: ./nostos my_app.nos &

      - name: Wait for app to start
        run: sleep 5

      - name: Run Selenium tests
        run: ./nostos tests/selenium_tests.nos

Summary

In this chapter, you learned how to:

Set up ChromeDriver for browser automation
Connect to ChromeDriver and navigate to web pages
Find elements using CSS selectors
Wait for elements to appear on dynamic pages
Click elements, type text, and read content
Write comprehensive end-to-end tests
Organize tests and follow best practices

Quick Reference

Selenium.connect(url) - Connect to ChromeDriver

Selenium.goto(driver, url) - Navigate to URL

Selenium.waitFor(driver, selector, ms) - Wait for element

Selenium.click(driver, selector) - Click element

Selenium.text(driver, selector) - Get element text

Selenium.type(driver, selector, text) - Type into input

Selenium.close(driver) - Close browser

Previous: Logging End of Tutorial