β

Real Browser Integration Testing with Selenium Sta

Scott Hanselman's Blog 8 阅读

I find your lack of tests disturbing Buckle up kids, this is nuts and I'm probably doing it wrong. ;) And it's 2am and I wrote this fast. I'll come back tomorrow and fix the spelling.

I want to have lots of tests to make sure my new podcast site is working well. As mentioned before, I've been updating the site to ASP.NET Core 2.1.

Here's some posts if you want to catch up:

I've been doing my testing with XUnit and I want to test in layers.

Basic Unit Testing

Simply create a Razor Page's Model in memory and call OnGet or WhateverMethod. At this point you are NOT calling Http, there is no WebServer.

public IndexModel pageModel;



public IndexPageTests()

{

    var testShowDb = new TestShowDatabase();

    pageModel = new IndexModel(testShowDb);

}



[Fact]

public async void MainPageTest()

{

    // FAKE HTTP GET "/"

    IActionResult result = await pageModel.OnGetAsync(null, null);



    Assert.NotNull(result);

    Assert.True(pageModel.OnHomePage); //we are on the home page, because "/"

    Assert.Equal(16, pageModel.Shows.Count()); //home page has 16 shows showing

    Assert.Equal(620, pageModel.LastShow.ShowNumber); //last test show is #620

} 

Moving out a layer...

In-Memory Testing with both Client and Server using WebApplicationFactory

Here we are starting up the app and calling it with a client, but the "HTTP" of it all is happening in memory/in process. There are no open ports, there's no localhost:5000. We can still test HTTP semantics though.

public class TestingFunctionalTests : IClassFixture<WebApplicationFactory<Startup>>

{

    public HttpClient Client { get; }

    public ServerFactory<Startup> Server { get; }



    public TestingFunctionalTests(ServerFactory<Startup> server)

    {

        Client = server.CreateClient();

        Server = server;

    }



    [Fact]

    public async Task GetHomePage()

    {

        // Arrange & Act

        var response = await Client.GetAsync("/");



        // Assert

        Assert.Equal(HttpStatusCode.OK, response.StatusCode);

    }

    ...

}

Testing with a real Browser and real HTTP using Selenium Standalone and Chrome

THIS is where it gets interesting with ASP.NET Core 2.1 as we are going to fire up both the complete web app, talking to the real back end (although it could talk to a local test DB if you want) as well as a real headless version of Chrome being managed by Selenium Standalone and talked to with the WebDriver. It sounds complex, but it's actually awesome and super useful.

First I add references to Selenium.Support and Selenium.WebDriver to my Test project:

dotnet add reference "Selenium.Support"

dotnet add reference "Selenium.WebDriver"

Make sure you have node and npm then you can get Selenium Standalone like this:

npm install -g selenium-standalone@latest

selenium-standalone install

Chrome is being controlled by automated test software Selenium, to be clear, puts your browser on a puppet's strings. Even Chrome knows it's being controlled! It's using the (soon to be standard, but clearly defacto standard) WebDriver protocol. Imagine if your browser had a localhost REST protocol where you could interrogate it and click stuff! I've been using Selenium for over 11 years. You can even test actual Windows apps (not in the browser) with WinAppDriver/Appium but that's for another post.

Now for this part, bear with me because my ServerFactory class I'm about to make is doing two things. It's setting up my ASP.NET Core 2. 1 app and actually running it so it's listening on https://localhost:5001 . It's assuming a few things that I'll point out. It also (perhaps questionable) is launching Selenium Standalone from within its constructor. Questionable, to be clear, and there's others ways to do this, but this is VERY simple.

If it offends you, remembering that you do need to start Selenium Standalone with "selenium-standalone start" you could do it OUTSIDE your test in a script.

Perhaps do the startup/teardown work in a PowerShell or Shell script. Start it up, save the process id, then stop it when you're done. Note I'm also doing checking code coverage here with Coverlet but that's not related to Selenium - I could just "dotnet test."

#!/usr/local/bin/powershell

$SeleniumProcess = Start-Process "selenium-standalone" -ArgumentList "start" -PassThru

dotnet test /p:CollectCoverage=true /p:CoverletOutputFormat=lcov /p:CoverletOutput=./lcov .\hanselminutes.core.tests

Stop-Process -Id $SeleniumProcess.Id

Here my SeleniumServerFactory is getting my Browser and Server ready.

SIDEBAR NOTE: I want to point out that this is NOT perfect and it's literally the simplest thing possible to get things working. It's my belief, though, that there are some problems here and that I shouldn't have to fake out the "new TestServer" in CreateServer there. While the new WebApplicationFactory is great for in-memory unit testing, it should be just as easy to fire up your app and use a real port for things like Selenium testing. Here I'm building and starting the IWebHostBuilder myself (!) and then making a fake TestServer only to satisfy the CreateServer method, which I think should not have a concrete class return type. For testing, ideally I could easily get either an "InMemoryWebApplicationFactory" and a "PortUsingWebApplicationFactory" (naming is hard). Hopefully this is somewhat clear and something that can be easily adjusted for ASP.NET Core 2.1.x.

My app is configured to listen on both http://localhost:5000 and https://localhost:5001 , so you'll note where I'm getting that last value (in an attempt to avoid hard-coding it). We also are sure to stop both Server and Brower in Dispose() at the bottom.

public class SeleniumServerFactory<TStartup> : WebApplicationFactory<Startup> where TStartup : class

{

    public string RootUri { get; set; } //Save this use by tests



    Process _process;

    IWebHost _host;



    public SeleniumServerFactory()

    {

        ClientOptions.BaseAddress = new Uri("https://localhost"); //will follow redirects by default



        _process = new Process() {

            StartInfo = new ProcessStartInfo {

                FileName = "selenium-standalone",

                Arguments = "start",

                UseShellExecute = true

            }

        };

        _process.Start();

    }



    protected override TestServer CreateServer(IWebHostBuilder builder)

    {

        //Real TCP port

        _host = builder.Build();

        _host.Start();

        RootUri = _host.ServerFeatures.Get<IServerAddressesFeature>().Addresses.LastOrDefault(); //Last is https://localhost:5001!



        //Fake Server we won't use...this is lame. Should be cleaner, or a utility class

        return new TestServer(new WebHostBuilder().UseStartup<TStartup>());

    }



    protected override void Dispose(bool disposing) 

    {

        base.Dispose(disposing);

        if (disposing) {

            _host.Dispose();

            _process.CloseMainWindow(); //Be sure to stop Selenium Standalone

        }

    }

}

But what does a complete series of tests look like? I have a Server, a Browser, and an (theoretically optional) HttpClient. Focus on the Browser and Server.

At the point when a single test starts, my site is up (the Server) and an invisible headless Chrome (the Browser) is actually being puppeted with local calls via WebDriver. All this is hidden from to you - if you want. You can certainly see Chrome (or other browsers) get automated, but what's nice about Selenium Standalone with hidden/headless Browser testing is that my unit tests now also include these complete Integration Tests and can run as part of my Continuous Integration Build.

Again, layers. I test classes, then move out and test Http Request/Response interactions, and finally the site is up and I'm making sure I can navigate, that data is loading. I'm automating the "smoke tests" that I used to do myself! And I can make as many of this a I'd like now that the scaffolding work is done.

public class SeleniumTests : IClassFixture<SeleniumServerFactory<Startup>>, IDisposable

{

    public SeleniumServerFactory<Startup> Server { get; }

    public IWebDriver Browser { get; }

    public HttpClient Client { get; }

    public ILogs Logs { get; }



    public SeleniumTests(SeleniumServerFactory<Startup> server)

    {

        Server = server;

        Client = server.CreateClient(); //weird side effecty thing here. This call shouldn't be required for setup, but it is.



        var opts = new ChromeOptions();

        opts.AddArgument("--headless"); //Optional, comment this out if you want to SEE the browser window

        opts.SetLoggingPreference(OpenQA.Selenium.LogType.Browser, LogLevel.All);



        var driver = new RemoteWebDriver(opts);

        Browser = driver;

        Logs = new RemoteLogs(driver); //TODO: Still not bringing the logs over yet

    }



    [Fact]

    public void LoadTheMainPageAndCheckTitle()

    {

        Browser.Navigate().GoToUrl(Server.RootUri);

        Assert.StartsWith("Hanselminutes Technology Podcast - Fresh Air and Fresh Perspectives for Developers", Browser.Title);

    }



    [Fact]

    public void ThereIsAnH1()

    {

        Browser.Navigate().GoToUrl(Server.RootUri);



        var headerSelector = By.TagName("h1");

        Assert.Equal("HANSELMINUTES PODCAST\r\nby Scott Hanselman", Browser.FindElement(headerSelector).Text);

    }



    [Fact]

    public void KevinScottTestThenGoHome()

    {

        Browser.Navigate().GoToUrl(Server.RootUri + "/631/how-do-you-become-a-cto-with-microsofts-cto-kevin-scott");



        var headerSelector = By.TagName("h1");

        var link = Browser.FindElement(headerSelector);

        link.Click();

        Assert.Equal(Browser.Url.TrimEnd('/'),Server.RootUri); //WTF

    }



    public void Dispose()

    {

        Browser.Dispose();

    }

}

Here's a build, unit test/selenium test with code coverage actually running. I started running it from PowerShell. The black window in the back is Selenium Standalone doing its thing (again, could be hidden).

Two consoles, one with PowerShell running XUnit and one running Selenium

If I comment out the "--headless" line, I'll see this as Chrome is automated. Cool.

Chrome is loading my site and being automated

Of course, I can also run these in the .NET Core Test Explorer in either Visual Studio Code, or Visual Studio.

image

Great fun. What are your thoughts?


Sponsor: Check out JetBrains Rider: a cross-platform .NET IDE . Edit, refactor, test and debug ASP.NET, .NET Framework, .NET Core, Xamarin or Unity applications. Learn more and download a 30-day trial !



© 2018 Scott Hanselman. All rights reserved.

作者:Scott Hanselman's Blog
Scott Hanselman on Programming, User Experience, The Zen of Computers and Life in General
原文地址:Real Browser Integration Testing with Selenium Sta, 感谢原作者分享。

发表评论