GTAC 2014

FINRA Launches MSL at GTAC

GTAC 2014: Fire Away Sooner And Faster With MSL

Google Testing Automation Conference (GTAC) - Kirkland, WA

October 2014

Presented by Daniel Koo, Director of FINRA Development Services and Bryan Robbins,
Senior Developer for Development Services

Koo and Robbins present MSL, an open source client side integration testing tool that blasts through testing bottlenecks, bypassing the need for full system level end-to-end browser automation.

[Intro] >>Announcer: So our next speakers are Daniel Koo and Bryan Robbins from FINRA. They are also the inventors of a framework which they call MSL (“Missile”). And they're going to tell us how MSL has helped them improve UI testing at FINRA. >>Daniel Koo: All right, thanks for the introduction. So my name is Daniel Koo. We're here to talk about a tool that we open-sourced this year called MSL. It stands for Mock Service Layer. And, you know, we're very excited to come here and share about the tool. And, hopefully, all of you guys will find this interesting and, you know, you guys will use this when you guys go back. I know we talked a lot about mocking and, you know, moving fast and hermetic testing. And I think our talk will definitely resonate with those topics. So we're very excited. All right. So what am I trying to say with this slide? I think these are some of the trends that we are seeing in the industry today; right? Things like agile, faster time to market, continuous delivery, having high quality. All of these are great goals for software projects. And, of course, these are goals that we had at FINRA, too. But while trying to achieve these goals, we observed a few things. First, we were observing that devs were actually moving faster, delivering faster, but tests were sort of struggling, and they were becoming bottlenecks and moving slower. So, you know, definitely there were challenges while trying to achieve these goals. And FINRA had its own challenges, too. So brief introduction on FINRA. FINRA stands for Financial Industry Regulatory Authority. We pretty much oversee and regulate the majority of the securities firms doing business in the U.S. We also monitor the market data. More than 6 billion trades per day, more than 30 billion transactions per day, and more than 10 terabytes of data is what we analyze on a day-to-day basis. Our mission is to protect investors like us and promote market integrity. So, you know, the nature of our business and our mission brought these unique challenges to us. First, the visibility aspect. If we make a mistake and it happens in production, we could end up on the Wall Street Journal the next day. So we pay very close attention to the quality of our software. And as I mentioned before, the enormous amount of data that we ingest every day and, you know, the type of analysis that we do and the workflow that gets into these analyses just makes our systems very complex in nature and in turn, it makes our testing that much more challenging. So this is when I started to think about, okay, I want to achieve those goals, move fast, deliver fast to our customers while not compromising quality. And that's how MSL was born. So I'm going to hand it over to Bryan. He's going to talk about how our automation approach has evolved throughout the years and we'll dive into more details on MSL. >>Bryan Robbins: Cool. So this is another theme that we've already heard a few times here at the conference. This is our rendition of it. It's basically to say that if you had come to us two and a half, three years ago within our company and said, “How should I automate my tests?” our primary recommendation would have been Selenium WebDriver. Go and make functional, browser, full-stack automation for every feature and by the time you're done you'll have great coverage and great -- it's a great approach to testing, right? But as we start to move faster, those things are flaky. So I've got execution time on my Y axis. You could put flakiness there. You could put development effort there. You could put a lot of different variables there. And the more of these full stack UI tests, there's so many dependencies, it just becomes intractable. Again, that's a theme you've already heard. So about a year and a half ago, one emphasis -- we started putting in place and swapping out, so we'll replace some of these end-to-end tests, the ones that are trying to test back-end features, we'll just use direct API tests for those. This depends on having devs that design things properly, but most of our current frameworks, right, they actually have MVC or something like that in place, right? So API testing isn't that big of a deal. MSL really comes in to the next level of this. So last year at GTAC we heard about things like Karma and test runners for UI-specific code, and could we take the same approach to isolation where we had been isolating the back-end, could we isolate the front-end in the same way? So that's how we view this sort of evolution at our company. So how are we doing this with MSL? It's basically a tool for what we call client-side integration testing. That means to take your full client side code -- the JavaScript, the CSS, HTML and whatnot -- and to locally deploy this using a Node.js server. And this is going to let you, without having to build even in some cases, to start doing some tests against your UI features. All right? So obviously we're removing a lot of dependencies when we do this, right? So the MSL server has your app code, and it's basically going to use a test runner to open up a test. The test will have API calls back to the MSL server, and we'll show plenty of examples of that later in the talk. So basically the idea is a client-server kind of model where the server is doing some mocking for you to aid you in setting up and executing tests. Demo time. >>Daniel Koo: It's demo time. So, we're going to do a live demo because live demos are cool. And we're going to show this with the Wi-Fi off, just to prove that we are doing hermetic testing; right? Okay. So to install MSL server it's pretty simple. It's up on npm so all you need to do is globally install MSL server. If I can type. Now, I'm not going to run this because it's already installed, so.... It’s already installed. All right. So, now, the code that we are interested in is right here. This has all of the front-end code. It‘s got the JavaScripts, the HTMLs and CSS files that we’re trying to test. All right, so I'm going to fire off the MSL server and we do it by this. MSL, and we give the port number and there you go. Now you have the MSL. MSL has launched from here. It's running on this directory, on this port. So now let's open up the app. Localhost. Sample app. And there you go. So it's as simple as that. We have the app up and running. And what I'm going to do is I'm going to perform an interactive action which actually triggers a REST call but it's going to fail because, well, we have no REST services actually up and running, right? So you can see here we get a 404 and there's nothing coming back. So I'm going to show you how we can mock this so we can actually test the front-end code. Alright. So we're going to use Jasmine for running our tests. So we've got the Jasmine library here. And here is where we include our MSL client library. So we've got the mock API browser. So this is the JavaScript that allows you to talk back to the MSL server. We also have the app container-driver JavaScript which allows you to drive your interactive scenario. And of course, here is my Jasmine spec file. So now I'll show you the spec file. So we have two tests in here. And the first thing we do before running any of the tests, what we do is we call this function called openApp that's part of our app container-driver which opens the app that's running on the MSL server. So we make sure we, you know, reset and always have the app running in the same state. Next, going into the test. So what we're going to do is we're going to stage the mocks before we do our interactions. So we call this function setMockRespond, which basically tells the MSL server to intercept all the URLs, all the requests that match this URL and respond with this JSON response, and you can give the status code 200. You can also add delay time but in our case we're not going to add any delays. And then what we're going to do is we're going to use again our app container-driver to get the elements and perform some actions. And we've implemented this using jQuery because everything is running inside a browser and we didn't want to use Selenium or anything like that to drive the test from outside. So what we do is we type, and we click, and then we're going to validate that the dropdown that comes down, when we do a type ahead, is actually -- it contains the items that we have mocked in the JSON response. And then we're going to click on the second item, and we're going to see that it actually selected the second item and the type ahead is populated with that. The second test is going to test that the app fired off an XHR correctly. So we do that by calling this function, setInterceptXHR, which again tells the MSL server go intercept everything, all HTTP requests that matches this URL. And then again, we do the interactions, and then we call this getInterceptedXHR method which will retrieve back everything that's intercepted, and we're going to validate that the URL was correct and it was a GET method. All right. So let's fire off the test. It's going to be very fast so you guys have to watch carefully. All right. Tests done. Two tests done. And you guys saw the dropdown that came down; right? So that's what we have mocked. So now Bryan is going to talk about how you can use this in Karma using our plug-in. And before I hand it off, I'm going to turn off the server because our plug-in is actually going to start up the MSL server by itself. >>Bryan Robbins: Great. So with the MSL stuff we've shown so far in hand, right, you’re taking advantage of a framework like Jasmine. And what we would need to add to that is some kind of runner. Something to launch the browser, something to do reporting. As it turns out there's some pretty smart runners out there. We chose Karma and we decided to implement a small plug-in with Karma that will start up your MSL server and kill it when it's finished and those kind of things, and maybe some other integration points. So here's an example of how to hook it up. You know, our plug-in is available on Karma's default repos; right? So you can just add MSL as a framework. And when you go through your files, there's no need to include any files other than just your spec. So you saw on our Jasmine spec, we had to add all of our specific files there. You don't have to do that with the Karma plug-in. And then there's a block for MSL-specific configuration here. So this is saying you know, we want to run MSL on port 8002. This is going to be behind Karma's own port which is usually something like 9876 or something like that. The Karma port needs to match the port in your test. The MSL port is sort of hidden from all of that, and the way we accomplish that is via Karma's proxy support. So we're going to route all requests on the root URL to the mocking server; right? So this is the way we hook this up. We can foresee maybe extending some of the features of this plug-in to maybe set that proxy for you automatically or do something cool, but not quite there yet. So let’s quickly just run this. >> Daniel Koo: The browser is behind the window, but it worked. >>Bryan Robbins: Oh yeah. Sure. >>>Daniel Koo: Try running it >>Bryan Robbins: One more time? >> Daniel Koo: There you go. >>Bryan Robbins: There we go. So again, Karma is running the same test but with JUnit reporting and all the nice features that Karma gives you. You can also swap with other browsers and that sort of thing. So one last thing. So, we're out on GitHub as we mentioned. So we've got full docs there, as well as the user -- you took off the Wi-Fi. Yes, our GitHub page is not a hermetic page. >> Daniel Koo: This is not part of the testing. It's all right. [ Laughter ] >>Bryan Robbins: Does that count? So we're out on GitHub. We have full docs of all of our APIs there so we've written not just the JavaScript client but also Java and Node.js and Python and Ruby. Of course you can use something like WireMock if you're totally Java all the timeJUnit, but if you need something that would maybe switch back and forth and could leverage the same mocking ideas, we do have the different clients. And we need to give a shout-out to the other half of our team back in D.C. Jake and Chien have put a lot of work into this stuff for us. So that's the rest of our team. And with that, we're done. >>Daniel Koo: Thanks. [ Applause ]