Tag: testing

Quick Test Automation with CasperJS

I find myself as the sole QA Engineer at a rapidly changing startup needing to automate integration tests.  There are legacy Capybara tests in our large Sinatra app, some javascript unit tests, and a bevy of test suites for the remainder of the stack. But, like any software product, its codebase grows faster than the test suite.

In looking for a quick way to get repeatable manual tests of the web application front end I settled on CasperJS.  Partly because of the speed of implementation, and the familiarity of Javascript and some robust features like great command line tools.

npm install casperjs --devel

And you’re on your way!

A great use i’ve found for casper scripts is to quickly hit all of the accessible dynamic FE page types we have and check for a 200 status to help regression test very quickly.

casper.test.begin('johanncodes.com returns an OK status', 1, function(test) {
    casper.start('https://johanncodes.com/', function() {
        test.assertHttpStatus(200);
    }).run(function() {
        test.done();
    });
});

Another great feature i’ve made use of is CasperJS’ great form fill and submission ability, which lets me easily work through tasks like user account creation, user profile changes, navigation through checkout flows and other fun things.

Even with very rudimentary javascript knowledge you can start to build very powerful tests and scripts almost immediately.  If you’re looking to automate very rote parts of your QA flow that have a binary success/fail state you can check for CasperJS is worth a look.

Check out the docs and installation instructions.

Happy testing!

How do you create a QA Process at a Startup?

QA Process header image

My last post was an overview, but this one will dive into some of the problems I encountered when I was first getting started as the de-facto quality stakeholder and dig into what REALLY mattered when it came to the actual process of QA. Fitting into the development cycle is critical.

Dissect the Workflow

There is no better way to learn the workflow than to follow a feature through the factory and out the door.  Make it a priority to see a feature from design to deploy as soon as possible.

Every team has idiosyncrasies that make experience a better teacher than a ReadMe.

When I was hired on my job description clearly included committing actual code so I also had the advantage of being able to make several early commits and deploys that far and away gave me a better idea than just being told. Do this if you can.

Once you know the workflow, agile or not, you can get a better picture of where QA fits.  Almost always at the end of a cycle, but often interspersed with development under certain conditions. If your team uses a kanban board, a flow chart, a spreadsheet, or project management software the next step is to carve out a QA section and OWN IT. This is your wheelhouse, and when a ticket or a story enters it it must bend to your will.

Remake the Workflow

If having a QA engineer/Tester/Test Engineer wasn’t part of the teams workflow, or if testing was less of a process that was owned and more of a vague responsibility placed on each developer… then the workflow will need to change. You’ll be the catalyst here.

In order for your work to be effective you’ll need to start asking for things to happen upstream, long before they land in your lap. One of the best ways is to get your hands into the ticket/story system and outline steps that need to happen as that ticket is passed along to a developer. If your lucky like me, buy in from your manager here will be easy and they’ll be eager to help make that happen.

One of the simplest process wins early on was to simply request that every dev write a few short summary of how the feature should work, how it shouldn’t work, and finally how they had tested it.

Now you know how the “Happy Path” was tested (or if it was tested at all..) and you can immediately compare this summary with the feature spec in the story and make sure nothing was missed.

Take Ownership

I was very skittish about walking into a new team, a new workflow, a new product, and a new codebase and then starting to make demands to make my life easier.

You need to suppress that doubting voice, ultimately you were hired to improve the product and build a process that makes more stability a foregone conclusion.

At a startup the only constant is change and you can rest easy knowing your changes will be the least of anyones concerns. As long as you explain the benefits of your decisions people are normally more than happy to hear the initiative being taken.

If you’re building a QA process for a startup then you are also the key stakeholder for Quality and by fiat you now have the opportunity to start instilling good habits. Don’t hesitate to advocate from a place of confidence if you know your changes will result in a higher quality end result.

When your team members start to reap the benefits of better quality any prior doubts will be washed away.

A Year of Startup QA

A picture of json that describes this post

This year marked the first full year of QA at the startup I work at, working as the first QA hire on an existing engineering team. There were a lot of challenges, but I wanted to share some of my take aways and challenges that were overcome.

Starting from Scratch

There was not a system in place for me to plug into when I arrived, testing was happening, deploys were happening, and code reviews were happening all somewhat organically and maybe haphazardly.

I had what amounted to a clean slate and worked to build up some systems from there.

My first step was to learn the product. The most important thing i’ve done since I started last year was to become a product expert. Knowing our product intimately continues to be one of the best values I bring to the table and allows me to be more targeted and succinct in testing.

As the product grew I also began to maintain documents outlining a lot of the more esoteric behavior, outside of repo ReadMes. This has saved my hair more than a few times.

My second step was to integrate myself into the Dev team workflow. Finding out where QA lives in the release lifecycle is important and defines a lot of the actual QA work that gets done. In our semi-agile team framework QA became the last step on the Kanban board. I like to think of that part of the board as ‘active development part II‘. In an ideal state i’m working hand in hand with the devs assigning and closing out bug tickets until we have a stable feature.

The third step was deciding on tools. We use Asana as our management software, ticketing system, and QA tracker. I’ve enjoyed using it as I’ve reinvented the QA ticket process several times because it affords a lot of flexibility in the organizational structure of your tasks (tickets).

I use Google Apps for spreadsheets, where I document feature behavior, or maintain checklists that are shared with third parties.

And I use CasperJS to automate a lot smoke testing that would otherwise eat a huge amount of my time.

I’m still searching for a good lightweight tool to organize functional test cases, an essential part of reducing the bus-factor of the QA team (1).

Wrap-Up

If I had to describe my year in one word it would be: Iterative

I haven’t built the best QA system, nor the best processes, but in the last year they’ve grown to better match the challenges we have everyday. The next year will be even (iteratively) better.

Testing http requests with VCR

Ruby Gem: VCR

VCR on Github

When working on Discuss-it the team found that testing our API http requests slowed down our testing in a big way and sometimes returned inconsistent results.  Since we didn’t want to test the reliability of our external dependencies VCR was the perfect tool to step in between our http request and our test.

What it does is create a ‘cassette’ file in your spec folder which records the first response to your http request and then on ‘replay’ when you rerun your tests it uses the cassette data instead.  If your tests are http request heavy you’ll be able to test how the requests are handled much more consistently with VCR.

 

 

 

Examine closely

There will be reading comprehension questions following this excerpt.

Project based testing versus question testing:

We’ve had both, but I think its more difficult to demonstrate mastery when you are just regurgitating memorized terms or not directly applying the concepts you are supposed to be demonstrating knowledge of.

An open ended project can quickly swallow you whole, but when you are given clear goals you can scale and plan a project that matches the scope of the guidelines.

I have definitely preferred the project based version of exams so far because they allow me to demonstrate to myself that I know the topics and have a work sample when i’m finished.  Whereas working on a much smaller code example won’t be as useful when its just sitting around by itself to show I can manage the syntax.  Project exams have the added benefit of grading doubling as a code review which means you get more valuable and applicable feedback.

In “Examine Closely” the author compared to different approaches to exams.  List the two methods and three key differences.

Discuss the effect of Plato’s Theory of Forms on testing in education.

Extra credit: In less than three tries and with no references spell the Author’s full name correctly.