London Tester Gathering – 18th June 2014

Last night I took the opportunity to have a night off.  This is very very rare for me.

 It was the monthly London Tester Gathering at the Shooting Star in Liverpool Street.    This is a meeting for anyone that loves testing.  Its a chance to meet other people and talk about how we do things and listen to a few presentations. Plus there is a free bar, this time sponsored by soasta.  However no cats were in attendance.


After some chit chat where I talked to a few different people about their approach to automation it was time for the talks.

 Recruiting testers

First up was Loan Todoran talking about how he recruitment 10 testers in 10 months.   He spend the first 5 mins telling everyone how to and how to not pronounce his name.   For anyone that every meets this guy its Yoan, not Yawn or Loan how it is spelt.

He spoke about how he searched for junior testers at graduate fairs his interview techniques, however the testing community questioned his approach and how he could support development of junior testers when recruiting so fast.

testing cats

Performance Metrics

Speed is important.  If this cheetah was slow it would never be able to catch its dinner.

2nd talk was from the sponsor Soasta, so was a bit of a sales pitch.  They explained how bad performing sites could cost the business money.    Users who experience bad load times are less likely to engage and become a registration or an application.

I thought about how many sites I have visited and just given up because its slow, and there’s a lot.   Perhaps its because I don’t have broadband and only use 3g/4g for all my internet needs at home.

However measuring metrics against performance can be subjective due to so many variations in the way people view websites and apps, plus expectations change.   It has been proved that Australians will allow for longer load times than the UK as their broadband isn’t quite as fast as here.   I wonder what New Zealand is?

 Soastas tool to achieve this is mPulse provides real-time, actionable intelligence | SOASTA  (However there are more and New Relic should give us some information)

In their case study they showed that as load time increased transactions on the site decreased and therefore users were less likely to engage.

They then used this to calculate how many lost sales this could potentially cause.

 We would need to look at current load times, where we want to be and how we do it.

 It is difficult to prove up front that increasing performance would result in however many more registrations.

It’s all about risk in the end.

Bad load time also affects Google Rankings so its pretty important, but here is some more information for those interested.

Poor Website Loading Times Irritate People, Matter to Search Engines, Plus Can Cost you Money – YouMoz – Moz



Liz goes to… The London next gen testing conference – 26th June 2014

Its the second edition of Liz goes to….


This time to a testing conference.

Its theme was to find out what the next generation of testers would be doing and how software development is changing.

This blog posts theme is linking, so I will try to provide links to interesting stuff where I can.


The one thing I took away from the day is that in order to keep up with competitors we need to react to change fast, deliver changes and understand the risks of those to help make decisions.    Making smaller more achievable goals that we can adapt to as we go will decrease the risk.  CI, automation (specifically unit testing), fast feedback from A/B testing are all part of this.


I have summarised the more interesting talks for now.

Dave Snowden – The Internet of Things

David Snowden has some pretty controversial views, so if you like controversy like I do google him.

Studying the way humans react to the internet and how quickly things change.  We adapt without even knowing!

He explained that having too much structure can mean we take longer to adapt to change.  Forcing specific requirements is great for efficiency but it gives no time to adapt and repurpose something that is already alive  We need to react fast to constantly evolve.  Its common now to try to replicate something you like, but as its the end point of a revolutionary process you need to understand the why before scaling the how.

Exaptation requires time for redundancy and efficiency, so being 100% efficient can cause problems as teams have no time to adapt and innovate.

The tools we use change our cognitive processes and we become dependent on them.  Don’t manage things you don’t need to manage.  e.g Internet Explorer 6! 

A recording of the talk is on his blog.


Stephen Janaway – Testing as an activity

Stephen explained that the future of testers is not just to be the tester, and is more about offering guidance to the team.

It is what we are trying to do at but it seems there are a lot of companies out there that still have a team of testers hidden in the basement.

Testing or Checking

Testing is an activity, a performance, something you do and that this can be any testing task, not just the actually checking it works.

Testing is evolutionary product learning

Checking is a performance.   You have a set of plans, and you run through those plans.  (Better for automated tests)

Checking is testing the back and white rules

The future testers role:

– collaborators

– Customer advocates

– Guiders

– Pairers

– Quality owned by teams

– Raising profile of the need for quality

– Testers invoke teams to own quality!

Slides   Testing As An Activity


Dan North presents…Dan North

Another one that if you haven’t watched him on You Tube is worth a watch.   Dans brief was to argue against all the points in the previous talks, plus to give his views about the future of the testing role.

Testing is the future!

Testing is to be more collaborative.   Collaboration brings individual blind spots to foreground – quality improves!

Dan disagreed that testing is an activity.  He described it as a mindset and its testers jobs to guide others how to use this mindset.

Taking risks

Getting fast feedback is about taking risks.   We can’t all test everything so we focus our efforts where they are needed the most

All risks have an impact and likelihood and the tendency is just to fix all the high things, but with risk we need to think about the context of it too.   Every risk will have a completely different context depending on the stakeholder. 

Taking risks will allow for faster feedback and therefor more innovation and exaptation.   The quicker we get feedback the quicker we can react to it!

Automation is a key in making these risks smaller as it provides fast feedback.  Building safely in with reduces risk.

Rather idealistically he described how a change was A/B tested without hours and refactored based on user feedback.

100% efficiency

He also agreed with Dave Snowden saying that ‘People with time invent things’   Becoming too focused on burndown charts and story points can actually be counter productive

Cognitive bias

Dan explained how testers need to use cognitive bias to aid questioning of requirements.  Its a tendency to only agree with things in your world view, especially after being given some really rigid requirements or being sold an idea and testers are really good at thinking of the things no one else can.

We are normally thinking about the selective reality as the what we class as unnecessary points are already removed.   This can lead to false assumptions, so its always good to question.

The ladder of inference is a good tool for this

Nothing pleases people more than to go on thinking what they have always thought, and at the same time imagine that they are thinking something new and daring: it combines the advantage of security and the delight of adventure’

T S Eliot

Net a Porter’s Mobile testing tips

Mobile testing is a complete minefield.  So many devices to choose from, so knowing business aim helps testing focus.  

Net a porter shared their mobile app testing tips:

  1. Date and time settings modify the app time –    we don’t want to be sending people reminders at 2am!
  2. Languages settings  
  3. keyboard settings  
  4. Location settings
  5. Network settings –   3g, wifi, wifi authentication (like on the tube)
  6. Native share
  7. Test Non retina devices
  8. Privacy settings – Don’t let the app use the camera, what would happen etc
  9. Storing data 
  10. Uninstalling

Fragmentation of devices is tricky, try to pick the most common ones.

Performance is super important – make sure this is in your definition of done!

If the app crashes or is slow, people will react badly to it.


Other bits

Automated Testing

I talked to a number of people about automated testing and its value.

Everyone agreed that unit testing is fundamental as they are quick to run and give us some level of confidence, but did question how valuable UI testing is as it can be incredibly time consuming to maintain, especially if you try to test more than basic functionality.

It took me back to the agile testing pyramid and the fact that we should be unit testing more.

Api testing

Net a porter spoke about their approach to API testing

They unit test all the core parameters and manually test using soap opera scenarios using a REST client.

Technical Testers

There is loads online at the moment about testers needing to be developers to stay in ‘agile’.   Today is so easy to learn online, so we talked that upskilling will have value but that testers are also useful elsewhere in an agile team.  The role of tester will become a lot more collaborative with other members of the team and guiding others about quality.

Dealing with sprint bugs

–  add unit tests for them!!